Signal Processing Fundamentals

Signal Processing, Stochastic Processes · course

Browse all électronique et automatique documents

Prerequisites

Wiener Filter

Kalman Filter

Estimation, Wiener, Kalman

Signal Processing

Di GE

[email protected]

LTSI, b timent 22, Campus de Beaulieu

1 / 45

N.B. : A particular realization with discret time sampling can be noted

as

x

{

}n

2

Q: How to characterize a SP ?

R : By using the joint probability density function (pdf ) px(

x

{

)

}n

2

Prerequisites

Wiener Filter

Kalman Filter

Stochastic process and stationarity

Covariance matrix (for WSS process)

Power Spectral Density

Optimal Linear ltering

Stochastic process

Stochastic process (SP) = random process

All statistical phenomenon that evolves with time and governed by probabil-

ity laws.

~ Ex : voice, radar, noise, bio-physical signals, stock indexes, etc

[email protected]

LTSI, b timent 22, Campus de Beaulieu

2 / 45

Prerequisites

Wiener Filter

Kalman Filter

Stochastic process and stationarity

Covariance matrix (for WSS process)

Power Spectral Density

Optimal Linear ltering

Stochastic process

Stochastic process (SP) = random process

All statistical phenomenon that evolves with time and governed by probabil-

ity laws.

~ Ex : voice, radar, noise, bio-physical signals, stock indexes, etc

x

N.B. : A particular realization with discret time sampling can be noted

as

Q: How to characterize a SP ?

R : By using the joint probability density function (pdf ) px(

x

}n

{

2

)

{

}n

2

[email protected]

LTSI, b timent 22, Campus de Beaulieu

2 / 45

~ The reverse cannot ne deduced except when px is Gaussian.

~ Note that both mx and rx(n, n

k) depend on n in general.

~ We thus introduce the notion of "stationarity" in an attempt to

simplify these statistics

Prerequisites

Wiener Filter

Kalman Filter

Stochastic process and stationarity

Covariance matrix (for WSS process)

Power Spectral Density

Optimal Linear ltering

Stationarity (1/2)

In practice, joint pdf (px(

{

Solution : partial characterization up to 2nd order

)) are dicult to obtain.

x

}

2

n

x

px(

{

n

2

}

)

)

mx = ]

rx(n, n

k) = x ]

1st order, mean

2nd order, auto-covariance

[email protected]

LTSI, b timent 22, Campus de Beaulieu

3 / 45

~ We thus introduce the notion of "stationarity" in an attempt to

simplify these statistics

Prerequisites

Wiener Filter

Kalman Filter

Stochastic process and stationarity

Covariance matrix (for WSS process)

Power Spectral Density

Optimal Linear ltering

Stationarity (1/2)

In practice, joint pdf (px(

{

Solution : partial characterization up to 2nd order

)) are dicult to obtain.

x

}

2

n

x

px(

{

n

2

}

)

)

mx = ]

rx(n, n

k) = x ]

1st order, mean

2nd order, auto-covariance

~ The reverse cannot ne deduced except when px is Gaussian.

k) depend on n in general.

~ Note that both mx and rx(n, n

[email protected]

LTSI, b timent 22, Campus de Beaulieu

3 / 45

Prerequisites

Wiener Filter

Kalman Filter

Stochastic process and stationarity

Covariance matrix (for WSS process)

Power Spectral Density

Optimal Linear ltering

Stationarity (1/2)

In practice, joint pdf (px(

{

Solution : partial characterization up to 2nd order

)) are dicult to obtain.

x

}

2

n

x

px(

{

n

2

}

)

)

mx = ]

rx(n, n

k) = x ]

1st order, mean

2nd order, auto-covariance

~ The reverse cannot ne deduced except when px is Gaussian.

~ Note that both mx and rx(n, n

k) depend on n in general.

~ We thus introduce the notion of "stationarity" in an attempt to

simplify these statistics

[email protected]

LTSI, b timent 22, Campus de Beaulieu

3 / 45

wide-sense stationarity (WSS)

A stochastic processes for which the mean and auto-covariance with time :

mx = ]

rx(k) = x ]

(1)

~ Any SS process is also WSS.

Find the relation between rx(0) and 2

x for WSS process.

Prerequisites

Wiener Filter

Kalman Filter

Stochastic process and stationarity

Covariance matrix (for WSS process)

Power Spectral Density

Optimal Linear ltering

Stationarity (2/2)

strongly stationary process (SS)

A stochastic process is called "strongly stationary process" when its joint

probability distribution does not change when shifted in time :

for all

m, n, k

2

px( , x , . . . , x ]t) = px( , x , . . . , x ]t)

Prove that for strongly stationary process, mx and rx(n, n

do not depend on n

k)

[email protected]

LTSI, b timent 22, Campus de Beaulieu

4 / 45

Find the relation between rx(0) and 2

x for WSS process.

Prerequisites

Wiener Filter

Kalman Filter

Stochastic process and stationarity

Covariance matrix (for WSS process)

Power Spectral Density

Optimal Linear ltering

Stationarity (2/2)

strongly stationary process (SS)

A stochastic process is called "strongly stationary process" when its joint

probability distribution does not change when shifted in time :

for all

m, n, k

2

px( , x , . . . , x ]t) = px( , x , . . . , x ]t)

Prove that for strongly stationary process, mx and rx(n, n

do not depend on n

wide-sense stationarity (WSS)

k)

A stochastic processes for which the mean and auto-covariance with time :

mx = ]

rx(k) = x ]

(1)

~ Any SS process is also WSS.

[email protected]

LTSI, b timent 22, Campus de Beaulieu

4 / 45

Prerequisites

Wiener Filter

Kalman Filter

Stochastic process and stationarity

Covariance matrix (for WSS process)

Power Spectral Density

Optimal Linear ltering

Stationarity (2/2)

strongly stationary process (SS)

A stochastic process is called "strongly stationary process" when its joint

probability distribution does not change when shifted in time :

for all

m, n, k

2

px( , x , . . . , x ]t) = px( , x , . . . , x ]t)

Prove that for strongly stationary process, mx and rx(n, n

do not depend on n

wide-sense stationarity (WSS)

k)

A stochastic processes for which the mean and auto-covariance with time :

mx = ]

rx(k) = x ]

(1)

~ Any SS process is also WSS.

Find the relation between rx(0) and 2

x for WSS process.

[email protected]

LTSI, b timent 22, Campus de Beaulieu

4 / 45

Consider the following SP :

X(t) = 8

sin t

sin t

cos t

cos t

>><

>>:

with probability 1

with probability 1

4

with probability 1

with probability 1

4

4

4

Evaluate its mean and autocorrelation function to determine if its WSS,

then the pdf at X(0) and X(

4 ) to see if its SS.

Prerequisites

Wiener Filter

Kalman Filter

Stochastic process and stationarity

Covariance matrix (for WSS process)

Power Spectral Density

Optimal Linear ltering

Quiz

. Is an i.i.d process SS? WSS?

. Is a WSS process always SS ? nd a counterexample if not

. Is a normal WSS process always SS ? i.i.d ?

. Is random walk WSS ? Poisson process ?

[email protected]

LTSI, b timent 22, Campus de Beaulieu

5 / 45

Prerequisites

Wiener Filter

Kalman Filter

Stochastic process and stationarity

Covariance matrix (for WSS process)

Power Spectral Density

Optimal Linear ltering

Quiz

. Is an i.i.d process SS? WSS?

. Is a WSS process always SS ? nd a counterexample if not

. Is a normal WSS process always SS ? i.i.d ?

. Is random walk WSS ? Poisson process ?

Advertisement

Consider the following SP :

sin t

sin t

cos t

cos t

with probability 1

4

with probability 1

4

with probability 1

4

with probability 1

4

X(t) = 8

>><

>>:

Evaluate its mean and autocorrelation function to determine if its WSS,

then the pdf at X(0) and X(

4 ) to see if its SS.

[email protected]

LTSI, b timent 22, Campus de Beaulieu

5 / 45

Remark : if only cond (1)(2) are satised, the asymptotic covariance P

is a solution of the Lyapunov equation. (Th. de Lyapunov)

1

Prerequisites

Wiener Filter

Kalman Filter

Stochastic process and stationarity

Covariance matrix (for WSS process)

Power Spectral Density

Optimal Linear ltering

Advanced Quiz

The Kalman ltering formulation writes : for k = 0, 1, . . .

Xk+1 = FXk + GUk

Yk = HXk + Bk,

evolution of states

observations

1. H, F and G are deterministic and known matrices;

2. Uk and Bk : i.i.d SP, Uk

3. Xk, Yk: SP with dim(Xk) = M and dim(Yk) = N initialized by X0.

(0, R), Bk

(0, Q)

N

N

Show that :

Xk

and

Yk

are WSS if

{

}

{

}

1. F is a stable "lter", i.e.all eigenvalues lies within the unit circle;

2. zero initial mean

3. the initial covariance P0 = var is a solution of the Lyapunov

= 0

equation: P = FPFt + GQGt.

[email protected]

LTSI, b timent 22, Campus de Beaulieu

6 / 45

Prerequisites

Wiener Filter

Kalman Filter

Stochastic process and stationarity

Covariance matrix (for WSS process)

Power Spectral Density

Optimal Linear ltering

Advanced Quiz

The Kalman ltering formulation writes : for k = 0, 1, . . .

Xk+1 = FXk + GUk

Yk = HXk + Bk,

evolution of states

observations

1. H, F and G are deterministic and known matrices;

2. Uk and Bk : i.i.d SP, Uk

3. Xk, Yk: SP with dim(Xk) = M and dim(Yk) = N initialized by X0.

(0, R), Bk

(0, Q)

N

N

Show that :

Xk

and

Yk

are WSS if

{

}

{

}

1. F is a stable "lter", i.e.all eigenvalues lies within the unit circle;

2. zero initial mean

3. the initial covariance P0 = var is a solution of the Lyapunov

= 0

equation: P = FPFt + GQGt.

Remark : if only cond (1)(2) are satised, the asymptotic covariance P

is a solution of the Lyapunov equation. (Th. de Lyapunov)

1

[email protected]

LTSI, b timent 22, Campus de Beaulieu

6 / 45

Prerequisites

Wiener Filter

Kalman Filter

Stochastic process and stationarity

Covariance matrix (for WSS process)

Power Spectral Density

Optimal Linear ltering

Covariance matrix and covariance function

Construction

Let xn = , . . . , x ]t denote the L-element observation of the

Rx =

rx(0)

rx(

1)

...

L + 1)

rx(

rx(1)

rx(0)

...

L + 2)

rx(

. . .

. . .

...

. . .

1)

2)

rx(L

rx(L

...

rx(0)

3

7

7

7

5

= 2

6

6

6

4

H denotes the conjugate (Hermitian) transpose and rx(k) the auto-

where

covariance function

~ The covariance matrix plays a key role in signal processing of WSS

process.

[email protected]

LTSI, b timent 22, Campus de Beaulieu

7 / 45

Toeplitz

Positive semidenite

Ri,j = Ri+1,j+1

R

0

,

zHRz

0,

z

8

or equivalently all eigenvalues are non-negative.

Prerequisites

Wiener Filter

Kalman Filter

Stochastic process and stationarity

Covariance matrix (for WSS process)

Power Spectral Density

Optimal Linear ltering

Properties of the covariance matrix R

Hermitian (or self-adjoint matrix) since rx(k) = rx(

k)

R = RH

for complex valued x, and symmetric R = Rt for real valued x.

[email protected]

LTSI, b timent 22, Campus de Beaulieu

8 / 45

Positive semidenite

R

0

,

zHRz

0,

z

8

or equivalently all eigenvalues are non-negative.

Prerequisites

Wiener Filter

Kalman Filter

Stochastic process and stationarity

Covariance matrix (for WSS process)

Power Spectral Density

Optimal Linear ltering

Properties of the covariance matrix R

Hermitian (or self-adjoint matrix) since rx(k) = rx(

k)

R = RH

for complex valued x, and symmetric R = Rt for real valued x.

Toeplitz

Ri,j = Ri+1,j+1

[email protected]

LTSI, b timent 22, Campus de Beaulieu

8 / 45

Prerequisites

Wiener Filter

Kalman Filter

Stochastic process and stationarity

Covariance matrix (for WSS process)

Power Spectral Density

Optimal Linear ltering

Properties of the covariance matrix R

Hermitian (or self-adjoint matrix) since rx(k) = rx(

k)

R = RH

for complex valued x, and symmetric R = Rt for real valued x.

Toeplitz

Positive semidenite

Ri,j = Ri+1,j+1

R

0

,

zHRz

0,

z

8

or equivalently all eigenvalues are non-negative.

[email protected]

LTSI, b timent 22, Campus de Beaulieu

8 / 45

. PSD x(f ) are periodic and of period 1;

. PSD x(f ) are always real-valued : x(f ) = x(f );

. PSD are even: x(

f ) = x(f ) for real x ;

. PSD are non-negative : x(f )

0 for all f ;

. rx[0] = [

x

2] =

x(f )df = m2

x + 2

x;

|

|

1

2

1

2

R

Show that :

Prerequisites

Wiener Filter

Kalman Filter

Stochastic process and stationarity

Covariance matrix (for WSS process)

Power Spectral Density

Optimal Linear ltering

Denition

Einstein-Wiener-Khintchine relation (discrete SP)

The power spectral density (PSD) is the frequency domain equivalent of the

autocorrelation function in the temporal domain :

x(f ) =

(rx) =

F

rx exp(

j2 f k),

f

1

2

,

1

2

2

(2)

rx =

F

1(x) =

x(f ) exp(

j2 f k)df,

k = 0,

1, . . .

(3)

+

1

Xk=

1

1

2

1

2

Z

[email protected]

LTSI, b timent 22, Campus de Beaulieu

9 / 45

. PSD x(f ) are always real-valued : x(f ) = x(f );

. PSD are even: x(

f ) = x(f ) for real x ;

. PSD are non-negative : x(f )

0 for all f ;

. rx[0] = [

x

Advertisement

2] =

x(f )df = m2

x + 2

x;

|

|

1

2

1

2

R

Prerequisites

Wiener Filter

Kalman Filter

Stochastic process and stationarity

Covariance matrix (for WSS process)

Power Spectral Density

Optimal Linear ltering

Denition

Einstein-Wiener-Khintchine relation (discrete SP)

The power spectral density (PSD) is the frequency domain equivalent of the

autocorrelation function in the temporal domain :

x(f ) =

(rx) =

F

rx exp(

j2 f k),

f

1

2

,

1

2

2

(2)

1(x) =

x(f ) exp(

j2 f k)df,

k = 0,

1, . . .

(3)

+

1

Xk=

1

1

2

1

2

Z

rx =

F

Show that :

. PSD x(f ) are periodic and of period 1;

[email protected]

LTSI, b timent 22, Campus de Beaulieu

9 / 45

. PSD are even: x(

f ) = x(f ) for real x ;

. PSD are non-negative : x(f )

0 for all f ;

. rx[0] = [

x

2] =

x(f )df = m2

x + 2

x;

|

|

1

2

1

2

R

Prerequisites

Wiener Filter

Kalman Filter

Stochastic process and stationarity

Covariance matrix (for WSS process)

Power Spectral Density

Optimal Linear ltering

Denition

Einstein-Wiener-Khintchine relation (discrete SP)

The power spectral density (PSD) is the frequency domain equivalent of the

autocorrelation function in the temporal domain :

x(f ) =

(rx) =

F

rx exp(

j2 f k),

f

1

2

,

1

2

2

(2)

1(x) =

x(f ) exp(

j2 f k)df,

k = 0,

1, . . .

(3)

+

1

Xk=

1

1

2

1

2

Z

rx =

F

Show that :

. PSD x(f ) are periodic and of period 1;

. PSD x(f ) are always real-valued : x(f ) = x(f );

[email protected]

LTSI, b timent 22, Campus de Beaulieu

9 / 45

. PSD are non-negative : x(f )

0 for all f ;

. rx[0] = [

x

2] =

x(f )df = m2

x + 2

x;

|

|

1

2

1

2

R

Prerequisites

Wiener Filter

Kalman Filter

Stochastic process and stationarity

Covariance matrix (for WSS process)

Power Spectral Density

Optimal Linear ltering

Denition

Einstein-Wiener-Khintchine relation (discrete SP)

The power spectral density (PSD) is the frequency domain equivalent of the

autocorrelation function in the temporal domain :

x(f ) =

(rx) =

F

rx exp(

j2 f k),

f

1

2

,

1

2

2

(2)

1(x) =

x(f ) exp(

j2 f k)df,

k = 0,

1, . . .

(3)

+

1

Xk=

1

1

2

1

2

Z

rx =

F

Show that :

. PSD x(f ) are periodic and of period 1;

. PSD x(f ) are always real-valued : x(f ) = x(f );

. PSD are even: x(

f ) = x(f ) for real x ;

[email protected]

LTSI, b timent 22, Campus de Beaulieu

9 / 45

. rx[0] = [

x

2] =

x(f )df = m2

x + 2

x;

|

|

1

2

1

2

R

Prerequisites

Wiener Filter

Kalman Filter

Stochastic process and stationarity

Covariance matrix (for WSS process)

Power Spectral Density

Optimal Linear ltering

Denition

Einstein-Wiener-Khintchine relation (discrete SP)

The power spectral density (PSD) is the frequency domain equivalent of the

autocorrelation function in the temporal domain :

x(f ) =

(rx) =

F

rx exp(

j2 f k),

f

1

2

,

1

2

2

(2)

1(x) =

x(f ) exp(

j2 f k)df,

k = 0,

1, . . .

(3)

+

1

Xk=

1

1

2

1

2

Z

rx =

F

Show that :

. PSD x(f ) are periodic and of period 1;

. PSD x(f ) are always real-valued : x(f ) = x(f );

. PSD are even: x(

. PSD are non-negative : x(f )

f ) = x(f ) for real x ;

0 for all f ;

[email protected]

LTSI, b timent 22, Campus de Beaulieu

9 / 45

Prerequisites

Wiener Filter

Kalman Filter

Stochastic process and stationarity

Covariance matrix (for WSS process)

Power Spectral Density

Optimal Linear ltering

Denition

Einstein-Wiener-Khintchine relation (discrete SP)

The power spectral density (PSD) is the frequency domain equivalent of the

autocorrelation function in the temporal domain :

x(f ) =

(rx) =

F

rx exp(

j2 f k),

f

1

2

,

1

2

2

(2)

1(x) =

x(f ) exp(

j2 f k)df,

k = 0,

1, . . .

(3)

+

1

Advertisement

Xk=

1

1

2

1

2

Z

rx =

F

Show that :

. PSD x(f ) are periodic and of period 1;

. PSD x(f ) are always real-valued : x(f ) = x(f );

. PSD are even: x(

. PSD are non-negative : x(f )

f ) = x(f ) for real x ;

0 for all f ;

. rx[0] = [

2] =

x

|

|

[email protected]

1

2

1

2

R

x(f )df = m2

x + 2

x;

LTSI, b timent 22, Campus de Beaulieu

9 / 45

Prerequisites

Wiener Filter

Kalman Filter

Stochastic process and stationarity

Covariance matrix (for WSS process)

Power Spectral Density

Optimal Linear ltering

Example of PSD functions 1/2

1.

2.

Examples

RX( ) = e

|

|

SX(f ) =

2

2 + (2f )2

f

RX( ) =

2

2

cos

SX(f )

2

4

f

2

2

4

2

EE 278B: Stationary Random Processes

7 15

[email protected]

LTSI, b timent 22, Campus de Beaulieu

10 / 45

3.

RX(n) = 2|

n

|

SX(f ) =

5

4 cos 2f

3

f

4

3

2

1

1 2 3 4

1

2

1

2

4. Discrete time white noise process: X1, X2, . . . , Xn, . . . zero mean, uncorrelated,

with average power N

RX(n) =

N n = 0

0

otherwise

SX(f )

N

N

If Xn is also a GRP, then we obtain a discrete time WGN process

1

2

+1

2

f

n

n

EE 278B: Stationary Random Processes

7 16

1.

2.

Examples

RX( ) = e

|

|

SX(f ) =

2

2 + (2f )2

RX( ) =

cos

2

2

f

f

SX(f )

2

4

2

2

4

2

Prerequisites

Wiener Filter

Kalman Filter

Stochastic process and stationarity

Covariance matrix (for WSS process)

Power Spectral Density

Optimal Linear ltering

EE 278B: Stationary Random Processes

Example of PSD functions 2/2

7 15

3.

RX(n) = 2|

n

|

SX(f ) =

3

4 cos 2f

5

4

3

2

1

1 2 3 4

n

1

2

1

2

f

4. Discrete time white noise process: X1, X2, . . . , Xn, . . . zero mean, uncorrelated,

with average power N

RX(n) =

N n = 0

0

otherwise

SX(f )

N

N

n

1

2

+1

2

f

If Xn is also a GRP, then we obtain a discrete time WGN process

[email protected]

EE 278B: Stationary Random Processes

LTSI, b timent 22, Campus de Beaulieu

7 16

11 / 45

Sol:

1. use the z-Transform to get X(z) = z

1X(z) + z

2X(z) + z

1

2. or X(z) =

z

z1

1 = A

z

+ B

z

z2

= A z

1

1

z

1 + B z

1

z

1

1

3. Apply inverse z-Transform, x = A n

1u + B( )n

1u

using

(x ) = z

kX(z)

Z

Prerequisites

Wiener Filter

Kalman Filter

Stochastic process and stationarity

Covariance matrix (for WSS process)

Power Spectral Density

Optimal Linear ltering

Dierence Eq and z-Transform (1/2)

Denition of z-Transform on discrete time series

X(z) =

(x ) =

Z

+

1

x z

n

(4)

n=

X

1

Example of solving a dierence equation using z-transform

Establish x for x = x = 0 for all n negative.

1] + x +

[email protected]

LTSI, b timent 22, Campus de Beaulieu

12 / 45

Prerequisites

Wiener Filter

Kalman Filter

Stochastic process and stationarity

Covariance matrix (for WSS process)

Power Spectral Density

Optimal Linear ltering

Dierence Eq and z-Transform (1/2)

Denition of z-Transform on discrete time series

X(z) =

(x ) =

Z

+

1

x z

n

(4)

n=

X

1

Example of solving a dierence equation using z-transform

Establish x for x = x = 0 for all n negative.

1] + x +

Sol:

1. use the z-Transform to get X(z) = z

1X(z) + z

2X(z) + z

1

2. or X(z) =

3. Apply inverse z-Transform, x = A n

+ B

1 = A

z

z1

z2

z

z

= A z

1

z

1u ) = z

kX(z)

Advertisement

1

1 + B z

1

z

1

1] + B( )n

1u

[email protected]

LTSI, b timent 22, Campus de Beaulieu

12 / 45

Prerequisites

Wiener Filter

Kalman Filter

Stochastic process and stationarity

Covariance matrix (for WSS process)

Power Spectral Density

Optimal Linear ltering

Recurrence Eq and z-Transform (2/2) : Table

[email protected]

LTSI, b timent 22, Campus de Beaulieu

13 / 45

Apply the z-Transform on both side, we have the transfer function H(z) for

the LTI lter:

Y (z)

X(z)

=

b0 + b1z

1 + . . . + bnb z

nb

a0 + a1z

1 + . . . + ana z

na

= H(z)

(6)

We can always normalize a0 = 1 and dene an LTI lter by b =

and a = . As in matlab/octave, the

y = filter(b,a,x)

command lters the input data x with H(z) dened by b and a:

If a0

= 1, then b and a are normalized by a0. Therefore, a0 must be nonzero !

Prerequisites

Wiener Filter

Kalman Filter

Stochastic process and stationarity

Covariance matrix (for WSS process)

Power Spectral Density

Optimal Linear ltering

LTI systems (lter)

Consider the ARMA system (as an LTI lter):

na

nb

which handles both FIR and IIR lters.

Xk=0

Xk=0

y =

x

(5)

[email protected]

LTSI, b timent 22, Campus de Beaulieu

14 / 45

6

command lters the input data x with H(z) dened by b and a:

As in matlab/octave, the

y = filter(b,a,x)

If a0

= 1, then b and a are normalized by a0. Therefore, a0 must be nonzero !

Prerequisites

Wiener Filter

Kalman Filter

Stochastic process and stationarity

Covariance matrix (for WSS process)

Power Spectral Density

Optimal Linear ltering

LTI systems (lter)

Consider the ARMA system (as an LTI lter):

na

nb

y =

x

(5)

Xk=0

Xk=0

which handles both FIR and IIR lters.

Apply the z-Transform on both side, we have the transfer function H(z) for

the LTI lter:

Y (z)

X(z)

=

b0 + b1z

a0 + a1z

1 + . . . + bnb z

1 + . . . + ana z

nb

= H(z)

na

(6)

We can always normalize a0 = 1 and dene an LTI lter by b =

and a = .

[email protected]

LTSI, b timent 22, Campus de Beaulieu

14 / 45

6

Prerequisites

Wiener Filter

Kalman Filter

Stochastic process and stationarity

Covariance matrix (for WSS process)

Power Spectral Density

Optimal Linear ltering

LTI systems (lter)

Consider the ARMA system (as an LTI lter):

na

nb

y =

x

(5)

Xk=0

Xk=0

which handles both FIR and IIR lters.

Apply the z-Transform on both side, we have the transfer function H(z) for

the LTI lter:

Y (z)

X(z)

=

b0 + b1z

a0 + a1z

1 + . . . + bnb z

1 + . . . + ana z

nb

= H(z)

na

(6)

We can always normalize a0 = 1 and dene an LTI lter by b =

and a = . As in matlab/octave, the

command lters the input data x with H(z) dened by b and a:

y = filter(b,a,x)

If a0

= 1, then b and a are normalized by a0. Therefore, a0 must be nonzero !

[email protected]

LTSI, b timent 22, Campus de Beaulieu

14 / 45

6

Prerequisites

Wiener Filter

Kalman Filter

Stochastic process and stationarity

Covariance matrix (for WSS process)

Power Spectral Density

Optimal Linear ltering

Frequency response and lter design 1/3

In matlab/octave, the = freqz(b,a,n) command is used to

calculate the n-point frequency response of LTI lter (H(z) dened by b

and a).

Example of frequency response

Compute and display the magnitude response of 3-rd order IIR lowpass lter:

H(z) =

Sol:

0.05(1 + z

0.7z

(1

1)(1

1)(1

1.2z

1.5z

2)

1 + z

2)

1 + 0.9z

1. b= .5*conv([1 1], [1 -1.2 1]); % for the b vector

2. a= conv([1 -.7], [1 -1.5 .9]); % for the a vector

3. = freqz(b,a,2001); % 2001-point frequency response in h

and corresponding frequency in w

4. plot(w/pi/2,20*log10(abs(h))); % normalize w by 2 , and

convert h to dB

[email protected]

LTSI, b timent 22, Campus de Beaulieu

15 / 45

Prerequisites

Wiener Filter

Kalman Filter

Stochastic process and stationarity

Covariance matrix (for WSS process)

Power Spectral Density

Optimal Linear ltering

Frequency response and lter design 2/3

Graphical results:

Bonus Questions:

1. how do we recognize the LP lter and calculate its cut-o frequency

3dB compared to H(e

j2 0) ?

2. how to trace the phase response using h (degrees vs normalized

(

frequency) ?

Note : freqz(b,a) plots magnitude and phase response with no output

[email protected]

LTSI, b timent 22, Campus de Beaulieu

16 / 45

Prerequisites

Wiener Filter

Kalman Filter

Stochastic process and stationarity

Covariance matrix (for WSS process)

Power Spectral Density

Optimal Linear ltering

Frequency response and lter design 3/3

Filter design specications (example with LP lter)

Digital lter design consists of specifying

1. passband, stopband frequencies and attenuation (transition width);

2. maximum passband and stopband ripples (in dB);

3. lter order (or length of the impulse response for FIR).

[email protected]

LTSI, b timent 22, Campus de Beaulieu

17 / 45

1. Wp = 40/500; Ws = 150/500; Rp=3; Rs = 60; % normalized

passband/stopband frequency, passband ripple and stopband

attenuation (in dB)

2. = cheb1ord(Wp,Ws,Rp,Rs); % returns the lowest order n

and normalized passband edge frequency Wp

3. = cheby1(n,Rp,Wp, low); % returns the transfer

function of an nth-order lowpass Chebyshev Type I filter

4. freqz(b,a,512,1000); % trace a 512-point frequency response

with Fs = 1000

Sol:

Prerequisites

Wiener Filter

Kalman Filter

Stochastic process and stationarity

Covariance matrix (for WSS process)

Power Spectral Density

Optimal Linear ltering

Example of Chebyshev Type I lter design 1/2

Design a lowpass Chebyshev Type I lter with less than 3 dB passband

([0, 40]Hz) ripple and at least 60dB stopband (> 150Hz) attenuation. Fs =

1000 Hz

[email protected]

LTSI, b timent 22, Campus de Beaulieu

18 / 45

2. = cheb1ord(Wp,Ws,Rp,Rs); % returns the lowest order n

and normalized passband edge frequency Wp

3. = cheby1(n,Rp,Wp, low); % returns the transfer

function of an nth-order lowpass Chebyshev Type I filter

4. freqz(b,a,512,1000); % trace a 512-point frequency response

with Fs = 1000

Prerequisites

Wiener Filter

Kalman Filter

Stochastic process and stationarity

Covariance matrix (for WSS process)

Power Spectral Density

Optimal Linear ltering

Example of Chebyshev Type I lter design 1/2

Design a lowpass Chebyshev Type I lter with less than 3 dB passband

([0, 40]Hz) ripple and at least 60dB stopband (> 150Hz) attenuation. Fs =

1000 Hz

Sol:

1. Wp = 40/500; Ws = 150/500; Rp=3; Rs = 60; % normalized

passband/stopband frequency, passband ripple and stopband

attenuation (in dB)

[email protected]

LTSI, b timent 22, Campus de Beaulieu

18 / 45

3. = cheby1(n,Rp,Wp, low); % returns the transfer

function of an nth-order lowpass Chebyshev Type I filter

4. freqz(b,a,512,1000); % trace a 512-point frequency response

with Fs = 1000

Prerequisites

Wiener Filter

Kalman Filter

Stochastic process and stationarity

Covariance matrix (for WSS process)

Power Spectral Density

Optimal Linear ltering

Example of Chebyshev Type I lter design 1/2

Design a lowpass Chebyshev Type I lter with less than 3 dB passband

([0, 40]Hz) ripple and at least 60dB stopband (> 150Hz) attenuation. Fs =

1000 Hz

Sol:

1. Wp = 40/500; Ws = 150/500; Rp=3; Rs = 60; % normalized

passband/stopband frequency, passband ripple and stopband

attenuation (in dB)

2. = cheb1ord(Wp,Ws,Rp,Rs); % returns the lowest order n

and normalized passband edge frequency Wp

[email protected]

LTSI, b timent 22, Campus de Beaulieu

18 / 45

4. freqz(b,a,512,1000); % trace a 512-point frequency response

with Fs = 1000

Prerequisites

Wiener Filter

Kalman Filter

Stochastic process and stationarity

Covariance matrix (for WSS process)

Power Spectral Density

Optimal Linear ltering

Example of Chebyshev Type I lter design 1/2

Design a lowpass Chebyshev Type I lter with less than 3 dB passband

([0, 40]Hz) ripple and at least 60dB stopband (> 150Hz) attenuation. Fs =

1000 Hz

Sol:

1. Wp = 40/500; Ws = 150/500; Rp=3; Rs = 60; % normalized

passband/stopband frequency, passband ripple and stopband

attenuation (in dB)

2. = cheb1ord(Wp,Ws,Rp,Rs); % returns the lowest order n

and normalized passband edge frequency Wp

3. = cheby1(n,Rp,Wp, low); % returns the transfer

function of an nth-order lowpass Chebyshev Type I filter

[email protected]

LTSI, b timent 22, Campus de Beaulieu

18 / 45

Prerequisites

Wiener Filter

Kalman Filter

Stochastic process and stationarity

Covariance matrix (for WSS process)

Power Spectral Density

Optimal Linear ltering

Example of Chebyshev Type I lter design 1/2

Design a lowpass Chebyshev Type I lter with less than 3 dB passband

([0, 40]Hz) ripple and at least 60dB stopband (> 150Hz) attenuation. Fs =

1000 Hz

Sol:

1. Wp = 40/500; Ws = 150/500; Rp=3; Rs = 60; % normalized

passband/stopband frequency, passband rip...