Prerequisites
Wiener Filter
Kalman Filter
Estimation, Wiener, Kalman
Signal Processing
Di GE
LTSI, b timent 22, Campus de Beaulieu
1 / 45
N.B. : A particular realization with discret time sampling can be noted
as
x
{
}n
2
Q: How to characterize a SP ?
R : By using the joint probability density function (pdf ) px(
x
{
)
}n
2
Prerequisites
Wiener Filter
Kalman Filter
Stochastic process and stationarity
Covariance matrix (for WSS process)
Power Spectral Density
Optimal Linear ltering
Stochastic process
Stochastic process (SP) = random process
All statistical phenomenon that evolves with time and governed by probabil-
ity laws.
~ Ex : voice, radar, noise, bio-physical signals, stock indexes, etc
LTSI, b timent 22, Campus de Beaulieu
2 / 45
Prerequisites
Wiener Filter
Kalman Filter
Stochastic process and stationarity
Covariance matrix (for WSS process)
Power Spectral Density
Optimal Linear ltering
Stochastic process
Stochastic process (SP) = random process
All statistical phenomenon that evolves with time and governed by probabil-
ity laws.
~ Ex : voice, radar, noise, bio-physical signals, stock indexes, etc
x
N.B. : A particular realization with discret time sampling can be noted
as
Q: How to characterize a SP ?
R : By using the joint probability density function (pdf ) px(
x
}n
{
2
)
{
}n
2
LTSI, b timent 22, Campus de Beaulieu
2 / 45
~ The reverse cannot ne deduced except when px is Gaussian.
~ Note that both mx and rx(n, n
k) depend on n in general.
~ We thus introduce the notion of "stationarity" in an attempt to
simplify these statistics
Prerequisites
Wiener Filter
Kalman Filter
Stochastic process and stationarity
Covariance matrix (for WSS process)
Power Spectral Density
Optimal Linear ltering
Stationarity (1/2)
In practice, joint pdf (px(
{
Solution : partial characterization up to 2nd order
)) are dicult to obtain.
x
}
2
n
x
px(
{
n
2
}
)
)
mx = ]
rx(n, n
k) = x ]
1st order, mean
2nd order, auto-covariance
LTSI, b timent 22, Campus de Beaulieu
3 / 45
~ We thus introduce the notion of "stationarity" in an attempt to
simplify these statistics
Prerequisites
Wiener Filter
Kalman Filter
Stochastic process and stationarity
Covariance matrix (for WSS process)
Power Spectral Density
Optimal Linear ltering
Stationarity (1/2)
In practice, joint pdf (px(
{
Solution : partial characterization up to 2nd order
)) are dicult to obtain.
x
}
2
n
x
px(
{
n
2
}
)
)
mx = ]
rx(n, n
k) = x ]
1st order, mean
2nd order, auto-covariance
~ The reverse cannot ne deduced except when px is Gaussian.
k) depend on n in general.
~ Note that both mx and rx(n, n
LTSI, b timent 22, Campus de Beaulieu
3 / 45
Prerequisites
Wiener Filter
Kalman Filter
Stochastic process and stationarity
Covariance matrix (for WSS process)
Power Spectral Density
Optimal Linear ltering
Stationarity (1/2)
In practice, joint pdf (px(
{
Solution : partial characterization up to 2nd order
)) are dicult to obtain.
x
}
2
n
x
px(
{
n
2
}
)
)
mx = ]
rx(n, n
k) = x ]
1st order, mean
2nd order, auto-covariance
~ The reverse cannot ne deduced except when px is Gaussian.
~ Note that both mx and rx(n, n
k) depend on n in general.
~ We thus introduce the notion of "stationarity" in an attempt to
simplify these statistics
LTSI, b timent 22, Campus de Beaulieu
3 / 45
wide-sense stationarity (WSS)
A stochastic processes for which the mean and auto-covariance with time :
mx = ]
rx(k) = x ]
(1)
~ Any SS process is also WSS.
Find the relation between rx(0) and 2
x for WSS process.
Prerequisites
Wiener Filter
Kalman Filter
Stochastic process and stationarity
Covariance matrix (for WSS process)
Power Spectral Density
Optimal Linear ltering
Stationarity (2/2)
strongly stationary process (SS)
A stochastic process is called "strongly stationary process" when its joint
probability distribution does not change when shifted in time :
for all
m, n, k
2
px( , x , . . . , x ]t) = px( , x , . . . , x ]t)
Prove that for strongly stationary process, mx and rx(n, n
do not depend on n
k)
LTSI, b timent 22, Campus de Beaulieu
4 / 45
Find the relation between rx(0) and 2
x for WSS process.
Prerequisites
Wiener Filter
Kalman Filter
Stochastic process and stationarity
Covariance matrix (for WSS process)
Power Spectral Density
Optimal Linear ltering
Stationarity (2/2)
strongly stationary process (SS)
A stochastic process is called "strongly stationary process" when its joint
probability distribution does not change when shifted in time :
for all
m, n, k
2
px( , x , . . . , x ]t) = px( , x , . . . , x ]t)
Prove that for strongly stationary process, mx and rx(n, n
do not depend on n
wide-sense stationarity (WSS)
k)
A stochastic processes for which the mean and auto-covariance with time :
mx = ]
rx(k) = x ]
(1)
~ Any SS process is also WSS.
LTSI, b timent 22, Campus de Beaulieu
4 / 45
Prerequisites
Wiener Filter
Kalman Filter
Stochastic process and stationarity
Covariance matrix (for WSS process)
Power Spectral Density
Optimal Linear ltering
Stationarity (2/2)
strongly stationary process (SS)
A stochastic process is called "strongly stationary process" when its joint
probability distribution does not change when shifted in time :
for all
m, n, k
2
px( , x , . . . , x ]t) = px( , x , . . . , x ]t)
Prove that for strongly stationary process, mx and rx(n, n
do not depend on n
wide-sense stationarity (WSS)
k)
A stochastic processes for which the mean and auto-covariance with time :
mx = ]
rx(k) = x ]
(1)
~ Any SS process is also WSS.
Find the relation between rx(0) and 2
x for WSS process.
LTSI, b timent 22, Campus de Beaulieu
4 / 45
Consider the following SP :
X(t) = 8
sin t
sin t
cos t
cos t
>><
>>:
with probability 1
with probability 1
4
with probability 1
with probability 1
4
4
4
Evaluate its mean and autocorrelation function to determine if its WSS,
then the pdf at X(0) and X(
4 ) to see if its SS.
Prerequisites
Wiener Filter
Kalman Filter
Stochastic process and stationarity
Covariance matrix (for WSS process)
Power Spectral Density
Optimal Linear ltering
Quiz
. Is an i.i.d process SS? WSS?
. Is a WSS process always SS ? nd a counterexample if not
. Is a normal WSS process always SS ? i.i.d ?
. Is random walk WSS ? Poisson process ?
LTSI, b timent 22, Campus de Beaulieu
5 / 45
Prerequisites
Wiener Filter
Kalman Filter
Stochastic process and stationarity
Covariance matrix (for WSS process)
Power Spectral Density
Optimal Linear ltering
Quiz
. Is an i.i.d process SS? WSS?
. Is a WSS process always SS ? nd a counterexample if not
. Is a normal WSS process always SS ? i.i.d ?
. Is random walk WSS ? Poisson process ?
Publicité
Consider the following SP :
sin t
sin t
cos t
cos t
with probability 1
4
with probability 1
4
with probability 1
4
with probability 1
4
X(t) = 8
>><
>>:
Evaluate its mean and autocorrelation function to determine if its WSS,
then the pdf at X(0) and X(
4 ) to see if its SS.
LTSI, b timent 22, Campus de Beaulieu
5 / 45
Remark : if only cond (1)(2) are satised, the asymptotic covariance P
is a solution of the Lyapunov equation. (Th. de Lyapunov)
1
Prerequisites
Wiener Filter
Kalman Filter
Stochastic process and stationarity
Covariance matrix (for WSS process)
Power Spectral Density
Optimal Linear ltering
Advanced Quiz
The Kalman ltering formulation writes : for k = 0, 1, . . .
Xk+1 = FXk + GUk
Yk = HXk + Bk,
evolution of states
observations
1. H, F and G are deterministic and known matrices;
2. Uk and Bk : i.i.d SP, Uk
3. Xk, Yk: SP with dim(Xk) = M and dim(Yk) = N initialized by X0.
(0, R), Bk
(0, Q)
N
N
Show that :
Xk
and
Yk
are WSS if
{
}
{
}
1. F is a stable "lter", i.e.all eigenvalues lies within the unit circle;
2. zero initial mean
3. the initial covariance P0 = var is a solution of the Lyapunov
= 0
equation: P = FPFt + GQGt.
LTSI, b timent 22, Campus de Beaulieu
6 / 45
Prerequisites
Wiener Filter
Kalman Filter
Stochastic process and stationarity
Covariance matrix (for WSS process)
Power Spectral Density
Optimal Linear ltering
Advanced Quiz
The Kalman ltering formulation writes : for k = 0, 1, . . .
Xk+1 = FXk + GUk
Yk = HXk + Bk,
evolution of states
observations
1. H, F and G are deterministic and known matrices;
2. Uk and Bk : i.i.d SP, Uk
3. Xk, Yk: SP with dim(Xk) = M and dim(Yk) = N initialized by X0.
(0, R), Bk
(0, Q)
N
N
Show that :
Xk
and
Yk
are WSS if
{
}
{
}
1. F is a stable "lter", i.e.all eigenvalues lies within the unit circle;
2. zero initial mean
3. the initial covariance P0 = var is a solution of the Lyapunov
= 0
equation: P = FPFt + GQGt.
Remark : if only cond (1)(2) are satised, the asymptotic covariance P
is a solution of the Lyapunov equation. (Th. de Lyapunov)
1
LTSI, b timent 22, Campus de Beaulieu
6 / 45
Prerequisites
Wiener Filter
Kalman Filter
Stochastic process and stationarity
Covariance matrix (for WSS process)
Power Spectral Density
Optimal Linear ltering
Covariance matrix and covariance function
Construction
Let xn = , . . . , x ]t denote the L-element observation of the
Rx =
rx(0)
rx(
1)
...
L + 1)
rx(
rx(1)
rx(0)
...
L + 2)
rx(
. . .
. . .
...
. . .
1)
2)
rx(L
rx(L
...
rx(0)
3
7
7
7
5
= 2
6
6
6
4
H denotes the conjugate (Hermitian) transpose and rx(k) the auto-
where
covariance function
~ The covariance matrix plays a key role in signal processing of WSS
process.
LTSI, b timent 22, Campus de Beaulieu
7 / 45
Toeplitz
Positive semidenite
Ri,j = Ri+1,j+1
R
0
,
zHRz
0,
z
8
or equivalently all eigenvalues are non-negative.
Prerequisites
Wiener Filter
Kalman Filter
Stochastic process and stationarity
Covariance matrix (for WSS process)
Power Spectral Density
Optimal Linear ltering
Properties of the covariance matrix R
Hermitian (or self-adjoint matrix) since rx(k) = rx(
k)
R = RH
for complex valued x, and symmetric R = Rt for real valued x.
LTSI, b timent 22, Campus de Beaulieu
8 / 45
Positive semidenite
R
0
,
zHRz
0,
z
8
or equivalently all eigenvalues are non-negative.
Prerequisites
Wiener Filter
Kalman Filter
Stochastic process and stationarity
Covariance matrix (for WSS process)
Power Spectral Density
Optimal Linear ltering
Properties of the covariance matrix R
Hermitian (or self-adjoint matrix) since rx(k) = rx(
k)
R = RH
for complex valued x, and symmetric R = Rt for real valued x.
Toeplitz
Ri,j = Ri+1,j+1
LTSI, b timent 22, Campus de Beaulieu
8 / 45
Prerequisites
Wiener Filter
Kalman Filter
Stochastic process and stationarity
Covariance matrix (for WSS process)
Power Spectral Density
Optimal Linear ltering
Properties of the covariance matrix R
Hermitian (or self-adjoint matrix) since rx(k) = rx(
k)
R = RH
for complex valued x, and symmetric R = Rt for real valued x.
Toeplitz
Positive semidenite
Ri,j = Ri+1,j+1
R
0
,
zHRz
0,
z
8
or equivalently all eigenvalues are non-negative.
LTSI, b timent 22, Campus de Beaulieu
8 / 45
. PSD x(f ) are periodic and of period 1;
. PSD x(f ) are always real-valued : x(f ) = x(f );
. PSD are even: x(
f ) = x(f ) for real x ;
. PSD are non-negative : x(f )
0 for all f ;
. rx[0] = [
x
2] =
x(f )df = m2
x + 2
x;
|
|
1
2
1
2
R
Show that :
Prerequisites
Wiener Filter
Kalman Filter
Stochastic process and stationarity
Covariance matrix (for WSS process)
Power Spectral Density
Optimal Linear ltering
Denition
Einstein-Wiener-Khintchine relation (discrete SP)
The power spectral density (PSD) is the frequency domain equivalent of the
autocorrelation function in the temporal domain :
x(f ) =
(rx) =
F
rx exp(
j2 f k),
f
1
2
,
1
2
2
(2)
rx =
F
1(x) =
x(f ) exp(
j2 f k)df,
k = 0,
1, . . .
(3)
+
1
Xk=
1
1
2
1
2
Z
LTSI, b timent 22, Campus de Beaulieu
9 / 45
. PSD x(f ) are always real-valued : x(f ) = x(f );
. PSD are even: x(
f ) = x(f ) for real x ;
. PSD are non-negative : x(f )
0 for all f ;
. rx[0] = [
x
Publicité
2] =
x(f )df = m2
x + 2
x;
|
|
1
2
1
2
R
Prerequisites
Wiener Filter
Kalman Filter
Stochastic process and stationarity
Covariance matrix (for WSS process)
Power Spectral Density
Optimal Linear ltering
Denition
Einstein-Wiener-Khintchine relation (discrete SP)
The power spectral density (PSD) is the frequency domain equivalent of the
autocorrelation function in the temporal domain :
x(f ) =
(rx) =
F
rx exp(
j2 f k),
f
1
2
,
1
2
2
(2)
1(x) =
x(f ) exp(
j2 f k)df,
k = 0,
1, . . .
(3)
+
1
Xk=
1
1
2
1
2
Z
rx =
F
Show that :
. PSD x(f ) are periodic and of period 1;
LTSI, b timent 22, Campus de Beaulieu
9 / 45
. PSD are even: x(
f ) = x(f ) for real x ;
. PSD are non-negative : x(f )
0 for all f ;
. rx[0] = [
x
2] =
x(f )df = m2
x + 2
x;
|
|
1
2
1
2
R
Prerequisites
Wiener Filter
Kalman Filter
Stochastic process and stationarity
Covariance matrix (for WSS process)
Power Spectral Density
Optimal Linear ltering
Denition
Einstein-Wiener-Khintchine relation (discrete SP)
The power spectral density (PSD) is the frequency domain equivalent of the
autocorrelation function in the temporal domain :
x(f ) =
(rx) =
F
rx exp(
j2 f k),
f
1
2
,
1
2
2
(2)
1(x) =
x(f ) exp(
j2 f k)df,
k = 0,
1, . . .
(3)
+
1
Xk=
1
1
2
1
2
Z
rx =
F
Show that :
. PSD x(f ) are periodic and of period 1;
. PSD x(f ) are always real-valued : x(f ) = x(f );
LTSI, b timent 22, Campus de Beaulieu
9 / 45
. PSD are non-negative : x(f )
0 for all f ;
. rx[0] = [
x
2] =
x(f )df = m2
x + 2
x;
|
|
1
2
1
2
R
Prerequisites
Wiener Filter
Kalman Filter
Stochastic process and stationarity
Covariance matrix (for WSS process)
Power Spectral Density
Optimal Linear ltering
Denition
Einstein-Wiener-Khintchine relation (discrete SP)
The power spectral density (PSD) is the frequency domain equivalent of the
autocorrelation function in the temporal domain :
x(f ) =
(rx) =
F
rx exp(
j2 f k),
f
1
2
,
1
2
2
(2)
1(x) =
x(f ) exp(
j2 f k)df,
k = 0,
1, . . .
(3)
+
1
Xk=
1
1
2
1
2
Z
rx =
F
Show that :
. PSD x(f ) are periodic and of period 1;
. PSD x(f ) are always real-valued : x(f ) = x(f );
. PSD are even: x(
f ) = x(f ) for real x ;
LTSI, b timent 22, Campus de Beaulieu
9 / 45
. rx[0] = [
x
2] =
x(f )df = m2
x + 2
x;
|
|
1
2
1
2
R
Prerequisites
Wiener Filter
Kalman Filter
Stochastic process and stationarity
Covariance matrix (for WSS process)
Power Spectral Density
Optimal Linear ltering
Denition
Einstein-Wiener-Khintchine relation (discrete SP)
The power spectral density (PSD) is the frequency domain equivalent of the
autocorrelation function in the temporal domain :
x(f ) =
(rx) =
F
rx exp(
j2 f k),
f
1
2
,
1
2
2
(2)
1(x) =
x(f ) exp(
j2 f k)df,
k = 0,
1, . . .
(3)
+
1
Xk=
1
1
2
1
2
Z
rx =
F
Show that :
. PSD x(f ) are periodic and of period 1;
. PSD x(f ) are always real-valued : x(f ) = x(f );
. PSD are even: x(
. PSD are non-negative : x(f )
f ) = x(f ) for real x ;
0 for all f ;
LTSI, b timent 22, Campus de Beaulieu
9 / 45
Prerequisites
Wiener Filter
Kalman Filter
Stochastic process and stationarity
Covariance matrix (for WSS process)
Power Spectral Density
Optimal Linear ltering
Denition
Einstein-Wiener-Khintchine relation (discrete SP)
The power spectral density (PSD) is the frequency domain equivalent of the
autocorrelation function in the temporal domain :
x(f ) =
(rx) =
F
rx exp(
j2 f k),
f
1
2
,
1
2
2
(2)
1(x) =
x(f ) exp(
j2 f k)df,
k = 0,
1, . . .
(3)
+
1
Publicité
Xk=
1
1
2
1
2
Z
rx =
F
Show that :
. PSD x(f ) are periodic and of period 1;
. PSD x(f ) are always real-valued : x(f ) = x(f );
. PSD are even: x(
. PSD are non-negative : x(f )
f ) = x(f ) for real x ;
0 for all f ;
. rx[0] = [
2] =
x
|
|
1
2
1
2
R
x(f )df = m2
x + 2
x;
LTSI, b timent 22, Campus de Beaulieu
9 / 45
Prerequisites
Wiener Filter
Kalman Filter
Stochastic process and stationarity
Covariance matrix (for WSS process)
Power Spectral Density
Optimal Linear ltering
Example of PSD functions 1/2
1.
2.
Examples
RX( ) = e
|
|
SX(f ) =
2
2 + (2f )2
f
RX( ) =
2
2
cos
SX(f )
2
4
f
2
2
4
2
EE 278B: Stationary Random Processes
7 15
LTSI, b timent 22, Campus de Beaulieu
10 / 45
3.
RX(n) = 2|
n
|
SX(f ) =
5
4 cos 2f
3
f
4
3
2
1
1 2 3 4
1
2
1
2
4. Discrete time white noise process: X1, X2, . . . , Xn, . . . zero mean, uncorrelated,
with average power N
RX(n) =
N n = 0
0
otherwise
SX(f )
N
N
If Xn is also a GRP, then we obtain a discrete time WGN process
1
2
+1
2
f
n
n
EE 278B: Stationary Random Processes
7 16
1.
2.
Examples
RX( ) = e
|
|
SX(f ) =
2
2 + (2f )2
RX( ) =
cos
2
2
f
f
SX(f )
2
4
2
2
4
2
Prerequisites
Wiener Filter
Kalman Filter
Stochastic process and stationarity
Covariance matrix (for WSS process)
Power Spectral Density
Optimal Linear ltering
EE 278B: Stationary Random Processes
Example of PSD functions 2/2
7 15
3.
RX(n) = 2|
n
|
SX(f ) =
3
4 cos 2f
5
4
3
2
1
1 2 3 4
n
1
2
1
2
f
4. Discrete time white noise process: X1, X2, . . . , Xn, . . . zero mean, uncorrelated,
with average power N
RX(n) =
N n = 0
0
otherwise
SX(f )
N
N
n
1
2
+1
2
f
If Xn is also a GRP, then we obtain a discrete time WGN process
EE 278B: Stationary Random Processes
LTSI, b timent 22, Campus de Beaulieu
7 16
11 / 45
Sol:
1. use the z-Transform to get X(z) = z
1X(z) + z
2X(z) + z
1
2. or X(z) =
z
z1
1 = A
z
+ B
z
z2
= A z
1
1
z
1 + B z
1
z
1
1
3. Apply inverse z-Transform, x = A n
1u + B( )n
1u
using
(x ) = z
kX(z)
Z
Prerequisites
Wiener Filter
Kalman Filter
Stochastic process and stationarity
Covariance matrix (for WSS process)
Power Spectral Density
Optimal Linear ltering
Dierence Eq and z-Transform (1/2)
Denition of z-Transform on discrete time series
X(z) =
(x ) =
Z
+
1
x z
n
(4)
n=
X
1
Example of solving a dierence equation using z-transform
Establish x for x = x = 0 for all n negative.
1] + x +
LTSI, b timent 22, Campus de Beaulieu
12 / 45
Prerequisites
Wiener Filter
Kalman Filter
Stochastic process and stationarity
Covariance matrix (for WSS process)
Power Spectral Density
Optimal Linear ltering
Dierence Eq and z-Transform (1/2)
Denition of z-Transform on discrete time series
X(z) =
(x ) =
Z
+
1
x z
n
(4)
n=
X
1
Example of solving a dierence equation using z-transform
Establish x for x = x = 0 for all n negative.
1] + x +
Sol:
1. use the z-Transform to get X(z) = z
1X(z) + z
2X(z) + z
1
2. or X(z) =
3. Apply inverse z-Transform, x = A n
+ B
1 = A
z
z1
z2
z
z
= A z
1
z
1u ) = z
kX(z)
Publicité
1
1 + B z
1
z
1
1] + B( )n
1u
LTSI, b timent 22, Campus de Beaulieu
12 / 45
Prerequisites
Wiener Filter
Kalman Filter
Stochastic process and stationarity
Covariance matrix (for WSS process)
Power Spectral Density
Optimal Linear ltering
Recurrence Eq and z-Transform (2/2) : Table
LTSI, b timent 22, Campus de Beaulieu
13 / 45
Apply the z-Transform on both side, we have the transfer function H(z) for
the LTI lter:
Y (z)
X(z)
=
b0 + b1z
1 + . . . + bnb z
nb
a0 + a1z
1 + . . . + ana z
na
= H(z)
(6)
We can always normalize a0 = 1 and dene an LTI lter by b =
and a = . As in matlab/octave, the
y = filter(b,a,x)
command lters the input data x with H(z) dened by b and a:
If a0
= 1, then b and a are normalized by a0. Therefore, a0 must be nonzero !
Prerequisites
Wiener Filter
Kalman Filter
Stochastic process and stationarity
Covariance matrix (for WSS process)
Power Spectral Density
Optimal Linear ltering
LTI systems (lter)
Consider the ARMA system (as an LTI lter):
na
nb
which handles both FIR and IIR lters.
Xk=0
Xk=0
y =
x
(5)
LTSI, b timent 22, Campus de Beaulieu
14 / 45
6
command lters the input data x with H(z) dened by b and a:
As in matlab/octave, the
y = filter(b,a,x)
If a0
= 1, then b and a are normalized by a0. Therefore, a0 must be nonzero !
Prerequisites
Wiener Filter
Kalman Filter
Stochastic process and stationarity
Covariance matrix (for WSS process)
Power Spectral Density
Optimal Linear ltering
LTI systems (lter)
Consider the ARMA system (as an LTI lter):
na
nb
y =
x
(5)
Xk=0
Xk=0
which handles both FIR and IIR lters.
Apply the z-Transform on both side, we have the transfer function H(z) for
the LTI lter:
Y (z)
X(z)
=
b0 + b1z
a0 + a1z
1 + . . . + bnb z
1 + . . . + ana z
nb
= H(z)
na
(6)
We can always normalize a0 = 1 and dene an LTI lter by b =
and a = .
LTSI, b timent 22, Campus de Beaulieu
14 / 45
6
Prerequisites
Wiener Filter
Kalman Filter
Stochastic process and stationarity
Covariance matrix (for WSS process)
Power Spectral Density
Optimal Linear ltering
LTI systems (lter)
Consider the ARMA system (as an LTI lter):
na
nb
y =
x
(5)
Xk=0
Xk=0
which handles both FIR and IIR lters.
Apply the z-Transform on both side, we have the transfer function H(z) for
the LTI lter:
Y (z)
X(z)
=
b0 + b1z
a0 + a1z
1 + . . . + bnb z
1 + . . . + ana z
nb
= H(z)
na
(6)
We can always normalize a0 = 1 and dene an LTI lter by b =
and a = . As in matlab/octave, the
command lters the input data x with H(z) dened by b and a:
y = filter(b,a,x)
If a0
= 1, then b and a are normalized by a0. Therefore, a0 must be nonzero !
LTSI, b timent 22, Campus de Beaulieu
14 / 45
6
Prerequisites
Wiener Filter
Kalman Filter
Stochastic process and stationarity
Covariance matrix (for WSS process)
Power Spectral Density
Optimal Linear ltering
Frequency response and lter design 1/3
In matlab/octave, the = freqz(b,a,n) command is used to
calculate the n-point frequency response of LTI lter (H(z) dened by b
and a).
Example of frequency response
Compute and display the magnitude response of 3-rd order IIR lowpass lter:
H(z) =
Sol:
0.05(1 + z
0.7z
(1
1)(1
1)(1
1.2z
1.5z
2)
1 + z
2)
1 + 0.9z
1. b= .5*conv([1 1], [1 -1.2 1]); % for the b vector
2. a= conv([1 -.7], [1 -1.5 .9]); % for the a vector
3. = freqz(b,a,2001); % 2001-point frequency response in h
and corresponding frequency in w
4. plot(w/pi/2,20*log10(abs(h))); % normalize w by 2 , and
convert h to dB
LTSI, b timent 22, Campus de Beaulieu
15 / 45
Prerequisites
Wiener Filter
Kalman Filter
Stochastic process and stationarity
Covariance matrix (for WSS process)
Power Spectral Density
Optimal Linear ltering
Frequency response and lter design 2/3
Graphical results:
Bonus Questions:
1. how do we recognize the LP lter and calculate its cut-o frequency
3dB compared to H(e
j2 0) ?
2. how to trace the phase response using h (degrees vs normalized
(
frequency) ?
Note : freqz(b,a) plots magnitude and phase response with no output
LTSI, b timent 22, Campus de Beaulieu
16 / 45
Prerequisites
Wiener Filter
Kalman Filter
Stochastic process and stationarity
Covariance matrix (for WSS process)
Power Spectral Density
Optimal Linear ltering
Frequency response and lter design 3/3
Filter design specications (example with LP lter)
Digital lter design consists of specifying
1. passband, stopband frequencies and attenuation (transition width);
2. maximum passband and stopband ripples (in dB);
3. lter order (or length of the impulse response for FIR).
LTSI, b timent 22, Campus de Beaulieu
17 / 45
1. Wp = 40/500; Ws = 150/500; Rp=3; Rs = 60; % normalized
passband/stopband frequency, passband ripple and stopband
attenuation (in dB)
2. = cheb1ord(Wp,Ws,Rp,Rs); % returns the lowest order n
and normalized passband edge frequency Wp
3. = cheby1(n,Rp,Wp, low); % returns the transfer
function of an nth-order lowpass Chebyshev Type I filter
4. freqz(b,a,512,1000); % trace a 512-point frequency response
with Fs = 1000
Sol:
Prerequisites
Wiener Filter
Kalman Filter
Stochastic process and stationarity
Covariance matrix (for WSS process)
Power Spectral Density
Optimal Linear ltering
Example of Chebyshev Type I lter design 1/2
Design a lowpass Chebyshev Type I lter with less than 3 dB passband
([0, 40]Hz) ripple and at least 60dB stopband (> 150Hz) attenuation. Fs =
1000 Hz
LTSI, b timent 22, Campus de Beaulieu
18 / 45
2. = cheb1ord(Wp,Ws,Rp,Rs); % returns the lowest order n
and normalized passband edge frequency Wp
3. = cheby1(n,Rp,Wp, low); % returns the transfer
function of an nth-order lowpass Chebyshev Type I filter
4. freqz(b,a,512,1000); % trace a 512-point frequency response
with Fs = 1000
Prerequisites
Wiener Filter
Kalman Filter
Stochastic process and stationarity
Covariance matrix (for WSS process)
Power Spectral Density
Optimal Linear ltering
Example of Chebyshev Type I lter design 1/2
Design a lowpass Chebyshev Type I lter with less than 3 dB passband
([0, 40]Hz) ripple and at least 60dB stopband (> 150Hz) attenuation. Fs =
1000 Hz
Sol:
1. Wp = 40/500; Ws = 150/500; Rp=3; Rs = 60; % normalized
passband/stopband frequency, passband ripple and stopband
attenuation (in dB)
LTSI, b timent 22, Campus de Beaulieu
18 / 45
3. = cheby1(n,Rp,Wp, low); % returns the transfer
function of an nth-order lowpass Chebyshev Type I filter
4. freqz(b,a,512,1000); % trace a 512-point frequency response
with Fs = 1000
Prerequisites
Wiener Filter
Kalman Filter
Stochastic process and stationarity
Covariance matrix (for WSS process)
Power Spectral Density
Optimal Linear ltering
Example of Chebyshev Type I lter design 1/2
Design a lowpass Chebyshev Type I lter with less than 3 dB passband
([0, 40]Hz) ripple and at least 60dB stopband (> 150Hz) attenuation. Fs =
1000 Hz
Sol:
1. Wp = 40/500; Ws = 150/500; Rp=3; Rs = 60; % normalized
passband/stopband frequency, passband ripple and stopband
attenuation (in dB)
2. = cheb1ord(Wp,Ws,Rp,Rs); % returns the lowest order n
and normalized passband edge frequency Wp
LTSI, b timent 22, Campus de Beaulieu
18 / 45
4. freqz(b,a,512,1000); % trace a 512-point frequency response
with Fs = 1000
Prerequisites
Wiener Filter
Kalman Filter
Stochastic process and stationarity
Covariance matrix (for WSS process)
Power Spectral Density
Optimal Linear ltering
Example of Chebyshev Type I lter design 1/2
Design a lowpass Chebyshev Type I lter with less than 3 dB passband
([0, 40]Hz) ripple and at least 60dB stopband (> 150Hz) attenuation. Fs =
1000 Hz
Sol:
1. Wp = 40/500; Ws = 150/500; Rp=3; Rs = 60; % normalized
passband/stopband frequency, passband ripple and stopband
attenuation (in dB)
2. = cheb1ord(Wp,Ws,Rp,Rs); % returns the lowest order n
and normalized passband edge frequency Wp
3. = cheby1(n,Rp,Wp, low); % returns the transfer
function of an nth-order lowpass Chebyshev Type I filter
LTSI, b timent 22, Campus de Beaulieu
18 / 45
Prerequisites
Wiener Filter
Kalman Filter
Stochastic process and stationarity
Covariance matrix (for WSS process)
Power Spectral Density
Optimal Linear ltering
Example of Chebyshev Type I lter design 1/2
Design a lowpass Chebyshev Type I lter with less than 3 dB passband
([0, 40]Hz) ripple and at least 60dB stopband (> 150Hz) attenuation. Fs =
1000 Hz
Sol:
1. Wp = 40/500; Ws = 150/500; Rp=3; Rs = 60; % normalized
passband/stopband frequency, passband rip...