(免费)信号与系统课件 第十九章(Hsieh版 共20章)

−1
Rxx Rxy Ryx Ryy
= RT can be
=
−1
−1 0 Rxx 0 0
+ +
−1 R −Rxx xy I
· Q−1 · P −1
−1 −Ryx Rxx
I
(6) (7)
=
0 0 −1 0 Ryy
I −1 R −Ryy yx
I
−1 − Rxy Ryy
where Q and P are called the Schur complements of R yy and Rxx , respectively.
z = Hs + n
Want s ˆ(z ). 1. ML If n ∼ N (0, Rnn ), then max p(z |s)
s
=⇒
max ln p(z |s)
s
=⇒
min
s
1 −1 (z − Hs)T · Rnn · (z − Hs) 2
Let ∇s (·)|s=ˆ sml = 0, we have
m
and y ∈
n
are jointly Gaussian:
Then the conditional pdf of y given x is also Gaussian:
−1 −1 y |x ∼ N µx + Cyx Cxx (y − µy ), Cyy − Cyx Cxx Rxy
3.2
Example: Linear Gaussian Model
−1 E(W ∗ ) ≡ E [e · eT ] = Rss − Rsz · Rzz · Rzs
Proof. Denote E (W ) ≡ E ||e||2 = E [eT e] = E (W z − s)T · (W z − s) = E z T W T W z − 2z T W T s + sT s We cannot take ∇W , because W is a matrix, not a vector. Suppose W ∗ minimizes E (W ). Let W = W∗ + δ · V where δ > 0 is a scalar and V is a matrix. Because d E (W ) dδ We have 2E z T W ∗ T V z − sT V z = 2E (z T W ∗ T − sT ) · V · z = 0 With tr (ABC ) = tr (BCA), we have tr V · E zz T W ∗ T − zsT = tr V Rzz · W ∗ T − Rzs = 0, ∀V =0
−1 Rxy Q = Ryy − Ryx Rxx
(8) (9)
P
=
−1 Rxx − Rxy Ryy Ryx
and the determinant of R can be written as: det det Rxx Rxy Ryx Ryy Rxx Rxy Ryx Ryy = det Rxx · det Q = det Ryy · det P (10) (11)
so that the right-hand-side is close to an upper-triangular matrix. A B C D A B C D
−1
= = = = =
A−1 0 −CA−1 I I A−1 B 0 Q
−1
·
−1
I A−1 B 0 Q A−1 0 −CA−1 I · A−1 0 − 1 I −CA
(13) (14) (15) (16) (17)
·
I −A−1 BQ−1 0 Q−1
A−1 + A−1 BQ−1 CA−1 −A−1 BQ−1 −Q−1 CA−1 Q−1 A−1 0 0 0
T
+
T
−A−1 B I
T
· [Q]−1 · −A−1 B I
Note the symmetry A−1 BQ−1 = Q−1 B T A−1 = Q−1 CA−1 . Besides, from Eqn (13), we have det R = det A · det Q. 3. When we plug the formula in Eqn (6) into Eqn (5), p(y |x) = 1 (2π )n/2 |Q| exp − 1 −1 y − Ryx Rxx x 2
x y
0 0
Then the conditional pdf of y given x is also Gaussian:
−1 −1x Rxx Rxy
(3)
Proof. 1. By definition of conditional pdf: p(y |x) ≡ p(x, y ) p(x) (2π )− =
n+m 2
(4)
T y T ]R−1 |R|−1/2 exp − 1 2 [x
m
x y
T −1 (2π )− 2 |Rxx |−1/2 exp − 1 2 x Rxx x
(5)
2. Matrix-inversion lemma: The inverse of a block symmetric matrix R ≡ written as 2 possible forms: Rxx Rxy Ryx Ryy Rxx Rxy Ryx Ryy
−1 −1 s ˆml = (H T Rnn H )−1 · H T Rnn ·z
XIX - 67
(a) s ˆml is a linear filter on z , and is similar to the Weighted-LS solution. (b) s ˆml is conditionally unbiased.
T −1 Q−1 y − Ryx Rxx x
(18)
which implies the conditional pdf is indeed Gaussian:
−1 −1 −1 y |x ∼ N Ryx Rxx x, Q = N Ryx Rxx x, Ryy − Ryx Rxx Rxy
(19)
Theorem 1 (Gauss-Markov theorem) If x ∈ x y ∼N µx µy , Cxx Cxy Cyx Cyy
E(W ∗ ) ≡ E [e · eT ] = E (W ∗ z − s) · (W ∗ z − s)T = E W ∗ zz T W ∗ T − 2W ∗ zsT + ssT = W ∗ Rzz W ∗ T − 2W ∗ Rzs + Rss = W ∗ Rzs − 2W ∗ Rzs + Rss
−1 = Rss − Rsz · Rzz · Rzs
−1 −1 −1
·z
(1)
· Hs = s
· H · E [s] = 0 = E [s]
3
3.1
Multivariate Gaussian Model
Gauss Markov Theorem
m
If x ∈
and y ∈ ∼N
n
are jointly Gaussian: , Rxx Rxy Ryx Ryy (2)
2
Example
z = Hs + n
where H, Rss , and Rnn are known; s and n are zero-mean and independent. Then Rsz ≡ E sz T = E s · (Hs + n)T = Rss H T Rzz ≡ E zz T = · · · = HRss H T + Rnn The LMMSE from z for s is: y (z ) = W ∗ z = Rss H T HRss H T + Rnn Performance of this LMMSE estimator: • y is conditionally biased. E [y |s] = Rss H T HRss H T + Rnn But y is uncondionally unbiased, E [y ] = Rss H T HRss H T + Rnn because s is zero-mean. • The error covariance matrix is E = · · · = Rss − Rss H T (HRss H T + Rnn )−1 HRss XIX - 65
ICM 5507 Detection & Estimation
Notes (12/28)
Fall 2001 S.F. Hsieh
XIX. Linear Minimum Mean Square Error Estimation(LMMSE)
ˆ(z ) may not be a linear function of z . An LMMSE model In general, a Bayes estimator(MS or MAP) θ aims to find a linear estimator s ˆ = W z so that the mean-square-error E [||s ˆ − s|| 2 ] can be minimized.
1
Linear model:
z → linear filter: W → y = Wz = s ˆ≈ s
where z W ∈ ∈
N
, is the received (observed) vector , M linear filters of length N
M
M ×N
y ≡ Wz ∈ s ∈
M
, estimated vector