# Chapter 2：多元正态分布的定义和性质

## 一、多元正态分布的定义和基本性质

### Part 1：标准正态的线性变换

${\rm E}(U)=0 \ , \quad {\rm Var}(U)=I_{q} \ .$

\begin{aligned} & {\rm E}(X)={\rm E}(AU+\mu)=A{\rm E}(U)+\mu=\mu \ . \\ \\ & {\rm Var}(X)={\rm Var}(AU+\mu)=A{\rm Var}(U)A'=AA' \ . \end{aligned}

$\Phi_X(t)=\exp\left\{it'\mu-\frac12t'AA't\right\} \ .$

$\Phi_{U_i}(t_i)={\rm E}\left[e^{it_iU_i}\right]=\exp\left\{-\frac12t_i^2\right\} \ .$

$\Phi_U(t)={\rm E}\left[e^{it'U}\right]=\exp\left\{-\frac12\sum_{i=1}^qt_i^2\right\}=\exp\left\{-\frac12t't\right\} \ .$

\begin{aligned} \Phi_X(t)&={\rm E}\left[e^{it'X}\right]={\rm E}\left[e^{it'(AU+\mu)}\right] \\ \\ &=\exp\left\{it'\mu\right\}\cdot{\rm E}\left[e^{it'AU}\right] \\ \\ &=\exp\left\{it'\mu\right\}\cdot{\rm E}\left[e^{i(A't)'U}\right] \\ \\ &=\exp\left\{it'\mu\right\}\cdot\exp\left\{-\frac12(A't)'(A't)\right\} \\ \\ &=\exp\left\{it'\mu-\frac12t'AA't\right\} \ . \end{aligned}

### Part 2：由特征函数定义

$\Phi_X(t)=\exp\left\{it'\mu-\frac12t'\Sigma t\right\} \ , \quad \Sigma\geq0 \ ,$

$Y=BX+d\xlongequal{d}B(AU+\mu)+d=BAU+B\mu+d \ ,$

${\rm E}(Y)=B\mu+d \ , \quad {\rm Var}(Y)=(BA)(BA)'=BAA'B'=B\Sigma B' \ .$

$Y\sim N_s(B\mu+d,B\Sigma B') \ .$

$\mu=\left[\begin{array}{c} \mu^{(1)} \\ \mu^{(2)} \end{array}\right]\begin{array}{l} r \\ p-r \end{array} \ , \quad \Sigma=\left[\begin{array}{c:c} \Sigma_{11} & \Sigma_{12} \\ \hdashline \Sigma_{21} & \Sigma_{22} \end{array}\right]\begin{array}{l} r \\ p-r \end{array} \ ,$

$B_1=(I_r,O),\,d_1=0$$B_2=(O,I_{p-r}),\,d_2=0$ ，其中 $d_1$$d_2$ 分别为 $r$ 维和 $p-r$ 维的零向量，由性质 2 可得

\begin{aligned} &X^{(1)}=B_1X+d_1\sim N_r\left(\mu^{(1)},\Sigma_{11}\right) \ , \\ \\ &X^{(2)}=B_2X+d_2\sim N_{p-r}\left(\mu^{(2)},\Sigma_{22}\right) \ . \end{aligned}

### Part 3：任意线性组合为正态随机变量

$\xi=a'X=\sum_{j=1}^pa_jX_j\sim N\left(a'\mu,a'\Sigma a\right) \ .$

$\Phi_\xi(s)={\rm E}\left[e^{is\xi}\right]=\exp\left\{is(t'\mu)-\frac12s^2(t'\Sigma t)\right\} \ .$

$s=1$ 则有

$\Phi_\xi(1)={\rm E}\left[e^{i\xi}\right]={\rm E}\left[e^{it'X}\right]=\Phi_X(t)=\exp\left\{it'\mu-\frac12t'\Sigma t\right\} \ .$

### Part 4：由联合密度函数定义

$f(x)=\frac{1}{(2\pi)^{p/2}|\Sigma|^{1/2}}\exp\left\{-\frac12(x-\mu)'\Sigma^{-1}(x-\mu)\right\} \ ,$

$f(x)=\frac{1}{(2\pi)^{p/2}|\Sigma|^{1/2}}\exp\left\{-\frac12(x-\mu)'\Sigma^{-1}(x-\mu)\right\} \ .$

$X\xlongequal{d}AU+\mu \ .$

$f_U(u)=\frac{1}{(2\pi)^{p/2}}\exp\left\{-\frac12u'u\right\} \ .$

\begin{aligned} J(x\to u)&=\left|\frac{\partial{x'}}{\partial u}\right|=\left|A'\right|=\left|AA'\right|^{1/2}=|\Sigma|^{1/2} \end{aligned}

\begin{aligned} f_X(x)&=\frac{1}{(2\pi)^{p/2}}\exp\left\{-\frac12u'u\right\}|J(u\to x)| \\ \\ &=\frac{1}{(2\pi)^{p/2}}\exp\left\{-\frac12\left[A^{-1}(x-\mu)\right]'\left[A^{-1}(x-\mu)\right]\right\}|\Sigma|^{-1/2} \\ \\ &=\frac{1}{(2\pi)^{p/2}|\Sigma|^{1/2}}\exp\left\{-\frac12(x-\mu)'\Sigma^{-1}(x-\mu)\right\} \ . \end{aligned}

## 二、独立性和条件分布

### Part 1：多元正态分布的独立性

$X=\left[\begin{array}{c} X^{(1)} \\ X^{(2)} \\ \end{array}\right] \ , \quad \mu=\left[\begin{array}{c} \mu^{(1)} \\ \mu^{(2)} \\ \end{array}\right] \ , \quad \Sigma=\left[\begin{array}{cc} \Sigma_{11} & \Sigma_{12} \\ \Sigma_{21} & \Sigma_{22} \\ \end{array}\right] \ .$

$X^{(1)}$$X^{(2)}$ 相互独立当且仅当 $\Sigma_{12}=O$ ，即 $X^{(1)}$$X^{(2)}$ 互不相关。

${\rm Cov}\left(X^{(1)},X^{(2)}\right)=\Sigma_{12}=O \ .$

\begin{aligned} f(x)&=\frac{1}{(2\pi)^{p/2}|\Sigma|^{1/2}}\exp\left\{-\frac12(x-\mu)'\left[ \begin{array}{cc} \Sigma_{11} & O \\ O & \Sigma_{22} \end{array}\right]^{-1}(x-\mu)\right\} \\ \\ &=\frac{1}{(2\pi)^{r/2}\left|\Sigma_{11}\right|^{1/2}}\exp\left\{-\frac12\left(x^{(1)}-\mu^{(1)}\right)' \Sigma_{11}^{-1}\left(x^{(1)}-\mu^{(1)}\right)\right\} \\ &\quad\ \times\frac{1}{(2\pi)^{(p-r)/2}\left|\Sigma_{22}\right|^{1/2}}\exp\left\{-\frac12\left(x^{(2)}-\mu^{(2)}\right)' \Sigma_{22}^{-1}\left(x^{(2)}-\mu^{(2)}\right)\right\} \\ \\ &=f_1\left(x^{(1)}\right)\cdot f_2\left(x^{(2)}\right) \ . \end{aligned}

$X=\left[\begin{array}{c} X^{(1)} \\ \vdots \\ X^{(k)} \end{array} \right]\begin{array}{c} r_1 \\ \vdots \\ r_k \end{array}\sim N_p\left(\left[\begin{array}{c} \mu^{(1)} \\ \vdots \\ \mu^{(k)} \end{array} \right],\left[\begin{array}{c} \Sigma_{11} &\cdots &\Sigma_{1k} \\ \vdots & & \vdots \\ \Sigma_{k1} &\cdots &\Sigma_{kk} \end{array} \right]\right) \ ,$

$X^{(1)},\cdots,X^{(k)}$ 相互独立当且仅当 $\Sigma_{ij}=O,\,\forall i\neq j$​ 。

### Part 2：多元正态分布的条件分布

$\left(X^{(1)}\big|X^{(2)}=x^{(2)}\right)\sim N_r\left(\mu_{1\cdot 2},\Sigma_{11\cdot2}\right) \ ,$

\begin{aligned} &\mu_{1\cdot2}=\mu^{(1)}+\Sigma_{12}\Sigma_{22}^{-1}\left(x^{(2)}-\mu^{(2)}\right) \ , \\ \\ & \Sigma_{11\cdot2}=\Sigma_{11}-\Sigma_{12}\Sigma_{22}^{-1}\Sigma_{21} \ . \end{aligned}

$Z=\left[\begin{array}{c} Z^{(1)} \\ Z^{(2)} \end{array}\right]= \left[\begin{array}{c} X^{(1)}-\Sigma_{12}\Sigma_{22}^{-1}X^{(2)} \\ X^{(2)} \end{array}\right]= \left[\begin{array}{c:c} I_r & -\Sigma_{12}\Sigma_{22}^{-1} \\ \hdashline O & I_{p-r} \end{array}\right]\left[\begin{array}{c} X^{(1)} \\ X^{(2)} \end{array}\right]=BX \ .$

\begin{aligned} &{\rm E}(Z)=B{\rm E}(X)=\left[\begin{array}{c} \mu^{(1)}-\Sigma_{12}\Sigma_{22}^{-1}\mu^{(2)} \\ \mu^{(2)} \end{array}\right] \\ \\ &\begin{aligned} {\rm Var}(Z)&=B{\rm Var}(X)B' \\ \\ &=\left[\begin{array}{cc} I_r & -\Sigma_{12}\Sigma_{22}^{-1} \\ O & I_{p-r} \end{array}\right]\left[\begin{array}{cc} \Sigma_{11} & \Sigma_{12} \\ \Sigma_{21} & \Sigma_{22} \\ \end{array}\right]\left[\begin{array}{cc} I_r & O \\ \left(-\Sigma_{12}\Sigma_{22}^{-1}\right)' & I_{p-r} \end{array}\right] \\ \\ &=\left[\begin{array}{cc} \Sigma_{11}-\Sigma_{12}\Sigma_{22}^{-1}\Sigma_{21} & O \\ O & \Sigma_{22} \end{array}\right] \ . \end{aligned} \end{aligned}

$Z\sim N_p\left(\left[\begin{array}{c} \mu^{(1)}-\Sigma_{12}\Sigma_{22}^{-1}\mu^{(2)} \\ \mu^{(2)} \end{array}\right],\left[\begin{array}{cc} \Sigma_{11\cdot2} & O \\ O & \Sigma_{22} \end{array}\right] \right)$

$g(z)=g\left(z^{(1)},z^{(2)}\right)=g_1\left(z^{(1)}\right)g_2\left(z^{(2)}\right)=g_1\left(z^{(1)}\right)f_2\left(x^{(2)}\right) \ ,$

\begin{aligned} f(x)&=f\left(x^{(1)},x^{(2)}\right)=g(Bx)\cdot |J(z\to x)| \\ \\ &=g_1\left(x^{(1)}-\Sigma_{12}\Sigma_{22}^{-1}x^{(2)}\right)g_2\left(x^{(2)}\right) \\ \\ &=g_1\left(x^{(1)}-\Sigma_{12}\Sigma_{22}^{-1}x^{(2)}\right)f_2\left(x^{(2)}\right) \ . \end{aligned}

\begin{aligned} f_1\left(x^{(1)}\big|x^{(2)}\right)&=\frac{f\left(x^{(1)},x^{(2)}\right)}{f_2\left(x^{(2)}\right)}=g_1\left(x^{(1)}-\Sigma_{12}\Sigma_{22}^{-1}x^{(2)}\right) \\ \\ &=\frac{1}{\left(2\pi\right)^{r/2}|\Sigma_{11\cdot2}|^{1/2}}\exp\left\{-\frac12\left(x^{(1)}-\mu_{1\cdot2}\right)'\Sigma_{11\cdot2}^{-1}\left(x^{(1)}-\mu_{1\cdot2}\right)\right\} \ . \end{aligned}

(1) $X^{(2)}$$X^{(1)}-\Sigma_{12}\Sigma_{22}^{-1}X^{(2)}$ 相互独立；

(2) $X^{(1)}$​ 与 $X^{(2)}-\Sigma_{21}\Sigma_{11}^{-1}X^{(1)}$​​ 相互独立；

(3) $\left(X^{(2)}\big|X^{(1)}=x^{(1)}\right)\sim N_{p-r}\left(\mu_{2\cdot1},\Sigma_{22\cdot1}\right)$ ，其中

\begin{aligned} &\mu_{2\cdot1}=\mu^{(2)}+\Sigma_{21}\Sigma_{11}^{-1}\left(x^{(1)}-\mu^{(1)}\right) \ , \\ \\ & \Sigma_{11\cdot2}=\Sigma_{22}-\Sigma_{21}\Sigma_{11}^{-1}\Sigma_{12} \ . \end{aligned}

### Part 3：多元正态分布的条件期望和回归

$\left(X^{(1)}\big|X^{(2)}=x^{(2)}\right)\sim N_r\left(\mu_{1\cdot 2},\Sigma_{11\cdot2}\right) \ ,$

$\mu_{1\cdot2}=\mu^{(1)}+\Sigma_{12}\Sigma_{22}^{-1}\left(x^{(2)}-\mu^{(2)}\right) \ .$

${\rm E}\left(X^{(1)}\big|X^{(2)}\right)=\mu^{(1)}+\Sigma_{12}\Sigma_{22}^{-1}\left(X^{(2)}-\mu^{(2)}\right) \ .$

$Z^{(1)}=X^{(1)}-\Sigma_{12}\Sigma_{22}^{-1}X^{(2)} \ ,$

${\rm E}\left(Z^{(1)}\right)=\mu^{(1)}-\Sigma_{12}\Sigma_{22}^{-1}\mu^{(2)} \ .$

\begin{aligned} {\rm E}\left(X^{(1)}\big|X^{(2)}\right)&={\rm E}\left(Z^{(1)}+\Sigma_{12}\Sigma_{22}^{-1}X^{(2)}\big|X^{(2)}\right) \\ \\ &={\rm E}\left(Z^{(1)}\big|X^{(2)}\right)+\Sigma_{12}\Sigma_{22}^{-1}X^{(2)} \\ \\ &={\rm E}\left(Z^{(1)}\right)+\Sigma_{12}\Sigma_{22}^{-1}X^{(2)} \\ \\ &=\mu^{(1)}+\Sigma_{12}\Sigma_{22}^{-1}\left(X^{(2)}-\mu^{(2)}\right) \ . \end{aligned}

\begin{aligned} {\rm Var}\left(X^{(1)}\big|X^{(2)}\right)&={\rm Var}\left(Z^{(1)}+\Sigma_{12}\Sigma_{22}^{-1}X^{(2)}\big|X^{(2)}\right) \\ \\ &={\rm Var}\left(Z^{(1)}\big|X^{(2)}\right) \\ \\ &={\rm Var}\left(X^{(1)}\right)+{\rm Var}\left(\Sigma_{12}\Sigma_{22}^{-1}X^{(2)}\right)-2{\rm Cov}\left(X^{(1)},\Sigma_{12}\Sigma_{22}^{-1}X^{(2)}\right) \\ \\ &=\Sigma_{11}+\Sigma_{12}\Sigma_{22}^{-1}\Sigma_{21}-2\Sigma_{12}\Sigma_{22}^{-1}\Sigma_{21} \\ \\ &=\Sigma_{11}-\Sigma_{12}\Sigma_{22}^{-1}\Sigma_{21} \ , \end{aligned}

$\Sigma_{11\cdot2}=\left(\sigma_{ij\cdot2}\right)_{r\times r} \ , \quad i,j=1,2,\cdots,r \ ,$

$r_{ij\cdot2}=\frac{\sigma_{ij\cdot2}}{\sqrt{\sigma_{ii\cdot2}}\sqrt{\sigma_{jj\cdot2}}} \ , \quad i,j=1,2,\cdots,r$

$X_i$$X_j$​ 的偏相关系数，其中 $X_i,X_j\in X^{(1)}$ 。​

### Part 4：全相关系数和最佳预测

$Z=\left[\begin{array}{c} X \\ Y \end{array}\right] \begin{array}{c} p \\ 1 \end{array}\sim N_{p+1}\left(\left[\begin{array}{c} \mu_X \\ \mu_y \end{array}\right],\left[\begin{array}{cc} \Sigma_{XX} & \Sigma_{Xy} \\ \Sigma_{yX} & \sigma_{yy} \end{array}\right]\right) \ .$

\begin{aligned} &{\rm E}(Y|X=x)=\mu_y+\Sigma_{yX}\Sigma_{XX}^{-1}(x-\mu_X) \ , \\ \\ &{\rm Var}(Y|X=x)=\sigma_{yy}-\Sigma_{yX}\Sigma_{XX}^{-1}\Sigma_{Xy} \ . \end{aligned}

$R=\left(\frac{\Sigma_{yX}\Sigma_{XX}^{-1}\Sigma_{Xy}}{\sigma_{yy}}\right)^{1/2} \ .$

${\rm E}\left[(Y-g(X))^2\right]\leq{\rm E}\left[(Y-\varphi(X))^2\right] \ .$

${\rm E}(g(X))={\rm E}[{\rm E}(Y|X)]={\rm E}(Y) \ .$

\begin{aligned} &{\rm E}\left[(Y-\varphi(X))^2\right] \\ \\ =\ &{\rm E}\left[(Y-g(X))^2\right]+{\rm E}\left[(g(X)-\varphi(X))^2\right]+2{\rm E}\left[(Y-g(X))(g(X)-\varphi(X))\right] \\ \\ \geq\ &{\rm E}\left[(Y-g(X))^2\right]+2{\rm E}\left[(Y-g(X))(g(X)-\varphi(X))\right] \\ \\ =\ & {\rm E}\left[(Y-g(X))^2\right]+2{\rm E}\left[{\rm E}\left((Y-g(X))(g(X)-\varphi(X))\right)|X\right] \\ \\ =\ & {\rm E}\left[(Y-g(X))^2\right] \ . \end{aligned}

posted @ 2021-10-21 11:29  这个XD很懒  阅读(244)  评论(0编辑  收藏  举报