# 协方差与协方差矩阵

## 协方差

### 随机变量的协方差

$\operatorname {cov} (X,Y)=\operatorname {E} {{\big [}(X-\operatorname {E} [X])(Y-\operatorname {E} [Y]){\big ]}}$

$$X$$$$Y$$是同一个随机变量时，$$X$$与其自身的协方差就是$$X$$的方差，可以说方差是协方差的一个特例。

$\operatorname{cov}(X,X)=\operatorname{E}\big[(X-\operatorname{E}[X])(X-\operatorname{E}[X])\big]$

$\operatorname{var}(X)=\operatorname{cov}(X,X)=\operatorname{E}\big[(X-\operatorname{E}[X])^2]$

$\eta ={\dfrac {\operatorname {cov} (X,Y)}{\sqrt {\operatorname {var} (X)\cdot \operatorname {var} (Y)}}}\$

### 样本的协方差

$q_{ab}=\dfrac {\sum_{j=1}^m{(x_{aj}-\bar x_a)(x_{bj}-\bar x_b)}}{m-1}$

## 协方差矩阵

### 多维随机变量的协方差矩阵

$\Sigma_{ij}=\operatorname{cov}(X_i,X_j)=\operatorname{E}\big[(X_i-\operatorname{E}[X_i])(X_j-\operatorname{E}[X_j])\big]$

$\Sigma=\operatorname{E}\big[(\textbf X-\operatorname{E}[\textbf X]\big)(\textbf X-\operatorname{E}[\textbf X])^T]$

$=\begin{bmatrix} \operatorname{cov}(X_1, X_1) & \operatorname{cov}(X_1, X_2) & \cdots & \operatorname{cov}(X_1, X_n) \\ \operatorname{cov}(X_2, X_1) & \operatorname{cov}(X_2, X_2) & \cdots & \operatorname{cov}(X_2, X_n) \\ \vdots & \vdots & \ddots & \vdots \\ \operatorname{cov}(X_n, X_1) & \operatorname{cov}(X_n, X_2) & \cdots & \operatorname{cov}(X_n, X_n) \end{bmatrix}$

$=\begin{bmatrix} \operatorname{E}\big[(X_1-\operatorname{E}[X_1])(X_1-\operatorname{E}[X_1])\big] & \operatorname{E}\big[(X_1-\operatorname{E}[X_1])(X_2-\operatorname{E}[X_2])\big] & \cdots & \operatorname{E}\big[(X_1-\operatorname{E}[X_1])(X_n-\operatorname{E}[X_n])\big] \\ \operatorname{E}\big[(X_2-\operatorname{E}[X_2])(X_1-\operatorname{E}[X_1])\big] & \operatorname{E}\big[(X_2-\operatorname{E}[X_2])(X_2-\operatorname{E}[X_2])\big] & \cdots & \operatorname{E}\big[(X_2-\operatorname{E}[X_2])(X_n-\operatorname{E}[X_n])\big] \\ \vdots & \vdots & \ddots & \vdots \\ \operatorname{E}\big[(X_n-\operatorname{E}[X_n])(X_1-\operatorname{E}[X_1])\big] & \operatorname{E}\big[(X_n-\operatorname{E}[X_n])(X_2-\operatorname{E}[X_2])\big] & \cdots & \operatorname{E}\big[(X_n-\operatorname{E}[X_n])(X_n-\operatorname{E}[X_n])\big] & \end{bmatrix}$

### 样本的协方差矩阵

$\hat \Sigma=\begin{bmatrix} q_{11} & q_{12} & \cdots & q_{1n} \\ q_{21} & q_{21} & \cdots & q_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ q_{n1} & q_{n2} & \cdots & q_{nn} \end{bmatrix}$

$=\frac {1}{m-1} \begin{bmatrix} {\sum_{j=1}^m{(x_{1j}-\bar x_1)(x_{1j}-\bar x_1)}} & {\sum_{j=1}^m{(x_{1j}-\bar x_1)(x_{2j}-\bar x_2)}} & \cdots & {\sum_{j=1}^m{(x_{1j}-\bar x_1)(x_{nj}-\bar x_n)}} \\ {\sum_{j=1}^m{(x_{2j}-\bar x_2)(x_{1j}-\bar x_1)}} & {\sum_{j=1}^m{(x_{2j}-\bar x_2)(x_{2j}-\bar x_2)}} & \cdots & {\sum_{j=1}^m{(x_{2j}-\bar x_2)(x_{nj}-\bar x_n)}} \\ \vdots & \vdots & \ddots & \vdots \\ {\sum_{j=1}^m{(x_{nj}-\bar x_n)(x_{1j}-\bar x_1)}} & {\sum_{j=1}^m{(x_{nj}-\bar x_n)(x_{2j}-\bar x_2)}} & \cdots & {\sum_{j=1}^m{(x_{nj}-\bar x_n)(x_{nj}-\bar x_n)}} \end{bmatrix}$

$=\frac {1}{m-1} \sum_{j=1}^m (\textbf x_{\cdot j} - \bar {\textbf x}) (\textbf x_{\cdot j} - \bar {\textbf x})^T$

1. $$\textbf y_{\cdot j} = \textbf x_{\cdot j } - \bar {\textbf x}$$。即对样本进行平移，使其重心在原点；
2. $$\textbf z_{i \cdot} = \textbf y_{i \cdot} / \sigma_i$$。其中$$\sigma_i$$是维度$$i$$的标准差。这样消除了数值大小的影响。

$\hat \Sigma=\frac {1}{m-1} \sum_{j=1}^{m}\textbf z_{\cdot j} \textbf z_{\cdot j}^T$

posted @ 2016-12-30 10:50  苦力笨笨  阅读(77405)  评论(7编辑  收藏  举报