# 线性判别分析（Linear Discriminant Analysis, LDA）

### 二、线性判别器的模型

$$p$$$$w$$ 同方向，表示为：$$\dfrac{w}{||w||}$$$$p$$ 的长度为：$$||x||cos\theta = ||x||\dfrac{x \cdot w}{||w||\quad||x||} = \dfrac{x \cdot w}{||w||}$$

#### 2.1 类间投影均值

$u_{0} = \dfrac{1}{m} \sum_{i=1}^{m}x_{i},\quad u_{1} = \dfrac{1}{n} \sum_{i=1}^{n}x_{i}$

#### 2.2 类内投影方差

$z_{0} = \dfrac{1}{n} \sum_{i=1}^{n}(w^{T}x_{i}-w^{T}u_{0})^{2}=\dfrac{1}{n} \sum_{i=1}^{n}(w^{T}x_{i}-w^{T}u_{0})(w^{T}x_{i}-w^{T}u_{0})^T$

$z_{0} = \dfrac{1}{n} \sum_{i=1}^{n}w^{T}(x_{i}-u_{0})(x_{i}-u_{0})^{T}w = w^{T}[\dfrac{1}{n} \sum_{i=1}^{n}(x_{i}-u_{0})(x_{i}-u_{0})^{T}]w$

$$x_{i}, u_{0}$$ 都是一维时， 式子 $$\dfrac{1}{n} \sum_{i=1}^{n}(x_{i}-u_{0})(x_{i}-u_{0})^{T}$$ 就表示所有输入 $$x_{i}$$ 的方差；

$$x_{i}, u_{0}$$ 都是二维时， 式子 $$\dfrac{1}{n} \sum_{i=1}^{n}(x_{i}-u_{0})(x_{i}-u_{0})^{T}$$ 就表示：

$\dfrac{1}{n} \sum_{i=1}^{n}\begin{bmatrix}x_{11}-u_{01} \\x_{12}-u_{02}\end{bmatrix} \begin{bmatrix} x_{11}-u_{01}\quad x_{12}-u_{02}\end{bmatrix} = \dfrac{1}{n} \sum_{i=1}^{n}\begin{bmatrix} (x_{11}-u_{01})^{2}\quad (x_{11}-u_{01})(x_{12}-u_{02})\\ (x_{12}-u_{02})(x_{11}-u_{01})\quad (x_{12}-u_{02})^{2} \end{bmatrix}$

$J(w) = \dfrac{||w^{T}u_{0}-w^{T}u_{1}||^{2}}{z_{0}+z_{1}} = \dfrac{w^{T}(u_{0}-u_{1})(u_{0}-u_{1})^{T}w}{w^{T}(M_{0}+M_{1})w}$

$J(w) = \dfrac{w^{T}S_{b}w}{w^{T}S_{w}w}$

### 三、线性判别器的求解

$\dfrac{\partial J(w)}{\partial w} = 2S_{b}w{(w^{T}S_{w}w)^{-1}} + {(w^{T}S_{b}w)}(-1){(w^{T}S_{w}w)^{-2}} \times 2S_{w}w = 0$

$S_{b}w = \dfrac{w^{T}S_{w}w}{w^{T}S_{w}w}S_{b}w$

$w = \dfrac{w^{T}S_{b}w}{w^{T}S_{w}w}S_{w}^{-1}S_{b}w$

$w = V diag(\lambda_{1},\lambda_{2},...,\lambda_{p})^{-1} U^{T} (u_{0}-u_{1})$

posted @ 2021-06-26 19:57  ZhiboZhao  阅读(421)  评论(0编辑  收藏  举报