线性降维-笔记(2)

4 - MDS

MDS全称"Multidimensional Scaling",多维缩放。其主要思想就是给定一个原始空间的，原始样本两两之间的距离矩阵；期望能在新空间中找到一个新的样本特征矩阵，使得其新样本两两之间的距离矩阵与原始的距离矩阵相等。因为$d' \leq d$，所以完成了降维的任务。

ps：MDS大多都还是使用的欧式距离来作为样本之间的测量方法，更多的方法看下面的表4.1.

${\bf D'}={\bf X'}^T{\bf X'}\in R^{m \times m}$,其中${\bf D'}$为降维后样本的内积矩阵，${D'}_{ij}={\bf x'}_i^T{\bf x'}_j$,则有：

$\begin{eqnarray}D_{ij}^2 &=&||{\bf x'}_i-{\bf x'}_j||^2\\ &=&||{\bf x'}_i||^2+||{\bf x'}_j||^2-2{\bf x'}_i^T{\bf x'}_j\\ &=&{D'}_{ii}+{D'}_{jj}-2{D'}_{ij} \end{eqnarray}\tag{4.1}$

$\sum_i^m{D'}_{ij}=\sum_j^m{D'}_{ij}=0\tag{4.2}$

$\begin{eqnarray}\sum_i^m{D}_{ij}^2 &=&\sum_i^m\left({D'}_{ii}+{D'}_{jj}-2{D'}_{ij}\right)\\ &=&\sum_i^m{D'}_{ii}+m{D'}_{jj}-2\sum_i^m{D'}_{ij}\\ &=&tr({\bf D'})+m{D'}_{jj} \end{eqnarray}\tag{4.3}$

$\sum_j^m{D}_{ij}^2=tr({\bf D'})+m{D'}_{ii}\tag{4.4}$

$\begin{eqnarray}\sum_i^m\sum_j^m{D}_{ij}^2 &=&\sum_i^m\left(tr({\bf D'})+m{D'}_{ii}\right)\\ &=&mtr({\bf D'})+\sum_i^mm{D'}_{ii}\\ &=&2mtr({\bf D'}) \end{eqnarray}\tag{4.5}$

${\overline D}_{i.}^2=\frac{1}{m}\sum_j^m{D}_{ij}^2\tag{4.6}$
${\overline D}_{.j}^2=\frac{1}{m}\sum_i^m{D}_{ij}^2\tag{4.7}$
${\overline D}_{..}^2=\frac{1}{m^2}\sum_i^m\sum_j^m{D}_{ij}^2\tag{4.8}$

$\begin{eqnarray}{D'}_{ij} &=&-\frac{1}{2}\left({D}_{ij}^2-{D'}_{ii}-{D'}_{jj}\right)\\ &=&-\frac{1}{2}\left[{D}_{ij}^2-\frac{1}{m}\left(\sum_j^m{D}_{ij}^2-\frac{1}{2m}\sum_i^m\sum_j^m{D}_{ij}^2\right)-\frac{1}{m}\left(\sum_i^m{D}_{ij}^2-\frac{1}{2m}\sum_i^m\sum_j^m{D}_{ij}^2\right)\right]\\ &=&-\frac{1}{2}\left(D_{ij}^2-{\overline D}_{i.}^2-{\overline D}_{.j}^2+{\overline D}_{..}^2\right) \end{eqnarray}$

${\bf X'}={\bf \Lambda_*^{1/2}}{\bf V}_*^T\in R^{{d^*}\times m}$

City block测量 $D_{rs}=\sum_i^d Minkowski测量$D_{rs}={\sum_i^dw_i
Canberra测量 $D_{rs}=\sum_i^d\frac{ Divergence $D_{rs}=\frac{1}{d}\sum_i^d\frac{(x_{ri}-x_{si})^2}{(x_{ri}+x_{si})^2}$ Bray-Curtis$D_{rs}=\frac{1}{d}\frac{\sum_i^d
Soergel \$D_{rs}=\frac{1}{d}\frac{\sum_i^d
Bhattacharyya距离 $D_{rs}=\sqrt{\sum_i^d\left(\sqrt{(x_{ri})}-\sqrt{(x_{si})}\right)^2}$
Wave-Hedges $D_{rs}=\sum_i^d\left(1-\frac{\min(x_{ri},x_{si})}{\max(x_{ri},x_{si})}\right)$
Angular separation $D_{rs}=1-\frac{\sum_i^dx_{ri}x_{si}}{\left[\sum_i^dx_{ri}^2\sum_i^dx_{si}^2\right]^{1/2}}$
Correlation $D_{rs}=1-\frac{\sum_i^d(x_{ri}-\overline x_r)(x_{si}-\overline x_s)}{\left[\sum_i^d(x_{ri}-\overline x_r)^2\sum_i^d(x_{si}-\overline x_s)^2\right]^{1/2}}$

7 - LPP

[] 周志华 机器学习
[] Michael A.A. Cox, Trevor F. Cox. Multidimensional Scaling

posted @ 2018-10-11 16:26  仙守  阅读(296)  评论(0编辑  收藏  举报