singular value decomposition for PCA

sigular value decomposition

$A\epsilon R^{m*n}$, A can be decomposed into $A=UDV^T$, where $U\epsilon R^{m*n}, D\epsilon R^{n*n}, V\epsilon R^{n*n}$

$D=diagonal[\sigma_i], \sigma_i=sigular value of A$; $U$'s columns are Eigenvectors of $AA^T$; $V$'s columns are Eigenvectors of $A^TA$

compute using "svd" in Matlab.

we can use it to compute Eigenvectors and PCA efficiently

$\Sigma=\sum_{i=1}^m x^{(i)}x^{(i)T}$

X is design matrix, $X=[x^{(1)}; x^{(2)}......;x^{(m)}]$

$\Sigma=X^TX$

to get top k eigenvectors, decomposite X: $X=UDV^T$, top k columns of V is top k eigenvectors of $X^TX=\Sigma$

When dealing with very high dimension data, use svd to compute eigenvectors. If dimension of data is not quite high, ordinary methods would do.