Matrix Factorization SVD 矩阵分解

Today we have learned the Matrix Factorization, and I want to record my study notes. Some kownledge which I have learned before is forgot...(呜呜)

1.Terminology

单位矩阵:identity matrix

特征值:eigenvalues

特征向量:eigenvectors

矩阵的秩:rank

对角矩阵:diagonal matrix

对角化矩阵:Diagonalizing a Matrix

矩阵分解:matrix factorization

奇异值分解:SVD(singular value decomposition)

2.Basic kowledge

<1>Eigenvalues and Eigenvectors

What: The basic equation is , the x is the eigenvector of A and the lamda is the eigenvalue of A.

How: We can transpose this equation as (I is identity matrix), so we will kown the .  We can calculate the eigenvalues and then get the eigenvectors.

<2>Diagonalizing a Matrix

What: (大家都知道,但是特别注意下形如下面的两种矩阵也是对角矩阵)

 

How: ,lamada is the eigenvalue of A, and the column of S is the eigenvector of A. Like follow:

<3>rank

  • The Rank and the Row Reduced Form (注:我们知道矩阵秩的定义或者求法有很多种,这里说的是行/列最简形矩阵的行/列数即为矩阵的秩,或者就是矩阵的最大非零r阶子式,则r称为矩阵的秩,即R(A)=r )
  • If the rank of matrix A(n*n) is r, what it's mean. 矩阵A的列向量或者行向量只有r个是非线性相关的,其他的n-r个向量是无价值的。(这个很重要,下面矩阵分解将会用到,自己的感悟不会用英文表达了,用中文。。。)

<4>singular value

3. Matrix Factorization

 What:

(注:这里注意,当k>=m或k>=n且矩阵U或者V是满秩的,矩阵无法分解)

Why: We can use this to 

  • Image Recovery

(recovery this image)

  • Recommendation

(evaluate the ?)

  • and so on

How: 

<1>Matrix completation

(注:在这里我们需要做出假设,即矩阵是低秩。为什么呢?在上面基础概念中我们说到了矩阵秩所代表的意义,那么这里我们假设矩阵是低秩的,则说明矩阵的列向量只有少数几列是真正重要的,其他的都是和这几列线性相关的,那么我们就可以通过这种线性相关来补全残缺的矩阵)

 

<2>SVD is one of the methods of matrix factorization, we will introduce this method below.

We have discussed the Diagonalizing a Matrix,but when A is any m by n matrix, square or rectangular. Its rank is r. We will diagonalize this A, but not by . The eigenvectors in S have three big problems: They are usually not orthogonal, there are not always enough eigenvectors, and  requires A to be square. The singular vectors of A solve all those problems in a perfect way.

singular value:(这里有一篇博文说的很好,推荐下

通过上面的方法,能够对矩阵X做出一定的降秩,但是如果这里d与m或者n较为接近,那么降秩的效果就不明显,所以我们使用一种近似的策略,如下:

这样我们就是实现了将一个矩阵A,经过SVD近似,转换成一个同维度的矩阵A*,但是它的秩远远低于A

to be continued...

posted @ 2013-10-23 11:00  xlturing  阅读(2525)  评论(0编辑  收藏  举报