因子分析与EM算法
Factor Analysis
Background
when m (number of samples) < n (dimension of samples), the covariance matrix might be singular. So it is necessary to find a way without having to fit a full covariance matrix.
Marginal and Conditional Distribution of Gaussian
\[x = [x1,x2]^T
\]
\(x \sim \mathcal{N}(\mu,\Sigma)\)
marginal distribution \(x_1 \sim \mathcal{N}(\mu_1,\Sigma_{11})\)
conditional distribution $x_1|x_2 \sim \mathcal{N}(\mu_{1|2},\Sigma_{1|2}) $ and
\[\mu_{1|2} = \mu_1 + \Sigma_{12}\Sigma_{22}^{-1}(x_2-\mu_2)
\\
\Sigma_{1|2}=\Sigma_{11}-\Sigma_{12}\Sigma_{22}^{-1}\Sigma_{21}
\]
Factor Analysis Model
\[z \sim \mathcal{N}(0,I)
\\
x|z \sim \mathcal{N}(\mu+\Lambda z,\Psi)
\]
