LS

Stats OS DOCUMENT:
The statistical model is assumed to be

Y=Xβ+μ, where μ∼N(0,Σ).
Depending on the properties of Σ, we have currently four classes available:

GLS : generalized least squares for arbitrary covariance
OLS : ordinary least squares for i.i.d. errors
WLS : weighted least squares for heteroskedastic errors
GLSAR : feasible generalized least squares with autocorrelated AR(p) errors
\(\sigma\)
\(\Sigma\)是一对角阵，且对角元素不完全相同，则y是不相关的，且有不相等的方差。
\(\Sigma\)是一非对角阵，则是相关的
\(\beta\)
由于\(\Sigma\)是误差\(y-X \beta\)的协方差矩阵，所以\(\Sigma\)是正定非奇异矩阵，可进行矩阵分解为
K'K=\(\Sigma\)。令\(z=K^{-1},B=K^{-1}X,g=K^{-1}\epsilon\)
\(y=X\beta+\epsilon\)，\(K^{-1}y=K^{-1}X\beta+K^{-1}\epsilon\),\(z=B\beta+g\)
其中\(E(g)=E(K^{-1}\epsilon)=K^{-1}E(\epsilon)=0\)

\[Var(g)=E(g^2)-E(g)^2=E(g^2)=E(gg')=E(K^{-1}\epsilon\epsilon 'K^{-1}) =K^{-1}E(\epsilon\epsilon ')K^{-1}=K^{-1}\Sigma K^{-1}=I\]

所以\(z=B\beta+g\)可用标准最小二乘求解
$ min \quad S(\beta)=g'g=\epsilon'V^{{-1}\epsilon=(y-X\beta)'V}(y-X\beta)\(求导得： \)\hat{\beta} = (X'V^{-1}X)X'V^{-1}y$

对于WLS,
另\(W= V^{-1}\)，得\(\hat{\beta}=(X'WX)^{-1}X'Wy\)

ALS
X=CS'(mxn=mxp pxn)m是样本数，n是变量数，p是因子数
init C0
C=C0
for i = 1:iter
S'=X\C
非负处理S'，S'=Dpinv(DC'CD)C'X
C=X\S'
非负处理C C=Dpinv(DS'SD)S'X'
end

pls
XB=Y(mxn nxk = mxk) m是样本数，n是自变量数,k是因变量数
xpp=process(x)
ypp=process(y)