# Bati's eHome of Tech

:: :: :: :: :: :: :: ::
 174 随笔 :: 0 文章 :: 32 评论 :: 0 引用

Define:

MLE: Maximum likelihood estimation

LSE: Least-squares estimation

SSE: Sum of squares error

_______________________________

wikipedia上说：The least-squares estimator optimizes a certain criterion (namely it minimizes the sum of the square of the residuals). 这句话解释了最小SSE和LSE之间的关系，即最小二乘估计LSE优化一种准则，称之为最小残差平方和，即SSE.

Least squares corresponds to the maximum likelihood criterion if the experimental errors have a normal distribution and can also be derived as a method of moments estimator. 这句话的意思是如果实验误差符合高斯分布，则最小二乘LSE对应于极大似然估计MLE. Method of moments 还不知道是什么东东，回头看看，有知道的ggmm也欢迎指教。

Tutorial on maximum likelihood estimation(Myung,2003)中有下列一段话：

As in MLE, ﬁnding the parameter values that minimize SSE generally requires use of a non-linear optimization algorithm. Minimization of LSE is also subject to the local minima problem, especially when the model is non-linear with respect to its parameters. The choice between the two methods of estimation can have non-trivial consequences. In general, LSE estimates tend to differ from MLE estimates, especially for data that are not normally distributed such as proportion correct and response time. An implication is that one might possibly arrive at different conclusions about the same data set depending upon which method of estimation is employed in analyzing the data. When this occurs, MLE should be preferred to LSE, unless the probability density function is unknown or difﬁcult to obtain in an easily computable form, for instance, for the diffusion model of recognition memory (Ratcliff, 1978). There is a situation, however, in which the two methods intersect. This is when observations are independent of one another and are normally distributed with a constant variance. In this case, maximization of the log-likelihood is equivalent to minimization of SSE, and therefore, the same parameter values are obtained under either MLE or LSE.

posted on 2009-05-20 11:33 Bati 阅读(...) 评论(...) 编辑 收藏