# Laplace（拉普拉斯）先验与L1正则化

## 最大似然估计

$f(X) = \sum_i(x_i\theta_i) + \epsilon = X\theta^T + \epsilon \tag{1.1}$

$P(Y_i|X_i, \theta) = \frac{1}{\delta\sqrt{2\pi}} \exp(-\frac{\|f(X_i) - Y_i\|^2}{2\delta^2}) \tag{1.2}$

$P(Y|X,\theta)= \prod_i\frac{1}{\delta\sqrt{2\pi}} \exp(-\frac{\|f(X_i) - Y_i\|^2}{2\delta^2}) \tag{1.3}$

$\begin {split} \theta^* &= argmax_{\theta} \left(\prod_i\frac{1}{\epsilon\sqrt{2\pi}} \exp(-\frac{\|f(X_i) - Y_i\|^2}{2\delta^2})\right) \cr &=argmax_{\theta} \left( -\frac{1}{2\delta^2} \sum_i \|f(X_i) - Y_i\|^2 + \sum_i ln(\delta\sqrt{2\pi}) \right) \cr &=argmin_{\theta} \left(\sum_i \|f(X_i) - Y_i\|^2 \right) \end {split} \tag{1.4}$

## Laplace分布

Laplace概率密度函数分布为：

$f(x|\mu, b) = \frac{1}{2b} \exp(-\frac{|x-\mu|}{b}) \tag{2.1}$

## Laplace先验导出L1正则化

$P(\theta_i) = \frac{\lambda}{2} \exp(-\lambda|\theta_i|) \tag{3.1}$

$\begin {split} \theta^* &= argmax_{\theta} \left(\prod_i P(Y_i|X_i, \theta) \prod_i P(\theta_i)\right) \cr &=argmin_{\theta} \left(\sum_i \|f(X_i) - Y_i\|^2 + \sum_i \ln(P(\theta_i))\right) \end {split} \tag{3.2}$

$\theta^* =argmin_{\theta} \left(\sum_i \|f(X_i) - Y_i\|^2 + \lambda \sum_i |\theta_i|)\right) \tag{3.3}$

### Gauss先验导出L2正则化

$P(\theta_i) = \frac{\lambda}{\sqrt{\pi}} \exp(-\lambda\|\theta_i\|^2) \tag{3.4}$

$\theta^* =argmin_{\theta} \left(\sum_i \|f(X_i) - Y_i\|^2 + \lambda \sum_i \|\theta_i\|^2)\right) \tag{3.5}$

【防止爬虫转载而导致的格式问题——链接】：
http://www.cnblogs.com/heguanyou/p/7688344.html

posted @ 2017-10-18 19:17  Thaurun  阅读(22182)  评论(5编辑  收藏  举报