# 拉格朗日乘子法和KKT条件

### 拉格朗日乘子法

$\min\;f(x)\\s.t.\;h_{i}(x)=0\;\;\;\;i=1,2...,n$

$$min\;[f(x)+\sum_{i=1}^{n}\lambda_{i}h_{i}(x)]$$

$$\min\;f(x,y)\\s.t.\;g(x,y)=c\label{eg1}$$

$$\bigtriangledown[f(x,y)+\lambda(g(x,y)-c)]=0\;\;\;\;\lambda\ne{0}\label{grad}$$

$$min\ F(x,y)=f(x,y)+\lambda(g(x,y)-c)\label{F}$$

## KKT条件

$$let\;L(x,\mu)=f(x)+\sum_{k=1}^q\mu_{k}g_{k}(x)$$

$\because \left.\begin{matrix}\mu_{k}\ge{0}\\g_{k}(x)\le{0}\end{matrix}\right\}$=>$\mu{g(x)}\le{0}$

$\therefore$ $$\max_{\mu}L(x,\mu)=f(x)\label{a}$$

$\therefore$$$\min_{x}f(x)=\min_{x}\max_{\mu}L(x,\mu)\label{firsthalf}$$

$\max_{\mu}\min_{x}L(x,\mu)=\max_{\mu}[\min_{x}f(x)+\min_{x}\mu{g(x)}]=\max_{\mu}\min_{x}f(x)+\max_{\mu}\min_{x}\mu{g(x)}=\min_{x}f(x)+\max_{\mu}\min_{x}\mu{g(x)}$

$\therefore \max_{\mu}\min_{x}\mu{g(x)}=0$此时$\mu=0\;or\;g(x)=0$

$$\therefore \max_{\mu}\min_{x}L(x,\mu)=\min_{x}f(x)+\max_{\mu}\min_{x}\mu{g(x)}=\min_{x}f(x)\label{secondhalf}$$此时$\mu=0\;or\;g(x)=0$

$\left.\begin{matrix}L(x,\mu)=f(x)+\sum_{k=1}^q\mu_{k}g_{k}(x)\\\mu_{k}\ge{0}\\g_{k}(x)\le{0}\end{matrix}\right\}$=>$\left\{\begin{matrix}\min_{x}\max_{\mu}L(x,\mu)=\max_{\mu}\min_{x}L(x,\mu)=\min_{x}f(x)=f(x^*)\\\mu_{k}{g_{k}(x^*)=0}\\\frac{\partial{L(x,\mu)}}{\partial{x}}|_{x=x^*}=0\end{matrix}\right.$

KKT条件是拉格朗日乘子法的泛化，如果我们把等式约束和不等式约束一并纳入进来则表现为：

$\left.\begin{matrix}L(x,\lambda,\mu)=f(x)+\sum_{i=1}^{n}\lambda_{i}h_{i}(x)+\sum_{k=1}^q\mu_{k}g_{k}(x)\\\lambda_{i}\ne{0}\\h_{i}(x)=0\\\mu_{k}\ge{0}\\g_{k}(x)\le{0}\end{matrix}\right\}$=>$\left\{\begin{matrix}\min_{x}\max_{\mu}L(x,\lambda,\mu)=\max_{\mu}\min_{x}L(x,\lambda,\mu)=\min_{x}f(x)=f(x^*)\\\mu_{k}{g_{k}(x^*)=0}\\\frac{\partial{L(x,\lambda,\mu)}}{\partial{x}}|_{x=x^*}=0\end{matrix}\right.$

$\frac{\partial{L(x,\lambda,\mu)}}{\partial{x}}|_{x=x^*}=0$表明$f(x)$在极值点$x^*$处的梯度是各个$h_{i}(x^*)$和$g_{k}(x^*)$梯度的线性组合。

posted @ 2012-10-27 19:49 Orisun 阅读(...) 评论(...)  编辑 收藏