逻辑回归——梯度下降

逻辑回归的代价函数

\[J\left( \theta  \right) =  - \frac{1}{m}[\sum\limits_{i = 1}^m {{y^{\left( i \right)}}\log \left( {{h_\theta }\left( {{x^{\left( i \right)}}} \right)} \right) + \left( {1 - {y^{\left( i \right)}}} \right)\log \left( {1 - {h_\theta }\left( {{x^{\left( i \right)}}} \right)} \right)]} \]

梯度下降算法

 重复{

\[{\theta _j}: = {\theta _j} - \alpha \frac{\partial }{{\partial {\theta _j}}}J\left( \theta  \right)\]

(同步更新所有的tθ)

}

其中

\[\frac{\partial }{{\partial {\theta _j}}}J\left( \theta  \right) = \sum\limits_{i = 1}^m {\left( {{h_\theta }\left( {{x^{\left( i \right)}}} \right) - {y^{\left( i \right)}}} \right)} {x^{\left( i \right)}}\]

所以梯度下降算法

重复{

\[{\theta _j}: = {\theta _j} - \alpha \sum\limits_{i = 1}^m {\left( {{h_\theta }\left( {{x^{\left( i \right)}}} \right) - {y^{\left( i \right)}}} \right)} {x^{\left( i \right)}}\]

}

乍一看这和线性回归中的梯度下降算法一样，其实不然

线性回归中\[{h_\theta }(x) = {\theta ^T}x\]

逻辑回归中\[{h_\theta }(x) = \frac{1}{{1 + {e^{ - {\theta ^T}x}}}}\]

---

逻辑回归也需要对数据进行标准化