script type="text/javascript" src="http://latex.codecogs.com/latex.js">

# 第三周：逻辑回归代价函数求导过程

$J(\theta ) = -\frac{1}{m}\left[\sum_{i=1}^{m}y^{(i)}log(h_\theta (x^{(i)}))+(1-y^{(i)})log(1-h_\theta (x^{(i)})) \right]$

$J(\theta ) = -\frac{1}{m}\left[\sum_{i=1}^{m}K(\theta)\right]$

$h_\theta (x^{(i)}) = \frac{1}{1+e^{-\theta^\mathrm {T} x}}$

OK，下面开始我们的推导过程：如果要求$J(\theta)$对某一个参数$\theta$的偏导数，则：

（1）根据求导公式，可以先把常数项$-\frac{1}{m}\sum_{i=1}^{m}$提取出来，这样就只需要对求和符号内部的表达式求导，即：

$J(\theta ){}' = -\frac{1}{m}\left[\sum_{i=1}^{m}K(\theta){}'\right]$

$K(\theta){}' = \left(ylog(h_\theta (x))+(1-y)log(1-h_\theta (x))\right ){}'$（为方便显示，先把右上角表示第i个样本的上标去掉）

（2）根据对数复合求导公式，$log(x){}' = \frac{1}{x}x{}'$，对$K(\theta)$继续求导可得：

$K(\theta){}' = y\frac{1}{h_\theta (x)}h_\theta (x){}'+(1-y)\frac{1}{1-h_\theta (x)}(1-h_\theta (x)){}'$

（3）根据幂函数复合求导公式，$(y^{x}){}' = xy^{x-1}x{}'$，及以e为底的指数求导公式，对$h_\theta(x)$继续求导可得：

$h_\theta (x){}' = \left( \frac{1}{1+e^{-\theta^\mathrm {T} x}} \right){}'=-\frac{(1+e^{-\theta^\mathrm {T} x}){}'}{(1+e^{-\theta^\mathrm {T} x})^{2}} = \frac{e^{-\theta^\mathrm {T}x}(\theta^\mathrm {T} x){}'}{(1+e^{-\theta^\mathrm {T} x})^{2}} = \left(\frac{1}{1+e^{-\theta^\mathrm{T}x}}(1-\frac{1}{1+e^{-\theta^\mathrm{T}x}})\right)(\theta^\mathrm{T}x){}' = h_\theta(x)(1-h_\theta(x))(\theta^\mathrm{T}x){}'$

（4）把步骤3的结果带入步骤2，化简后可得：

$K(\theta){}' = (y-h_\theta(x))(\theta^\mathrm{T}x){}'$

$J(\theta){}' = \frac{1}{m}\left[\sum_{i=1}^{m}(h_\theta(x)-y)(\theta^\mathrm{T}x){}'\right]$

$\frac{\partial J(\theta)}{\partial \theta_{j}} = \frac{1}{m}\left[\sum_{i=1}^{m}(h_\theta(x^{(i)})-y^{(i)})x_{j}^{(i)}\right]$

posted @ 2016-12-10 10:54  玄天妙地  Views(9589)  Comments(1Edit  收藏  举报