机器学习常用公式

Linear regression-hypothesis

\[h_{\theta}(x)=\theta_{0} + \theta_{1}x\\ J(\theta)=\frac{1}{2m}\sum_{i=1}^m(h_{\theta}(x^{(i)})-y^{(i)})^2 \]

Linear Regression Gradient descent

\[\theta_j:= \theta_j-\alpha\frac{\partial}{\partial\theta_j}J(\theta)\\ Linear Regression:h_{\theta}(x)=\theta_0+\theta_1x\\ j=0=>\theta_j:=\theta_j-\alpha\frac{1}{m}\sum_{i=1}^m(h_{\theta}(x^{(i)})-y^{(i)})\\ j=1=>\theta_j:=\theta_j-\alpha\frac{1}{m}\sum_{i=1}^m(h_{\theta}(x^{(i)})-y^{(i)})x^{(i)}\\ Multiple Features:h_{\theta}(x)=\theta_0+\theta_1x_1+\theta_2x_2+...\theta_nx_n\\ 引入x_0 = 1\\ =>h_{\theta}(x) = \theta^TX\\ X = [x_0,x_1,x_2,x_3,...,x_n]其中x_0=1\\ for(j=0,1,...,n)\\ \theta_j:=\theta_j -\alpha\frac{1}{m}\sum_{i=1}^{m}(h_{\theta}(x^{(i)})-y^{(i)})x^{(i)}_j \]

Logistic Regression- hypothesis

\[h_{\theta}(x)=\frac{1}{1+e^{-\theta^Tx}}h_{\theta}(x)=\frac{1}{1+e^{-\theta^Tx}} \]

Logistic Regression - Cost Function

\[\left\{ \begin{aligned} J(\theta)&= \frac{1}{m}\sum_{i=1}^mCost(h_{\theta}(x^{(i)},y^{(i)})\\ Cost(h_{\theta}x^{(i)},y^{(i)})&=\frac{1}{2}(h_{\theta}(x^{(i)}-y^{(i)})^2\\ Cost(h_{\theta}(x^{(i)}),y^{(i)})&= \left\{ \begin{aligned} -log(h_{\theta}(x)) & &y=1\\ -log(1-h_{{\theta}}(x)) & &y=0 \end{aligned} \right.\\ \end{aligned} \right. \]

\[Cost(h_{\theta}(x),y)=-ylog(h_{\theta}(x))-(1-y)log(1-h_{\theta}(x)) \]

Regularization （Logistic Regression Cost Function）

\[J(\theta)=\frac{1}{2m}\left[\sum_{i=1}^m(h_{\theta}(x^{(i)})-y^{(i)})^2+\lambda\sum_{j=1}^n\theta_j^2\right] \]

\[J(\theta)=-\frac{1}{m}\left[\sum_{i=1}^my^{(i)}log(h_{\theta}(x^{(i)}))+(1-y^{(i)})log(1-h_{\theta}(x^{(i)}))\right]+\frac{\lambda}{2m}\sum_{j=1}^{m}\theta_j^2 \]

Gradient descent

\[\theta_j:=\theta_j-\alpha \frac{\partial}{\partial \theta_j}J(\theta) \]

\[\theta_j:=\theta_j-\alpha\frac{1}{m}\sum_{i=1}^m(h_{\theta}(x^{(i)})-y^{(i)})x_j^{(i)} \]

After Regularizing Gradient descent

\[\theta_j:=\theta_j(1-\alpha\frac{\lambda}{m})-\alpha\frac{1}{m}\sum_{i=1}^{m}(h_{\theta}(x^{(i)})-y^{(i)})x_j^{(i)} \]

Neural Network Cost Function

\[J(\Theta)=-\frac{1}{m}\left[\sum_{i=1}^m\sum _{k=1}^Ky_k^{(i)}log(h_{\Theta}(x^{(i)}))_k+(1-y^{(i)}_k)log(1-h_{\Theta}(x^{(i)}))_k\right]\\ +\frac{\lambda}{2m}\sum_{l=1}^{L-1}\sum_{i=1}^{s_l}\sum_{j=1}^{s_l+1}(\theta_{ji}^{(l)})^2 \]