ML-2Gradient Descent

The gradient descent algorithm is:

repeat until convergence:

where

j=0,1 represents the feature index number.

Tip1：同时更新

Repeat until convergence:

 

Tip2:Regardless of the slope's sign for     eventually converges to its minimum value.

The following graph shows that when the slope is negative, the value of increase，and when it is positive, decrease

Tip3:

At the minimum, the derivative will always be 0 and thus we get:

 

Tip4:步长选择，不能太大也不要太小