ML:多变量代价函数和梯度下降(Linear Regression with Multiple Variables)

代价函数cost function

• 公式：
  
  其中，变量θ（Rⁿ⁺¹或者R^(n+1)*1）

• 向量化：
  

Octave实现：

function J = computeCost(X, y, theta)
%COMPUTECOST Compute cost for linear regression
% J = COMPUTECOST(X, y, theta) computes the cost of using theta as the
% parameter for linear regression to fit the data points in X and y

% Initialize some useful values
m = length(y); % number of training examples

% You need to return the following variables correctly
J = 0;

% ====================== YOUR CODE HERE ======================
% Instructions: Compute the cost of a particular choice of theta
% You should set J to the cost.

prediction=Xtheta;
sqerror=(prediction-y).^2;
J=1/(2m)*sum(sqerror)

% =========================================================================

end

多变量梯度下降（gradient descent for multiple variable）

• 公式：
  
  也即，
  
• 矩阵化：
  梯度下降可以表示为，
  
  其中，为，
  
  其中微分可以求得，
  
  将其向量化后，
  
  则最终的梯度下降的矩阵化版本，
  

Octave版本：

function [theta, J_history] = gradientDescent(X, y, theta, alpha, num_iters)
%GRADIENTDESCENT Performs gradient descent to learn theta
% theta = GRADIENTDESCENT(X, y, theta, alpha, num_iters) updates theta by
% taking num_iters gradient steps with learning rate alpha

% Initialize some useful values
m = length(y); % number of training examples
J_history = zeros(num_iters, 1);

for iter = 1:num_iters

% ====================== YOUR CODE HERE ======================
% Instructions: Perform a single gradient step on the parameter vector
% theta.
%
% Hint: While debugging, it can be useful to print out the values
% of the cost function (computeCost) and gradient here.
%

predictions=Xtheta;
updates=X'(predictions-y);
theta=theta-alpha(1/m)updates;

% ============================================================

% Save the cost J in every iteration
J_history(iter) = computeCost(X, y, theta);

end

end