# Matlab实现线性回归和逻辑回归: Linear Regression & Logistic Regression

Matlab 实现各种回归函数

=========================

Y=θ0+θ1X1型---线性回归（直线拟合）

Y=1/(1+e^X)型---逻辑回归（sigmod 函数拟合）

=========================

①   cost function（指出真实值y与拟合值h<hypothesis>之间的距离）：给出cost function 的表达式，每次迭代保证cost function的量减小；给出梯度gradient，即cost function对每一个参数θ的求导结果。

function [ jVal,gradient ] = costFunction ( theta )

cost function对各θj的求导请自行求取，看第三章最后一图，或者参见后文代码。

cost function：
 1 function [ jVal,gradient ] = costFunction2( theta )
2 %COSTFUNCTION2 Summary of this function goes here
3 %   linear regression -> y=theta0 + theta1*x
4 %   parameter: x:m*n  theta:n*1   y:m*1   (m=4,n=1)
5 %
6
7 %Data
8 x=[1;2;3;4];
9 y=[1.1;2.2;2.7;3.8];
10 m=size(x,1);
11
12 hypothesis = h_func(x,theta);
13 delta = hypothesis - y;
14 jVal=sum(delta.^2);
15
18
19 end

1 function [res] = h_func(inputx,theta)
2 %H_FUNC Summary of this function goes here
3 %   Detailed explanation goes here
4
5
6 %cost function 2
7 res= theta(1)+theta(2)*inputx;function [res] = h_func(inputx,theta)
8 end

1 function [optTheta,functionVal,exitFlag]=Gradient_descent( )
2 %GRADIENT_DESCENT Summary of this function goes here
3 %   Detailed explanation goes here
4
6   initialTheta = zeros(2,1);
7   [optTheta,functionVal,exitFlag] = fminunc(@costFunction2,initialTheta,options);
8
9 end

result：

 1 >> [optTheta,functionVal,exitFlag] = Gradient_descent()
2
3 Local minimum found.
4
5 Optimization completed because the size of the gradient is less than
6 the default value of the function tolerance.
7
8 <stopping criteria details>
9
10
11 optTheta =
12
13     0.3000
14     0.8600
15
16
17 functionVal =
18
19     0.0720
20
21
22 exitFlag =
23
24      1

 1 function [ parameter ] = checkcostfunc(  )
2 %CHECKC2 Summary of this function goes here
3 %   check if the cost function works well
4 %   check with the matlab fit function as standard
5
6 %check cost function 2
7 x=[1;2;3;4];
8 y=[1.1;2.2;2.7;3.8];
9
10 EXPR= {'x','1'};
11 p=fittype(EXPR);
12 parameter=fit(x,y,p);
13
14 end

1 >> checkcostfunc()
2
3 ans =
4
5      Linear model:
6      ans(x) = a*x + b
7      Coefficients (with 95% confidence bounds):
8        a =        0.86  (0.4949, 1.225)
9        b =         0.3  (-0.6998, 1.3)

 1 function PlotFunc( xstart,xend )
2 %PLOTFUNC Summary of this function goes here
3 %   draw original data and the fitted
4
5
6
7 %===================cost function 2====linear regression
8 %original data
9 x1=[1;2;3;4];
10 y1=[1.1;2.2;2.7;3.8];
11 %plot(x1,y1,'ro-','MarkerSize',10);
12 plot(x1,y1,'rx','MarkerSize',10);
13 hold on;
14
15 %fitted line - 拟合曲线
16 x_co=xstart:0.1:xend;
17 y_co=0.3+0.86*x_co;
18 %plot(x_co,y_co,'g');
19 plot(x_co,y_co);
20
21 hold off;
22 end

 1 function [ jVal,gradient ] = costFunction( theta )
2 %COSTFUNCTION Summary of this function goes here
3 %   Detailed explanation goes here
4
5 jVal= (theta(1)-5)^2+(theta(2)-5)^2;
6
8 %code to compute derivative to theta
9 gradient(1) = 2 * (theta(1)-5);
10 gradient(2) = 2 * (theta(2)-5);
11
12 end

1 function [optTheta,functionVal,exitFlag]=Gradient_descent( )
2 %GRADIENT_DESCENT Summary of this function goes here
3 %   Detailed explanation goes here
4
6  initialTheta = zeros(2,1)
7  [optTheta,functionVal,exitFlag] = fminunc(@costFunction,initialTheta,options);
8
9 end

matlab主窗口中调用，得到优化厚的参数(θ1,θ2)=(5,5)

 1  [optTheta,functionVal,exitFlag] = Gradient_descent()
2
3 initialTheta =
4
5      0
6      0
7
8
9 Local minimum found.
10
11 Optimization completed because the size of the gradient is less than
12 the default value of the function tolerance.
13
14 <stopping criteria details>
15
16
17 optTheta =
18
19      5
20      5
21
22
23 functionVal =
24
25      0
26
27
28 exitFlag =
29
30      1

hypothesis function:

1 function [res] = h_func(inputx,theta)
2
3 %cost function 3
4 tmp=theta(1)+theta(2)*inputx;%m*1
5 res=1./(1+exp(-tmp));%m*1
6
7 end

cost function:

 1 function [ jVal,gradient ] = costFunction3( theta )
2 %COSTFUNCTION3 Summary of this function goes here
3 %   Logistic Regression
4
5 x=[-3;      -2;     -1;     0;      1;      2;     3];
6 y=[0.01;    0.05;   0.3;    0.45;   0.8;    1.1;    0.99];
7 m=size(x,1);
8
9 %hypothesis  data
10 hypothesis = h_func(x,theta);
11
12 %jVal-cost function  &  gradient updating
13 jVal=-sum(log(hypothesis+0.01).*y + (1-y).*log(1-hypothesis+0.01))/m;
15 gradient(2)=sum((hypothesis-y).*x)/m;    %reflect to theta 2
16
17 end

1 function [optTheta,functionVal,exitFlag]=Gradient_descent( )
2
4  initialTheta = [0;0];
5  [optTheta,functionVal,exitFlag] = fminunc(@costFunction3,initialTheta,options);
6
7 end

 1  [optTheta,functionVal,exitFlag] = Gradient_descent()
2
3 Local minimum found.
4
5 Optimization completed because the size of the gradient is less than
6 the default value of the function tolerance.
7
8 <stopping criteria details>
9
10
11 optTheta =
12
13     0.3526
14     1.7573
15
16
17 functionVal =
18
19     0.2498
20
21
22 exitFlag =
23
24      1

 1 function PlotFunc( xstart,xend )
2 %PLOTFUNC Summary of this function goes here
3 %   draw original data and the fitted
4
5 %===================cost function 3=====logistic regression
6
7 %original data
8 x=[-3;      -2;     -1;     0;      1;      2;     3];
9 y=[0.01;    0.05;   0.3;    0.45;   0.8;    1.1;    0.99];
10 plot(x,y,'rx','MarkerSize',10);
11 hold on
12
13 %fitted line
14 x_co=xstart:0.1:xend;
15 theta = [0.3526,1.7573];
16 y_co=h_func(x_co,theta);
17 plot(x_co,y_co);
18 hold off
19
20 end

posted @ 2015-12-14 22:42  莫小  阅读(5414)  评论(0编辑  收藏