线性回归——代价函数

Training Set

训练集

Size in feet²(x)	Price in 1000's(y)
2104	460
1416	232
1534	315
852	178

Hypothesis:

\[{h_\theta }\left( x \right) = {\theta _0} + \theta {x}\]

Notation:

θ_i's: Parameters

θ_i's: 参数

How to choose θ_i's?

如何选择θ_i's？

Idea: Choose θ₀, θ₁so that h(x) is close to y for our training examples(x, y)

思想：对于训练样本（x, y）来说，选择θ_0，θ₁ 使h(x) 接近y。

minimize（θ₀, θ₁）\[\sum\limits_{i = 1}^m {{{\left( {{h_\theta }\left( {{x^{(i)}}} \right) - {y^i}} \right)}^2}} \]

选择合适的（θ₀, θ₁）使得 \[\sum\limits_{i = 1}^m {{{\left( {{h_\theta }\left( {{x^{(i)}}} \right) - {y^i}} \right)}^2}} \] 最小。

为了使公式的数学意义更好，将公式改为 \[\frac{1}{{2m}}\sum\limits_{i = 1}^m {{{\left( {{h_\theta }\left( {{x^{(i)}}} \right) - {y^i}} \right)}^2}} \]

这并不影响（θ₀, θ₁）的取值。

定义代价函数（Cost function） \[J\left( {{\theta _0},{\theta _1}} \right) = \frac{1}{{2m}}\sum\limits_{i = 1}^m {{{\left( {{h_\theta }\left( {{x^{(i)}}} \right) - {y^i}} \right)}^2}} \]

目标是 \[\mathop {\min imize}\limits_{{\theta _0},{\theta _1}} J\left( {{\theta _0},{\theta _1}} \right)\]

这个代价函数也称为平方误差代价函数（Squared error function）

总结：

Hypothesis: \[{h_\theta }\left( x \right) = {\theta _0} + \theta {x}\]

Parameters: （θ₀, θ₁）

Cost Functions: \[J\left( {{\theta _0},{\theta _1}} \right) = \frac{1}{{2m}}\sum\limits_{i = 1}^m {{{\left( {{h_\theta }\left( {{x^{(i)}}} \right) - {y^i}} \right)}^2}} \]

Goal: \[\mathop {\min imize}\limits_{{\theta _0},{\theta _1}} J\left( {{\theta _0},{\theta _1}} \right)\]

例子帮助理解

首先令 θ₀=0，则代价函数变为 \[J\left( {{\theta _1}} \right) = \frac{1}{{2m}}\sum\limits_{i = 1}^m {{{\left( {{h_\theta }\left( {{x^{(i)}}} \right) - {y^i}} \right)}^2}} \]

h_θ(x)	J(θ₁)
对于给定θ₁的情况，它是x的函数	是θ₁的函数

三个训练样本

x	y
1	1
2	2
3	3

当θ₁=1时，\[J\left( {{\theta _1}} \right) = \frac{1}{{2m}}\left( {{0^2} + {0^2} + {0^2}} \right) = 0\]

当θ₁=0.5时，\[J\left( {0.5} \right) = \frac{1}{{2*3}}\left( {{{\left( {0.5 - 1} \right)}^2} + {{\left( {{\rm{1 - 2}}} \right)}^2} + {{\left( {{\rm{1}}{\rm{.5 - 3}}} \right)}^2}} \right) \approx {\rm{0}}{\rm{.58}}\]