高斯核函数（径向基函数）

Consider the polynomial kernel of degree 2 defined by, k(x, y) = (x^Ty)2 where x, y \in \mathbb{R}^2 and x = (x_1, x_2), y = (y_1, y_2).

Thereby, the kernel function can be written as,
k(x, y) = (x_1y_1 + x_2y_2)^2
= x_{1}^2y_{1}2 + 2x_1x_2y_1y_2 + x_{2}^2y_{2}2.
Now, let us try to come up with a feature map \Phi such that the kernel function can be written as k(x, y) = \Phi(x)^T\Phi(y).

Consider the following feature map, \Phi(x) = (x_1^2, \sqrt{2}x_1x_2, x_2^2). Basically, this feature map is mapping the points in \mathbb{R}^2 to points in \mathbb{R}^3. Also, notice that, \Phi(x)^T\Phi(y) = x_1^2y_12 + 2x_1x_2y_1y_2 + x_2^2y_22 which is essentially our kernel function.

This means that our kernel function is actually computing the inner/dot product of points in \mathbb{R}^3. That is, it is implicitly mapping our points from \mathbb{R}^2 to \mathbb{R}^3.

Exercise Question : If your points are in \mathbb{R}^n, a polynomial kernel of degree 2 will map implicitly map it to some vector space F. What is the dimension of this vector space F? Hint: Everything I did above is a clue.

Now, coming to RBF.

Let us consider the RBF kernel again for points in \mathbb{R}^2. Then, the kernel can be written as
k(x, y) = \exp(-|x - y|^2)
= \exp(- (x_1 - y_1)^2 - (x_2 - y_2)^2)
= \exp(- x_1^2 + 2x_1y_1 - y_1^2 - x_2^2 + 2x_2y_2 - y_2^2)
= \exp(-|x|^2) \exp(-|y|^2) \exp(2x^Ty)
(assuming gamma = 1). Using the taylor series you can write this as,
k(x, y) = \exp(-|x|^2) \exp(-|y|^2) \sum_{n = 0}^{\infty} \frac{(2x^Ty)n}{n!}
Now, if we were to come up with a feature map \Phi just like we did for the polynomial kernel, you would realize that the feature map would map every point in our \mathbb{R}^2 to an infinite vector. Thus, RBF implicitly maps every point to an infinite dimensional space.

http://www.quora.com/Why-does-the-RBF-radial-basis-function-kernel-map-into-infinite-dimensional-space
http://stackoverflow.com/questions/23581508/why-gaussian-radial-basis-function-maps-the-examples-into-an-infinite-dimensiona