softmax

1. Softmax is a non-linear activation function generally used as the last layer of a classification model

the defination and formula of softmax is:

$S_{i}=\frac{e^{i}}{\sum_{j} e^{j}}$

 

 

(1) why it use $e^{i}$ ?

because $e^{i}$ can transform every $i$ into a possitive value.

(2) why use $frac{item}{sum}$ form?

because the function of softmax is to output probability values, it should allow:

　　① the probability value lies in [0,1]

　　② the sum of probability values is 1

 

from the following example, we can see this network use softmax layer as the last layer to output 10 probability value

from keras import models
from keras import layers

network = models.Sequential() # sequential model
network.add(layers.Dense(512, activation='relu', input_shape=(28*28,)))
network.add(layers.Dense(10, activation='softmax'))