Activation Functions and Their Derivatives

1. Sigmoid Function: when z=0,g'(z)=0.25

2. tanh Function: when x=0,tanh'(x)=1

3. Relu