Machine Learning Review_Models(McCullah Pitts Model, Linear Networks, MLPBP)

McCullah Pitts Model

---

 Perceptron Model

 

Step:

1. initialise weight value

2. present input vector and obtain net input with 

3. calculate output(with binary switch net > 0 or net <= 0)

4. update weight

5. repeat uitil no change in weights or the maximum number of iterations is reached

6. repeat steps 2-4 for all training samples

---

Perceptron Model with Multiple Neurons

Step:

1. initialise weight value

2. present input vector and obtain net input with 

3. calculate output(with binary switch net > 0 or net <= 0)

4. update weight

 

5. repeat uitil no change in weights or the maximum number of iterations is reached

6. repeat steps 2-4 for all training samples

---

 Linear Networks – ADALINE

Transfer function is linear

Learns Least-Mean- Square Error (LMSE) or Delta Rule

Weights are updated to minimise MSE between y̅ and y

Input-output relationship must be linear

Step:

1. initialise weight value

2. present input vector and obtain net input with 

3. Change in weight

 

4. Update weights:

5. repeat steps 2-4 until  or the maximum number of iterations is reached

Pros: 

1. Easy to implement
2. Can generalise which is not the case with Perceptron

Cons:

Like the Perceptrons, it cannot train patterns belonging to non-linearly separable classes
Solution – cascaded ADALINE networks - MADALINE

MADALINE – Multiple ADALINE

---

 Single Layer Networks with Nonlinear Transfer Functions

Basis for Back Propagation networks

The sigmoid transfer function is a non-linear function and it has a simple derivative

 

Alogrithm:

1. initialise weight value

2. present input vector and obtain net input with 

3. Change in weight

4. Update weights:

5. repeat steps 2-3 until no change in weights or the maximum number of iterations is reached

---

 Multi-Layer Perceptron (MLP) with Back Propagation (BP)

A MLP with BP is suitable for linearly separable inputs.

Generalising the Windrow-Hoff learning rule to ML networks and non-linear differential activation functions created the BP algorithm

Learning involves the presentation of input-output pairs, finding the mean squared error between desired and actual outputs, and adjusting the weights to reduce the difference

Learning uses a gradient descent search technique to minimise the cost function which is the MSE

Algorithms:

1. initialise weight value

2. present input vector x1(p),x2(p).....xn(p) and desired outputs yd1(p) yd2(p) ..... ydn(p) and calculate actual output:

 where n is the number of inputs of neuron j in the hidden layer, and sigmod is the sigmod activation function.

3. weight training--- update the weights in the back-propagation network propagating backward the errors associated with output neurons.

4. iteration - increase iteration p by one, go back to step 2 and repeat the process until the selected error criterion is satisfied.