Rosenblatt's perceptron

Rosenblatt's perceptron

 

 

 

 

 

Page 19 z = w0x0+w1x1+ ... + wmxm= ∑mj0xjwj=wTx

In [59]:

import numpy as np 

class Perceptron(object):
    """
    Perceptron classifier.
    Parameters 
    -----------
    eta : float  Learning rate (between 0.0 and 1.0) 

    n_iter : int Passes over the training dataset.

    Attributes 
    ----------
    w_ : 1d-array  Weights after fitting. 

    errors_ : list  Number of misclassifications in every epoch.
    """ 
    def __init__(self, eta=0.01, n_iter=10):
        self.eta = eta
        self.n_iter = n_iter

    def fit(self, X, y):
        """
        Fit training data.
        Parameters 
        ---------
        X : {array-like}, shape = [n_samples, n_features]   
        Training vectors, where n_samples is the number of samples and n_features is the number of features. 

        y : array-like, shape = [n_samples] Target values. 

        Returns 
        ------
        self : object
        """ 
        self.w_ = np.zeros(1 + X.shape[1]) 
        self.errors_ = []

        for _ in range(self.n_iter):
            errors = 0 
            for xi, target in zip(X, y):
                update = self.eta * (target - self.predict(xi)) 
                self.w_[1:] += update * xi 
                self.w_[0] += update 
                errors += int(update != 0.0) 
            self.errors_.append(errors) 
        return self

    def net_input(self, X):
        """Calculate net input""" 
        return np.dot(X, self.w_[1:]) + self.w_[0]

    def predict(self, X):
        """Return class label after unit step""" 
        return np.where(self.net_input(X) >= 0.0, 1, -1)

In [56]:

import pandas as pd
df = pd.read_csv('iris.data', header=None)
print(df.tail())

 

       0    1    2    3               4
145  6.7  3.0  5.2  2.3  Iris-virginica
146  6.3  2.5  5.0  1.9  Iris-virginica
147  6.5  3.0  5.2  2.0  Iris-virginica
148  6.2  3.4  5.4  2.3  Iris-virginica
149  5.9  3.0  5.1  1.8  Iris-virginica

 

we extract the first 100 class labels that correspond to the 50 Iris-Setosa and 50 Iris-Versicolor flowers, respectively, and convert the class labels into the two integer class labels 1 (Versicolor) and -1 (Setosa) that we assign to a vector y where the values method of a pandas DataFrame yields the corresponding NumPy representation. Similarly, we extract the first feature column (sepal length) and the third feature column (petal length) of those 100 training samples and assign them to a feature matrix X, which we can visualize via a two-dimensional scatter plot:

In [57]:

import matplotlib.pyplot as plt
y = df.iloc[0:100, 4].values
y = np.where( y=='Iris-setosa',-1,1)
X = df.iloc[0:100, [0, 2]].values
plt.scatter(X[:50, 0], X[:50, 1], color='red', marker='o', label='setosa')
plt.scatter(X[50:100, 0], X[50:100, 1], color='blue', marker='x', label='versicolor')
plt.xlabel('sepal length')
plt.ylabel('petal length')
plt.legend(loc='upper left')
plt.show()

 

In [60]:

ppn = Perceptron(eta=0.1, n_iter=10)
ppn.fit(X, y)
plt.plot(range(1, len(ppn.errors_) + 1), ppn.errors_,marker='o')  
plt.xlabel('Epochs')
plt.ylabel('Number of misclassifications')
plt.show()

 

 

we should see the plot of the misclassification errors versus the number of epochs

In [69]:

from matplotlib.colors import ListedColormap

def plot_decision_regions(X, y, classifier, resolution=0.02):

# setup marker generator and color map 
    markers = ('s', 'x', 'o', '^', 'v') 
    colors = ('red', 'blue', 'lightgreen', 'gray', 'cyan')
    cmap = ListedColormap(colors[:len(np.unique(y))])

# plot the decision surface 
    x1_min, x1_max = X[:, 0].min() - 1, X[:, 0].max() + 1 
    x2_min, x2_max = X[:, 1].min() - 1, X[:, 1].max() + 1 
    xx1, xx2 = np.meshgrid(np.arange(x1_min, x1_max, resolution), np.arange(x2_min, x2_max, resolution)) 
    Z = classifier.predict(np.array([xx1.ravel(), xx2.ravel()]).T) 
    Z = Z.reshape(xx1.shape) 
    plt.contourf(xx1, xx2, Z, alpha=0.4, cmap=cmap) 
    plt.xlim(xx1.min(), xx1.max()) 
    plt.ylim(xx2.min(), xx2.max())
    # plot class samplesc=cmap(idx),,
    for idx, cl in enumerate(np.unique(y)):
        plt.scatter(x=X[y == cl, 0], y=X[y == cl, 1],  marker=markers[idx],alpha=0.8,   label=cl)

In [70]:

plot_decision_regions(X, y, classifier=ppn)
plt.xlabel('sepal length [cm]')
plt.ylabel('petal length [cm]')
plt.legend(loc='upper left')
plt.show()

 

 

Although the perceptron classified the two Iris flower classes perfectly, convergence is one of the biggest problems of the perceptron. Frank Rosenblatt proved mathematically that the perceptron learning rule converges if the two classes can be separated by a linear hyperplane. However, if classes cannot be separated perfectly by such a linear decision boundary, the weights will never stop updating unless we set a maximum number of epochs.