斯坦福CS231n课程作业# 1（代码笔记1）

k-Nearest Neighbor (kNN) exercise

# Run some setup code for this notebook.

import random
import numpy as np
from cs231n.data_utils import load_CIFAR10
import matplotlib.pyplot as plt

# This is a bit of magic to make matplotlib figures appear inline in the notebook
# rather than in a new window.
%matplotlib inline
plt.rcParams['figure.figsize'] = (10.0, 8.0) # set default size of plots
plt.rcParams['image.interpolation'] = 'nearest'
plt.rcParams['image.cmap'] = 'gray'

# Some more magic so that the notebook will reload external python modules;
# see http://stackoverflow.com/questions/1907993/autoreload-of-modules-in-ipython
%load_ext autoreload
%autoreload 2

# Load the raw CIFAR-10 data.
cifar10_dir = 'cs231n/datasets/cifar-10-batches-py'

# Cleaning up variables to prevent loading data multiple times (which may cause memory issue)
try:
   del X_train, y_train
   del X_test, y_test
   print('Clear previously loaded data.')
except:
   pass

X_train, y_train, X_test, y_test = load_CIFAR10(cifar10_dir)

# As a sanity check, we print out the size of the training and test data.
print('Training data shape: ', X_train.shape)
print('Training labels shape: ', y_train.shape)
print('Test data shape: ', X_test.shape)
print('Test labels shape: ', y_test.shape)

# Visualize some examples from the dataset.
# We show a few examples of training images from each class.
classes = ['plane', 'car', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']
num_classes = len(classes)#类别数目
samples_per_class = 7 # 每个类别采样个数
#enumerate() 函数用于将一个可遍历的数据对象(如列表、元组或字符串)组合为一个索引序列，同时列出数据和数据下标，一般用在 for 循环当中。
for y, cls in enumerate(classes):
    #输入一个矩阵，返回了其中非零元素的位置.找出标签中y类的位置
    idxs = np.flatnonzero(y_train == y) 
    
    '''choice(a, size=None, replace=True, p=None)表示从a中随机选取size个数,
    replacement 代表的意思是抽样之后还放不放回去，如果是False的话，那么通一次挑选出来的数都不一样，
    如果是True的话， 有可能会出现重复的，因为前面的抽的放回去了。
    p表示每个元素被抽取的概率，如果没有指定，a中所有元素被选取的概率是相等的。
    '''
    idxs = np.random.choice(idxs, samples_per_class, replace=False) #从中选出我们所需的7个样本
    for i, idx in enumerate(idxs): #对所选的样本的位置和样本所对应的图片在训练集中的位置进行循环
        plt_idx = i * num_classes + y + 1 # 在子图中所占位置的计算
        #前两个数表示子图组成的矩阵的行列数，比如有6个子图，排列成3行2列，那就是subplot(3,2,X)。最后一个数表示要画第X个图了。
        plt.subplot(samples_per_class, num_classes, plt_idx) # 说明要画的子图的编号
        plt.imshow(X_train[idx].astype('uint8')) # 画图
        plt.axis('off')
        if i == 0:
            plt.title(cls) # 写上标题，也就是类别名
plt.show()

# Subsample the data for more efficient code execution in this exercise
num_training = 5000
mask = list(range(num_training))
X_train = X_train[mask]
y_train = y_train[mask]

num_test = 500
mask = list(range(num_test))
X_test = X_test[mask]
y_test = y_test[mask]
print(X_train.shape, X_test.shape)
# Reshape the image data into rows
X_train = np.reshape(X_train, (X_train.shape[0], -1))#也可以用X_train.reshape( (X_train.shape[0], -1))，-1代表后面多维数据整合成一个一维数据
X_test = np.reshape(X_test, (X_test.shape[0], -1))
print(X_train.shape, X_test.shape)

from cs231n.classifiers import KNearestNeighbor

# Create a kNN classifier instance. 
# Remember that training a kNN classifier is a noop: 
# the Classifier simply remembers the data and does no further processing 
classifier = KNearestNeighbor()#创建一个实例
classifier.train(X_train, y_train)

We would now like to classify the test data with the kNN classifier. Recall that we can break down this process into two steps:

1. First we must compute the distances between all test examples and all train examples.
2. Given these distances, for each test example we find the k nearest examples and have them vote for the label

Lets begin with computing the distance matrix between all training and test examples. For example, if there are Ntr training examples and Nte test examples, this stage should result in a Nte x Ntr matrix where each element (i,j) is the distance between the i-th test and j-th train example.

Note: For the three distance computations that we require you to implement in this notebook, you may not use the np.linalg.norm() function that numpy provides.

First, open cs231n/classifiers/k_nearest_neighbor.py and implement the function compute_distances_two_loops that uses a (very inefficient) double loop over all pairs of (test, train) examples and computes the distance matrix one element at a time.

注：np.linalg.norm()功能是求范数

1、linalg=linear（线性）+algebra（代数），norm则表示范数。

2、函数参数

x_norm=np.linalg.norm(x, ord=None, axis=None, keepdims=False)  　

　　①x: 表示矩阵（也可以是一维）

　　②ord：范数类型

　　向量的范数：

　　

　　矩阵的范数：

　　　　ord=1：列和的最大值

　　　　ord=2：|λE-ATA|=0，求特征值，然后求最大特征值得算术平方根

　　　　ord=∞：行和的最大值

　　③axis：处理类型

　　　　axis=1表示按行向量处理，求多个行向量的范数

　　　　axis=0表示按列向量处理，求多个列向量的范数

　　　　axis=None表示矩阵范数。

　　④keepding：是否保持矩阵的二维特性

　　　　True表示保持矩阵的二维特性，False相反

# Open cs231n/classifiers/k_nearest_neighbor.py and implement
# compute_distances_two_loops.

# Test your implementation:
dists = classifier.compute_distances_two_loops(X_test)
print(dists.shape)

# We can visualize the distance matrix: each row is a single test example and
# its distances to training examples
plt.imshow(dists, interpolation='none')
plt.show()

 

**Inline Question 1**

Notice the structured patterns in the distance matrix, where some rows or columns are visible brighter. (Note that with the default color scheme black indicates low distances while white indicates high distances.)

- What in the data is the cause behind the distinctly bright rows?
- What causes the columns?

$\color{blue}{\textit Your Answer:}$ *fill this in.*
1.Either this is an observation from a class not in the training data set, or is at least very different from all/most of the training data, probably in terms of background color.可能这是一个不在训练集类别里的一张图片，或者至少是跟训练集所有/大多数图片都非常不同的图片
2.This training data point doesn’t have any similar points in the test set.白色的列是因为某张训练图片跟测试集的图片都不相同。
*Difference between two pictures, row = one test image vs 5000 train images, column = one train image vs 500 test images*

 

以上完成了一个两层循环的方法调用。

调用的方法代码如下：

def compute_distances_two_loops(self, X):
        """
        Compute the distance between each test point in X and each training point
        in self.X_train using a nested loop over both the training data and the
        test data.

        Inputs:
        - X: A numpy array of shape (num_test, D) containing test data.

        Returns:
        - dists: A numpy array of shape (num_test, num_train) where dists[i, j]
          is the Euclidean distance between the ith test point and the jth training
          point.
        """
        num_test = X.shape[0]
        num_train = self.X_train.shape[0]
        dists = np.zeros((num_test, num_train))
        for i in range(num_test):
            for j in range(num_train):
                #####################################################################
                # TODO:                                                             #
                # Compute the l2 distance between the ith test point and the jth    #
                # training point, and store the result in dists[i, j]. You should   #
                # not use a loop over dimension, nor use np.linalg.norm().          #
                #####################################################################
                # *****START OF YOUR CODE (DO NOT DELETE/MODIFY THIS LINE)*****
                dists[i][j] = np.sqrt(np.sum((X[i] - self.X_train[j])**2)) #用numpy的矩阵功能
                # *****END OF YOUR CODE (DO NOT DELETE/MODIFY THIS LINE)*****
        return dists