作业15

补交：6.逻辑回归 https://www.cnblogs.com/Roromiya/p/13088857.html

一、手写数字数据集

• from sklearn.datasets import load_digits
• digits = load_digits()

# 1 - Load dataset
digits = load_digits()
X = digits['images']
Y = digits['target']

二、图片数据预处理

• x：归一化MinMaxScaler()
• y：独热编码OneHotEncoder()或to_categorical
• 训练集测试集划分
• 张量结构

使用自定义归一化函数normalization()，与独立热编码函数onehot_encode()

# 2.1 - Reshape the images, from (8, 8) to (8, 8, 1)
def img_reshape(images):
    return [tf.reshape(image, (8, 8, 1)).numpy() for image in images]
X = img_reshape(X)

# 2.2 - Normalize the images, scale the value to 0.0 ~ 1.0
def normalization(data):
    _range = np.max(data) - np.min(data)
    return (data - np.min(data)) / _range
X = normalization(X)

# 2.3 - Onehot encoding on labels
def onehot_encode(labels):
    onehot_labels = []
    for y in labels:
        label = [0. for i in range(10)]
        label[y] = 1.
        onehot_labels.append(label)
    return onehot_labels
def onehot_decode(labels):
    return np.array([label.index(True) for label in labels.tolist()])
Y = onehot_encode(Y)
Y = np.array(Y)

# 2.4 - Split the train-set and test-set
X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.2, random_state=0)

print('Images shape: ', X.shape)
print('Labels shape: ', Y.shape)
print('Images: ', X)
print('Labels: ', Y)

处理后的数据集尺寸(数量, 高度, 宽度, 通道数)：

处理后的标签集尺寸(数量, 类别个数)：

查看数据集：

查看标签集：

三、设计卷积神经网络结构

• 绘制模型结构图，并说明设计依据。

网络结构：

 

设计依据：

1）数据尺寸太小，特征较少，没必要使用过深的网络提取特征，三层就够了。

2）由于图像尺寸较小，故卷积核尺寸设置为2×2，三个卷积-池化层的卷积核数量分别设置为32、64、128。

3）该模型不使用Dropout，因为在使用小卷积核时Dropout丢弃过多特征会导致准确率变差。

4）在卷积层后使用BatchNormalization替代Dropout，BN同样可以起到防止过拟合的作用，还可以加速模型收敛。

5）池化前使用ReLu而不是sigmod作为激活函数，目的是避免“梯度消失”。

6）由于这是一个用于分类的神经网络，独热编码值介于0.0～1.0之间，故全连接之后使用Softmax激活函数进行分类。

创建模型：

设置初始学习率为1e-4，优化器为Adam，损失函数为交叉熵损失

def CNN():
    nn = models.Sequential()
    # 1st Convolution->BN->ReLu->Pool
    nn.add(layers.Conv2D(filters=32, kernel_size=(2, 2), input_shape=(8, 8, 1)))
    nn.add(layers.BatchNormalization())
    nn.add(layers.ReLU())
    nn.add(layers.MaxPool2D(pool_size=(2, 2), strides=1))

    # 2nd Convolution->BN->ReLu->Pool
    nn.add(layers.Conv2D(filters=64, kernel_size=(2, 2)))
    nn.add(layers.BatchNormalization())
    nn.add(layers.ReLU())
    nn.add(layers.MaxPool2D(pool_size=(2, 2), strides=1))

    # 3rd Convolution->Convolution->BN->ReLu->Pool
    nn.add(layers.Conv2D(filters=128, kernel_size=(2, 2)))
    nn.add(layers.Conv2D(filters=128, kernel_size=(2, 2)))
    nn.add(layers.BatchNormalization())
    nn.add(layers.ReLU())
    nn.add(layers.MaxPool2D(pool_size=(2, 2), strides=2))

    # Flatten->Dense->Softmax
    nn.add(layers.Flatten())
    nn.add(layers.Dense(units=10))
    nn.add(layers.Softmax())
    
    # Compile and ready to use
    nn.compile(
        optimizer=optimizers.Adam(learning_rate=0.0001),
        loss=losses.categorical_crossentropy,
        metrics=['accuracy']
    )
    return nn

四、模型训练

设置batch_size为4，epoch为80轮。

创建回调函数TensorBoard()，用于记录loss下降曲线。

创建回调函数ReduceLROnPlateau()，当验证集超过10个epoch不下降时自动降低学习率。

def train(nn, train_image, train_label, test_image, test_label, batch_size, epoch, log_path):
    # Create Tensorboard callback
    tb = tf.keras.callbacks.TensorBoard(log_dir=log_path, histogram_freq=1)
    # Create learning rate reducer
    reduce_lr = ReduceLROnPlateau(monitor='val_loss', patience=10, mode='auto')
    history = nn.fit(train_image, train_label,
              batch_size=batch_size, epochs=epoch, validation_data=(test_image, test_label), callbacks=[tb, reduce_lr])
    return history

log_dir = "./logs/fit/" + datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
train(model, X_train, Y_train, X_test, Y_test, batch_size=4, epoch=80, log_path=log_dir)
model.save("./weights.hdf5") # Save the weights

训练过程：

Linux终端内启动tensorboard

打开Tensorboard网页界面，观察loss_epoch曲线、accuracy_epoch曲线:

  

打开Tensorboard网页界面，观察全连接层的张量直方图：

五、模型评价

• model.evaluate()
• 交叉表与交叉矩阵
• pandas.crosstab
• seaborn.heatmap

使用model.evaluate()进行模型评估：

# 5.1 - model.evaluate()
model.load_weights(args.weights)
print("\nModel Evaluate:")
model.evaluate(X_test, Y_test, batch_size=128)

评估结果——模型在测试集上的准确率为 99.72%

绘制交叉表：

# 5.2 - pandas.crosstab()
pred = model.predict_classes(X_test)
table = pandas.DataFrame(pandas.crosstab(onehot_decode(Y_test), pred, rownames=['true'], colnames=['pred']))
heatmap(table, annot=True, cmap="Reds", linewidths=0.2, linecolor='G')
plt.show()

 

 

完整代码：

Python版本：3.8.3

Tensorflow版本：2.2.0

Tensorboard版本：2.2.2

import datetime
import argparse
from sklearn.datasets import load_digits
from sklearn.model_selection import train_test_split
import tensorflow as tf
from tensorflow.keras import layers, models, optimizers, losses
import numpy as np
import os
import pandas
from seaborn import heatmap
from matplotlib import pyplot as plt

# From [9] to [0., 0., 0., 0., 0., 0., 0., 0., 1., 0.]
from tensorflow.python.keras.callbacks import ReduceLROnPlateau


def onehot_encode(labels):
    onehot_labels = []
    for y in labels:
        label = [0. for i in range(10)]
        label[y] = 1.
        onehot_labels.append(label)
    return onehot_labels

# From [0., 0., 0., 0., 0., 0., 0., 0., 1., 0.] to [9]
def onehot_decode(labels):
    return np.array([label.index(True) for label in labels.tolist()])

# Scale to 0. ~ 1.
def normalization(data):
    _range = np.max(data) - np.min(data)
    return (data - np.min(data)) / _range

# From (n, 8, 8) to (n, 8, 8, 1)
def img_reshape(images):
    return [tf.reshape(image, (8, 8, 1)).numpy() for image in images]

def CNN():
    nn = models.Sequential()
    # 1st Convolution->BN->ReLu->Pool
    nn.add(layers.Conv2D(filters=32, kernel_size=(2, 2), input_shape=(8, 8, 1)))
    nn.add(layers.BatchNormalization())
    nn.add(layers.ReLU())
    nn.add(layers.MaxPool2D(pool_size=(2, 2), strides=1))

    # 2nd Convolution->BN->ReLu->Pool
    nn.add(layers.Conv2D(filters=64, kernel_size=(2, 2)))
    nn.add(layers.BatchNormalization())
    nn.add(layers.ReLU())
    nn.add(layers.MaxPool2D(pool_size=(2, 2), strides=1))

    # 3rd Convolution->Convolution->BN->ReLu->Pool
    nn.add(layers.Conv2D(filters=128, kernel_size=(2, 2)))
    nn.add(layers.Conv2D(filters=128, kernel_size=(2, 2)))
    nn.add(layers.BatchNormalization())
    nn.add(layers.ReLU())
    nn.add(layers.MaxPool2D(pool_size=(2, 2), strides=2))

    # Flatten->Dense->Softmax
    nn.add(layers.Flatten())
    nn.add(layers.Dense(units=10))
    nn.add(layers.Softmax())

    # Compile and ready to use
    nn.compile(
        optimizer=optimizers.Adam(learning_rate=0.0001),
        loss=losses.categorical_crossentropy,
        metrics=['accuracy']
    )
    return nn

def train(nn, train_image, train_label, test_image, test_label, batch_size, epoch, log_path):
    # Create Tensorboard callback
    tb = tf.keras.callbacks.TensorBoard(log_dir=log_path, histogram_freq=1)
    # Create learning rate reducer
    reduce_lr = ReduceLROnPlateau(monitor='val_loss', patience=10, mode='auto')
    history = nn.fit(train_image, train_label,
              batch_size=batch_size, epochs=epoch, validation_data=(test_image, test_label), callbacks=[tb, reduce_lr])
    return history


if __name__ == '__main__':
    parser = argparse.ArgumentParser()
    parser.add_argument('--cpu', type=bool, default=False)
    parser.add_argument('--weights', type=str, default=None)
    args = parser.parse_args()

    # Force CPU or GPU
    os.environ["CUDA_DEVICE_ORDER"] = "PCI_BUS_ID"
    os.environ["CUDA_VISIBLE_DEVICES"] = "-1" if args.cpu else "0"
    if not args.cpu:
        # Fix CUDNN_STATUS_INTERNAL_ERROR
        gpu_devices = tf.config.experimental.list_physical_devices('GPU')
        for device in gpu_devices:
            tf.config.experimental.set_memory_growth(device, True)

    # 1 - Load dataset
    digits = load_digits()
    X = digits['images']
    Y = digits['target']

    # 2 - Pretreatment
    # 2.1 - Reshape the images, from (8, 8) to (8, 8, 1)
    X = img_reshape(X)
    # 2.2 - Normalize the images, scale the value to 0.0 ~ 1.0
    X = normalization(X)
    # 2.3 - Onehot encoding on labels
    Y = onehot_encode(Y)
    Y = np.array(Y)
    # 2.4 - Split the train-set and test-set
    X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size=0.2, random_state=0)
    print('Images shape: ', X.shape)
    print('Labels shape: ', Y.shape)
    print('Images: ', X)
    print('Labels: ', Y)

    # 3 - Create model
    model = CNN()
    model.summary()

    # 4 - Training
    if not args.weights:
        log_dir = "./logs/fit/" + datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
        train(model, X_train, Y_train, X_test, Y_test, batch_size=4, epoch=80, log_path=log_dir)
        model.save("./weights.hdf5") # Save the weights

    # 5 - Evaluate
    # 5.1 - model.evaluate()
    if args.weights:
        model.load_weights(args.weights)
    print("\nModel Evaluate:")
    model.evaluate(X_test, Y_test, batch_size=4)
    # 5.2 - pandas.crosstab()
    pred = model.predict_classes(X_test)
    table = pandas.DataFrame(pandas.crosstab(onehot_decode(Y_test), pred, rownames=['true'], colnames=['pred']))
    heatmap(table, annot=True, cmap="Reds", linewidths=0.2, linecolor='G')
    plt.show()