使用 Keras 卷積神經網路 (CNN) 辨識 Cifar-10 圖片 (二)

上一篇 Keras 機器學習測試中, 利用 CNN+MLP 結構將 Cifar-10 資料集的圖片預測準確度從單用 MLP 的 0.45 提升到 0.74, 足見 CNN 的學習效果非常顯著. 如果將此學習架構深度化, 多加一層卷積 + 池化層, 隱藏層也增加一層, 這樣應該可使準確度再拉高, 當然也將耗費更多運算時間.

本系列之前的測試紀錄參考 :

# Windows 安裝深度學習框架 TensorFlow 與 Keras
# 使用 Keras 測試 MNIST 手寫數字辨識資料集
# 使用 Keras 多層感知器 MLP 辨識手寫數字 (一)
# 使用 Keras 多層感知器 MLP 辨識手寫數字 (二)

# 使用 Keras 卷積神經網路 (CNN) 辨識手寫數字
# 使用 Keras 測試 Cifar-10 圖片資料集
# 使用 Keras 多層感知器 MLP 辨識 Cifar-10 圖片
# 使用 Keras 卷積神經網路 (CNN) 辨識 Cifar-10 圖片 (一)

以下根據林大貴寫的 "TensorFlow+Keras 深度學習人工智慧實務應用" 一書第 10 章進行測試並紀錄測試結果. 為簡略計不再於 IDLE 介面執行, 而是修改上一篇測試所用之程式直接在命令列執行, 主要是加入第三個卷積層, 每一個卷積重複兩次, 同時 MLP 添加第二個隱藏層, 模型之結構如下圖所示 :

完整之測試程式如下 :

#show_cifar10_deep_cnn3_predict.py
#載入資料集
import numpy as np
np.random.seed(10)
from keras.datasets import cifar10
(x_train_image, y_train_label), (x_test_image, y_test_label)=cifar10.load_data()

#資料預處理
x_train_normalize=x_train_image.astype('float32')/255.0
x_test_normalize=x_test_image.astype('float32')/255.0
from keras.utils import np_utils
y_train_onehot=np_utils.to_categorical(y_train_label)
y_test_onehot=np_utils.to_categorical(y_test_label)

#建立模型
#建立三層卷積 (丟棄 30% 神經元) + 池化層, 每個卷積做兩次
from keras.models import Sequential
from keras.layers import Dense,Dropout,Flatten,Conv2D,MaxPooling2D
from keras.layers import ZeroPadding2D,Activation
model=Sequential()
#卷積層1+池化層1
model.add(Conv2D(filters=32,
                 kernel_size=(3,3),
                 padding='same',
                 input_shape=(32,32,3),
                 activation='relu'))
model.add(Dropout(0.3))
model.add(Conv2D(filters=32,
                 kernel_size=(3,3),
                 padding='same',
                 activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
#卷積層2+池化層2
model.add(Conv2D(filters=64,
                 kernel_size=(3,3),
                 padding='same',
                 activation='relu'))
model.add(Dropout(0.3))
model.add(Conv2D(filters=64,
                 kernel_size=(3,3),
                 padding='same',
                 activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
#卷積層3+池化層3
model.add(Conv2D(filters=128,
                 kernel_size=(3,3),
                 padding='same',
                 activation='relu'))
model.add(Dropout(0.3))
model.add(Conv2D(filters=128,
                 kernel_size=(3,3),
                 padding='same',
                 activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
#建立分類模型 MLP
model.add(Flatten())                      #平坦層
model.add(Dropout(0.3))
model.add(Dense(2500,activation='relu'))  #隱藏層1
model.add(Dropout(0.3))
model.add(Dense(1500,activation='relu'))  #隱藏層2
model.add(Dropout(0.3))
model.add(Dense(10,activation='softmax')) #輸出層
model.summary()

#訓練模型
import time
start=time.time()
model.compile(loss='categorical_crossentropy',optimizer='adam',metrics=['accuracy'])
train_history=model.fit(x=x_train_normalize, y=y_train_onehot, validation_split=0.2, epochs=10, batch_size=128,verbose=2)
elapsed=time.time()-start
print("Training time=" + str(elapsed) + " Seconds")
#繪製訓練結果
def show_train_history(train_history):
    fig=plt.gcf()
    fig.set_size_inches(16, 6)
    plt.subplot(121)
    plt.plot(train_history.history["acc"])
    plt.plot(train_history.history["val_acc"])
    plt.title("Train History")
    plt.xlabel("Epoch")
    plt.ylabel("Accuracy")
    plt.legend(["train", "validation"], loc="upper left")
    plt.subplot(122)
    plt.plot(train_history.history["loss"])
    plt.plot(train_history.history["val_loss"])
    plt.title("Train History")
    plt.xlabel("Epoch")
    plt.ylabel("Loss")
    plt.legend(["train", "validation"], loc="upper left")
    plt.show()
import matplotlib.pyplot as plt
show_train_history(train_history)

#評估預測準確率
scores=model.evaluate(x_test_normalize, y_test_onehot)
print("Accuracy=", scores[1])

#預測測試集圖片
prediction=model.predict_classes(x_test_normalize)
print(prediction)


注意, 程式中我使用 time 模組的 time() 函數來計算模型訓練所需時間.

程式執行結果如下 :

D:\Python\test>python show_cifar10_cnn3_predict.py
Using TensorFlow backend.
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
conv2d_1 (Conv2D)            (None, 32, 32, 32)        896
_________________________________________________________________
dropout_1 (Dropout)          (None, 32, 32, 32)        0
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 32, 32, 32)        9248
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 16, 16, 32)        0
_________________________________________________________________
conv2d_3 (Conv2D)            (None, 16, 16, 64)        18496
_________________________________________________________________
dropout_2 (Dropout)          (None, 16, 16, 64)        0
_________________________________________________________________
conv2d_4 (Conv2D)            (None, 16, 16, 64)        36928
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 8, 8, 64)          0
_________________________________________________________________
conv2d_5 (Conv2D)            (None, 8, 8, 128)         73856
_________________________________________________________________
dropout_3 (Dropout)          (None, 8, 8, 128)         0
_________________________________________________________________
conv2d_6 (Conv2D)            (None, 8, 8, 128)         147584
_________________________________________________________________
max_pooling2d_3 (MaxPooling2 (None, 4, 4, 128)         0
_________________________________________________________________
flatten_1 (Flatten)          (None, 2048)              0
_________________________________________________________________
dropout_4 (Dropout)          (None, 2048)              0
_________________________________________________________________
dense_1 (Dense)              (None, 2500)              5122500
_________________________________________________________________
dropout_5 (Dropout)          (None, 2500)              0
_________________________________________________________________
dense_2 (Dense)              (None, 1500)              3751500
_________________________________________________________________
dropout_6 (Dropout)          (None, 1500)              0
_________________________________________________________________
dense_3 (Dense)              (None, 10)                15010
=================================================================
Total params: 9,176,018
Trainable params: 9,176,018
Non-trainable params: 0
_________________________________________________________________
Train on 40000 samples, validate on 10000 samples
Epoch 1/10
2018-04-16 09:58:59.432800: I C:\tf_jenkins\workspace\rel-win\M\windows\PY\36\tensorflow\core\platfo
rm\cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not comp
iled to use: AVX AVX2
 - 412s - loss: 1.8321 - acc: 0.3131 - val_loss: 1.6129 - val_acc: 0.4149
Epoch 2/10
 - 415s - loss: 1.3994 - acc: 0.4883 - val_loss: 1.2652 - val_acc: 0.5418
Epoch 3/10
 - 402s - loss: 1.2031 - acc: 0.5636 - val_loss: 1.1099 - val_acc: 0.6075
Epoch 4/10
 - 398s - loss: 1.0691 - acc: 0.6193 - val_loss: 0.9544 - val_acc: 0.6652
Epoch 5/10
 - 397s - loss: 0.9675 - acc: 0.6552 - val_loss: 0.9089 - val_acc: 0.6871
Epoch 6/10
 - 397s - loss: 0.8953 - acc: 0.6818 - val_loss: 0.8561 - val_acc: 0.6951
Epoch 7/10
 - 397s - loss: 0.8211 - acc: 0.7098 - val_loss: 0.8310 - val_acc: 0.7025
Epoch 8/10
 - 397s - loss: 0.7664 - acc: 0.7281 - val_loss: 0.7770 - val_acc: 0.7261
Epoch 9/10
 - 397s - loss: 0.7188 - acc: 0.7458 - val_loss: 0.7487 - val_acc: 0.7370
Epoch 10/10
 - 398s - loss: 0.6786 - acc: 0.7602 - val_loss: 0.7298 - val_acc: 0.7467
Training time=4010.7789998054504 Seconds
10000/10000 [==============================] - 28s 3ms/step
Accuracy= 0.7403

訓練結果 train_history 繪圖如下 :

結果還是 0.74 左右, 花了 4010 秒 (約 1 小時又 7 分鐘) 運算不是白忙一場嗎? 但書中作者的預測結果卻能提升到 0.7889, Why? 

參考 :

# Deep learning for complete beginners: convolutional neural networks with keras

2018-04-16 補充 :

我用另一台電腦跑出 0.7552 準確率 :

D:\Python\test>python show_cifar10_cnn3_predict.py
Using TensorFlow backend.
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
conv2d_1 (Conv2D)            (None, 32, 32, 32)        896
_________________________________________________________________
dropout_1 (Dropout)          (None, 32, 32, 32)        0
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 32, 32, 32)        9248
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 16, 16, 32)        0
_________________________________________________________________
conv2d_3 (Conv2D)            (None, 16, 16, 64)        18496
_________________________________________________________________
dropout_2 (Dropout)          (None, 16, 16, 64)        0
_________________________________________________________________
conv2d_4 (Conv2D)            (None, 16, 16, 64)        36928
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 8, 8, 64)          0
_________________________________________________________________
conv2d_5 (Conv2D)            (None, 8, 8, 128)         73856
_________________________________________________________________
dropout_3 (Dropout)          (None, 8, 8, 128)         0
_________________________________________________________________
conv2d_6 (Conv2D)            (None, 8, 8, 128)         147584
_________________________________________________________________
max_pooling2d_3 (MaxPooling2 (None, 4, 4, 128)         0
_________________________________________________________________
flatten_1 (Flatten)          (None, 2048)              0
_________________________________________________________________
dropout_4 (Dropout)          (None, 2048)              0
_________________________________________________________________
dense_1 (Dense)              (None, 2500)              5122500
_________________________________________________________________
dropout_5 (Dropout)          (None, 2500)              0
_________________________________________________________________
dense_2 (Dense)              (None, 1500)              3751500
_________________________________________________________________
dropout_6 (Dropout)          (None, 1500)              0
_________________________________________________________________
dense_3 (Dense)              (None, 10)                15010
=================================================================
Total params: 9,176,018
Trainable params: 9,176,018
Non-trainable params: 0
_________________________________________________________________
Train on 40000 samples, validate on 10000 samples
Epoch 1/10
 - 685s - loss: 1.8330 - acc: 0.3154 - val_loss: 1.6589 - val_acc: 0.3792
Epoch 2/10
 - 638s - loss: 1.3598 - acc: 0.5026 - val_loss: 1.3293 - val_acc: 0.5270
Epoch 3/10
 - 635s - loss: 1.1501 - acc: 0.5855 - val_loss: 1.0248 - val_acc: 0.6400
Epoch 4/10
 - 690s - loss: 1.0238 - acc: 0.6368 - val_loss: 0.9540 - val_acc: 0.6686
Epoch 5/10
 - 736s - loss: 0.9270 - acc: 0.6697 - val_loss: 0.8981 - val_acc: 0.6858
Epoch 6/10
 - 667s - loss: 0.8542 - acc: 0.6982 - val_loss: 0.8487 - val_acc: 0.7018
Epoch 7/10
 - 704s - loss: 0.7938 - acc: 0.7179 - val_loss: 0.7624 - val_acc: 0.7340
Epoch 8/10
 - 673s - loss: 0.7427 - acc: 0.7371 - val_loss: 0.7709 - val_acc: 0.7295
Epoch 9/10
 - 696s - loss: 0.7013 - acc: 0.7507 - val_loss: 0.7527 - val_acc: 0.7375
Epoch 10/10
 - 675s - loss: 0.6597 - acc: 0.7637 - val_loss: 0.6890 - val_acc: 0.7587
Training time=6801.2324249744415 Seconds
10000/10000 [==============================] - 51s 5ms/step
Accuracy= 0.7552

即使如此, 準確率似乎也沒有顯著提升.