深度残差网络+自适应参数化ReLU激活函数（调参记录6）

续上一篇：
深度残差网络+自适应参数化ReLU激活函数（调参记录5）
https://www.cnblogs.com/shisuzanian/p/12907423.html
本文继续调整超参数，测试Adaptively Parametric ReLU（APReLU）激活函数在Cifar10图像集上的效果。APReLU的基本原理如下图所示：

首先，从之前的调参发现，当学习率从0.1降到0.01和从0.01降到0.001的时候，loss会有大幅的下降。之前学习率降到0.001就结束了，那么如果学习率继续往下降的话，是不是loss还会继续下降呢？

其次，当采用APReLU激活函数时，深度残差网络的结构比较复杂，更难训练，也许需要更多的迭代次数。

因此，这次测试将迭代次数恢复成1000个epoch，将1-300、301-600、601-900、901-1000个epoch的学习率分别设置成了0.1、0.01、0.001、0.0001。

同时，最后全局均值池化之前，如果采用APReLU的话，似乎是不利于模型训练的。这是因为APReLU里面用到了sigmoid函数。因此，全局均值池化之前的APReLU改成了普通的ReLU。（对于残差模块里面的APReLU，由于恒等路径的存在，其所导致训练难度的增加应该是可以容忍的）

Keras代码如下：

  1 #!/usr/bin/env python3
  2 # -*- coding: utf-8 -*-
  3 """
  4 Created on Tue Apr 14 04:17:45 2020
  5 Implemented using TensorFlow 1.10.0 and Keras 2.2.1
  6 
  7 Minghang Zhao, Shisheng Zhong, Xuyun Fu, Baoping Tang, Shaojiang Dong, Michael Pecht,
  8 Deep Residual Networks with Adaptively Parametric Rectifier Linear Units for Fault Diagnosis, 
  9 IEEE Transactions on Industrial Electronics, 2020,  DOI: 10.1109/TIE.2020.2972458 
 10 
 11 @author: Minghang Zhao
 12 """
 13 
 14 from __future__ import print_function
 15 import keras
 16 import numpy as np
 17 from keras.datasets import cifar10
 18 from keras.layers import Dense, Conv2D, BatchNormalization, Activation, Minimum
 19 from keras.layers import AveragePooling2D, Input, GlobalAveragePooling2D, Concatenate, Reshape
 20 from keras.regularizers import l2
 21 from keras import backend as K
 22 from keras.models import Model
 23 from keras import optimizers
 24 from keras.preprocessing.image import ImageDataGenerator
 25 from keras.callbacks import LearningRateScheduler
 26 K.set_learning_phase(1)
 27 
 28 # The data, split between train and test sets
 29 (x_train, y_train), (x_test, y_test) = cifar10.load_data()
 30 
 31 # Noised data
 32 x_train = x_train.astype('float32') / 255.
 33 x_test = x_test.astype('float32') / 255.
 34 x_test = x_test-np.mean(x_train)
 35 x_train = x_train-np.mean(x_train)
 36 print('x_train shape:', x_train.shape)
 37 print(x_train.shape[0], 'train samples')
 38 print(x_test.shape[0], 'test samples')
 39 
 40 # convert class vectors to binary class matrices
 41 y_train = keras.utils.to_categorical(y_train, 10)
 42 y_test = keras.utils.to_categorical(y_test, 10)
 43 
 44 # Schedule the learning rate, multiply 0.1 every 300 epoches
 45 def scheduler(epoch):
 46     if epoch % 300 == 0 and epoch != 0:
 47         lr = K.get_value(model.optimizer.lr)
 48         K.set_value(model.optimizer.lr, lr * 0.1)
 49         print("lr changed to {}".format(lr * 0.1))
 50     return K.get_value(model.optimizer.lr)
 51 
 52 # An adaptively parametric rectifier linear unit (APReLU)
 53 def aprelu(inputs):
 54     # get the number of channels
 55     channels = inputs.get_shape().as_list()[-1]
 56     # get a zero feature map
 57     zeros_input = keras.layers.subtract([inputs, inputs])
 58     # get a feature map with only positive features
 59     pos_input = Activation('relu')(inputs)
 60     # get a feature map with only negative features
 61     neg_input = Minimum()([inputs,zeros_input])
 62     # define a network to obtain the scaling coefficients
 63     scales_p = GlobalAveragePooling2D()(pos_input)
 64     scales_n = GlobalAveragePooling2D()(neg_input)
 65     scales = Concatenate()([scales_n, scales_p])
 66     scales = Dense(channels, activation='linear', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(scales)
 67     scales = BatchNormalization()(scales)
 68     scales = Activation('relu')(scales)
 69     scales = Dense(channels, activation='linear', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(scales)
 70     scales = BatchNormalization()(scales)
 71     scales = Activation('sigmoid')(scales)
 72     scales = Reshape((1,1,channels))(scales)
 73     # apply a paramtetric relu
 74     neg_part = keras.layers.multiply([scales, neg_input])
 75     return keras.layers.add([pos_input, neg_part])
 76 
 77 # Residual Block
 78 def residual_block(incoming, nb_blocks, out_channels, downsample=False,
 79                    downsample_strides=2):
 80     
 81     residual = incoming
 82     in_channels = incoming.get_shape().as_list()[-1]
 83     
 84     for i in range(nb_blocks):
 85         
 86         identity = residual
 87         
 88         if not downsample:
 89             downsample_strides = 1
 90         
 91         residual = BatchNormalization()(residual)
 92         residual = aprelu(residual)
 93         residual = Conv2D(out_channels, 3, strides=(downsample_strides, downsample_strides), 
 94                           padding='same', kernel_initializer='he_normal', 
 95                           kernel_regularizer=l2(1e-4))(residual)
 96         
 97         residual = BatchNormalization()(residual)
 98         residual = aprelu(residual)
 99         residual = Conv2D(out_channels, 3, padding='same', kernel_initializer='he_normal', 
100                           kernel_regularizer=l2(1e-4))(residual)
101         
102         # Downsampling
103         if downsample_strides > 1:
104             identity = AveragePooling2D(pool_size=(1,1), strides=(2,2))(identity)
105             
106         # Zero_padding to match channels
107         if in_channels != out_channels:
108             zeros_identity = keras.layers.subtract([identity, identity])
109             identity = keras.layers.concatenate([identity, zeros_identity])
110             in_channels = out_channels
111         
112         residual = keras.layers.add([residual, identity])
113     
114     return residual
115 
116 
117 # define and train a model
118 inputs = Input(shape=(32, 32, 3))
119 net = Conv2D(16, 3, padding='same', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(inputs)
120 net = residual_block(net, 9, 16, downsample=False)
121 net = residual_block(net, 1, 32, downsample=True)
122 net = residual_block(net, 8, 32, downsample=False)
123 net = residual_block(net, 1, 64, downsample=True)
124 net = residual_block(net, 8, 64, downsample=False)
125 net = BatchNormalization()(net)
126 net = Activation('relu')(net)
127 net = GlobalAveragePooling2D()(net)
128 outputs = Dense(10, activation='softmax', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(net)
129 model = Model(inputs=inputs, outputs=outputs)
130 sgd = optimizers.SGD(lr=0.1, decay=0., momentum=0.9, nesterov=True)
131 model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])
132 
133 # data augmentation
134 datagen = ImageDataGenerator(
135     # randomly rotate images in the range (deg 0 to 180)
136     rotation_range=30,
137     # randomly flip images
138     horizontal_flip=True,
139     # randomly shift images horizontally
140     width_shift_range=0.125,
141     # randomly shift images vertically
142     height_shift_range=0.125)
143 
144 reduce_lr = LearningRateScheduler(scheduler)
145 # fit the model on the batches generated by datagen.flow().
146 model.fit_generator(datagen.flow(x_train, y_train, batch_size=100),
147                     validation_data=(x_test, y_test), epochs=1000, 
148                     verbose=1, callbacks=[reduce_lr], workers=4)
149 
150 # get results
151 K.set_learning_phase(0)
152 DRSN_train_score = model.evaluate(x_train, y_train, batch_size=100, verbose=0)
153 print('Train loss:', DRSN_train_score[0])
154 print('Train accuracy:', DRSN_train_score[1])
155 DRSN_test_score = model.evaluate(x_test, y_test, batch_size=100, verbose=0)
156 print('Test loss:', DRSN_test_score[0])
157 print('Test accuracy:', DRSN_test_score[1])

实验结果如下：

   1 Using TensorFlow backend.
   2 x_train shape: (50000, 32, 32, 3)
   3 50000 train samples
   4 10000 test samples
   5 Epoch 1/1000
   6 90s 179ms/step - loss: 2.6847 - acc: 0.4191 - val_loss: 2.2382 - val_acc: 0.5544
   7 Epoch 2/1000
   8 62s 125ms/step - loss: 2.1556 - acc: 0.5605 - val_loss: 1.8942 - val_acc: 0.6254
   9 Epoch 3/1000
  10 63s 125ms/step - loss: 1.8590 - acc: 0.6206 - val_loss: 1.6930 - val_acc: 0.6629
  11 Epoch 4/1000
  12 62s 125ms/step - loss: 1.6407 - acc: 0.6615 - val_loss: 1.4932 - val_acc: 0.6958
  13 Epoch 5/1000
  14 62s 125ms/step - loss: 1.4706 - acc: 0.6923 - val_loss: 1.3326 - val_acc: 0.7317
  15 Epoch 6/1000
  16 62s 125ms/step - loss: 1.3352 - acc: 0.7167 - val_loss: 1.2327 - val_acc: 0.7465
  17 Epoch 7/1000
  18 62s 125ms/step - loss: 1.2271 - acc: 0.7365 - val_loss: 1.1326 - val_acc: 0.7583
  19 Epoch 8/1000
  20 62s 125ms/step - loss: 1.1426 - acc: 0.7512 - val_loss: 1.0737 - val_acc: 0.7718
  21 Epoch 9/1000
  22 62s 125ms/step - loss: 1.0724 - acc: 0.7643 - val_loss: 1.0268 - val_acc: 0.7720
  23 Epoch 10/1000
  24 62s 125ms/step - loss: 1.0256 - acc: 0.7687 - val_loss: 0.9672 - val_acc: 0.7842
  25 Epoch 11/1000
  26 62s 125ms/step - loss: 0.9772 - acc: 0.7766 - val_loss: 0.9104 - val_acc: 0.8032
  27 Epoch 12/1000
  28 63s 125ms/step - loss: 0.9385 - acc: 0.7839 - val_loss: 0.8971 - val_acc: 0.8017
  29 Epoch 13/1000
  30 62s 125ms/step - loss: 0.9109 - acc: 0.7910 - val_loss: 0.8675 - val_acc: 0.8073
  31 Epoch 14/1000
  32 62s 125ms/step - loss: 0.8799 - acc: 0.7961 - val_loss: 0.8410 - val_acc: 0.8118
  33 Epoch 15/1000
  34 62s 125ms/step - loss: 0.8680 - acc: 0.7975 - val_loss: 0.8337 - val_acc: 0.8106
  35 Epoch 16/1000
  36 62s 125ms/step - loss: 0.8426 - acc: 0.8045 - val_loss: 0.7960 - val_acc: 0.8194
  37 Epoch 17/1000
  38 62s 124ms/step - loss: 0.8230 - acc: 0.8088 - val_loss: 0.8293 - val_acc: 0.8065
  39 Epoch 18/1000
  40 62s 125ms/step - loss: 0.8143 - acc: 0.8094 - val_loss: 0.7952 - val_acc: 0.8215
  41 Epoch 19/1000
  42 62s 125ms/step - loss: 0.7971 - acc: 0.8148 - val_loss: 0.7876 - val_acc: 0.8169
  43 Epoch 20/1000
  44 62s 125ms/step - loss: 0.7856 - acc: 0.8204 - val_loss: 0.7765 - val_acc: 0.8247
  45 Epoch 21/1000
  46 62s 124ms/step - loss: 0.7774 - acc: 0.8191 - val_loss: 0.7441 - val_acc: 0.8361
  47 Epoch 22/1000
  48 62s 125ms/step - loss: 0.7718 - acc: 0.8247 - val_loss: 0.7552 - val_acc: 0.8325
  49 Epoch 23/1000
  50 62s 125ms/step - loss: 0.7674 - acc: 0.8272 - val_loss: 0.7786 - val_acc: 0.8241
  51 Epoch 24/1000
  52 62s 125ms/step - loss: 0.7582 - acc: 0.8271 - val_loss: 0.7566 - val_acc: 0.8282
  53 Epoch 25/1000
  54 62s 125ms/step - loss: 0.7448 - acc: 0.8315 - val_loss: 0.7507 - val_acc: 0.8336
  55 Epoch 26/1000
  56 62s 125ms/step - loss: 0.7459 - acc: 0.8336 - val_loss: 0.7725 - val_acc: 0.8217
  57 Epoch 27/1000
  58 62s 125ms/step - loss: 0.7418 - acc: 0.8340 - val_loss: 0.7581 - val_acc: 0.8335
  59 Epoch 28/1000
  60 62s 124ms/step - loss: 0.7335 - acc: 0.8354 - val_loss: 0.7402 - val_acc: 0.8360
  61 Epoch 29/1000
  62 62s 125ms/step - loss: 0.7332 - acc: 0.8372 - val_loss: 0.7429 - val_acc: 0.8394
  63 Epoch 30/1000
  64 62s 125ms/step - loss: 0.7243 - acc: 0.8405 - val_loss: 0.7322 - val_acc: 0.8393
  65 Epoch 31/1000
  66 62s 124ms/step - loss: 0.7227 - acc: 0.8422 - val_loss: 0.7098 - val_acc: 0.8468
  67 Epoch 32/1000
  68 62s 125ms/step - loss: 0.7189 - acc: 0.8392 - val_loss: 0.7359 - val_acc: 0.8396
  69 Epoch 33/1000
  70 62s 125ms/step - loss: 0.7144 - acc: 0.8455 - val_loss: 0.7071 - val_acc: 0.8442
  71 Epoch 34/1000
  72 62s 125ms/step - loss: 0.7111 - acc: 0.8460 - val_loss: 0.7401 - val_acc: 0.8404
  73 Epoch 35/1000
  74 62s 125ms/step - loss: 0.7061 - acc: 0.8480 - val_loss: 0.7155 - val_acc: 0.8497
  75 Epoch 36/1000
  76 62s 124ms/step - loss: 0.7072 - acc: 0.8488 - val_loss: 0.7355 - val_acc: 0.8430
  77 Epoch 37/1000
  78 62s 125ms/step - loss: 0.7077 - acc: 0.8496 - val_loss: 0.7167 - val_acc: 0.8521
  79 Epoch 38/1000
  80 62s 125ms/step - loss: 0.6971 - acc: 0.8518 - val_loss: 0.7595 - val_acc: 0.8315
  81 Epoch 39/1000
  82 62s 125ms/step - loss: 0.6971 - acc: 0.8508 - val_loss: 0.7278 - val_acc: 0.8423
  83 Epoch 40/1000
  84 62s 125ms/step - loss: 0.6923 - acc: 0.8553 - val_loss: 0.7252 - val_acc: 0.8452
  85 Epoch 41/1000
  86 62s 125ms/step - loss: 0.6935 - acc: 0.8538 - val_loss: 0.7169 - val_acc: 0.8461
  87 Epoch 42/1000
  88 62s 125ms/step - loss: 0.6902 - acc: 0.8560 - val_loss: 0.7214 - val_acc: 0.8500
  89 Epoch 43/1000
  90 62s 125ms/step - loss: 0.6874 - acc: 0.8576 - val_loss: 0.7078 - val_acc: 0.8492
  91 Epoch 44/1000
  92 62s 125ms/step - loss: 0.6869 - acc: 0.8585 - val_loss: 0.7122 - val_acc: 0.8526
  93 Epoch 45/1000
  94 62s 124ms/step - loss: 0.6830 - acc: 0.8587 - val_loss: 0.7509 - val_acc: 0.8411
  95 Epoch 46/1000
  96 62s 124ms/step - loss: 0.6867 - acc: 0.8583 - val_loss: 0.7015 - val_acc: 0.8555
  97 Epoch 47/1000
  98 62s 124ms/step - loss: 0.6795 - acc: 0.8614 - val_loss: 0.7051 - val_acc: 0.8529
  99 Epoch 48/1000
 100 62s 125ms/step - loss: 0.6790 - acc: 0.8597 - val_loss: 0.7037 - val_acc: 0.8524
 101 Epoch 49/1000
 102 62s 125ms/step - loss: 0.6790 - acc: 0.8612 - val_loss: 0.7121 - val_acc: 0.8526
 103 Epoch 50/1000
 104 62s 125ms/step - loss: 0.6713 - acc: 0.8638 - val_loss: 0.7031 - val_acc: 0.8556
 105 Epoch 51/1000
 106 62s 125ms/step - loss: 0.6655 - acc: 0.8658 - val_loss: 0.6827 - val_acc: 0.8617
 107 Epoch 52/1000
 108 62s 124ms/step - loss: 0.6725 - acc: 0.8649 - val_loss: 0.7000 - val_acc: 0.8566
 109 Epoch 53/1000
 110 62s 125ms/step - loss: 0.6669 - acc: 0.8677 - val_loss: 0.7089 - val_acc: 0.8599
 111 Epoch 54/1000
 112 62s 125ms/step - loss: 0.6654 - acc: 0.8652 - val_loss: 0.6769 - val_acc: 0.8662
 113 Epoch 55/1000
 114 62s 125ms/step - loss: 0.6674 - acc: 0.8668 - val_loss: 0.7016 - val_acc: 0.8570
 115 Epoch 56/1000
 116 62s 124ms/step - loss: 0.6670 - acc: 0.8670 - val_loss: 0.6838 - val_acc: 0.8647
 117 Epoch 57/1000
 118 62s 125ms/step - loss: 0.6667 - acc: 0.8672 - val_loss: 0.7112 - val_acc: 0.8595
 119 Epoch 58/1000
 120 62s 125ms/step - loss: 0.6629 - acc: 0.8688 - val_loss: 0.7012 - val_acc: 0.8587
 121 Epoch 59/1000
 122 62s 125ms/step - loss: 0.6649 - acc: 0.8678 - val_loss: 0.6854 - val_acc: 0.8656
 123 Epoch 60/1000
 124 62s 125ms/step - loss: 0.6592 - acc: 0.8699 - val_loss: 0.6989 - val_acc: 0.8614
 125 Epoch 61/1000
 126 62s 125ms/step - loss: 0.6591 - acc: 0.8696 - val_loss: 0.6978 - val_acc: 0.8603
 127 Epoch 62/1000
 128 62s 124ms/step - loss: 0.6589 - acc: 0.8711 - val_loss: 0.6866 - val_acc: 0.8626
 129 Epoch 63/1000
 130 62s 124ms/step - loss: 0.6516 - acc: 0.8736 - val_loss: 0.6845 - val_acc: 0.8612
 131 Epoch 64/1000
 132 62s 125ms/step - loss: 0.6520 - acc: 0.8743 - val_loss: 0.7003 - val_acc: 0.8597
 133 Epoch 65/1000
 134 62s 125ms/step - loss: 0.6544 - acc: 0.8736 - val_loss: 0.6992 - val_acc: 0.8593
 135 Epoch 66/1000
 136 62s 125ms/step - loss: 0.6529 - acc: 0.8735 - val_loss: 0.6723 - val_acc: 0.8708
 137 Epoch 67/1000
 138 62s 125ms/step - loss: 0.6534 - acc: 0.8740 - val_loss: 0.6958 - val_acc: 0.8610
 139 Epoch 68/1000
 140 62s 124ms/step - loss: 0.6468 - acc: 0.8737 - val_loss: 0.6829 - val_acc: 0.8640
 141 Epoch 69/1000
 142 62s 125ms/step - loss: 0.6463 - acc: 0.8760 - val_loss: 0.7142 - val_acc: 0.8552
 143 Epoch 70/1000
 144 62s 125ms/step - loss: 0.6461 - acc: 0.8764 - val_loss: 0.6814 - val_acc: 0.8661
 145 Epoch 71/1000
 146 62s 125ms/step - loss: 0.6459 - acc: 0.8764 - val_loss: 0.6884 - val_acc: 0.8656
 147 Epoch 72/1000
 148 62s 125ms/step - loss: 0.6430 - acc: 0.8768 - val_loss: 0.6644 - val_acc: 0.8760
 149 Epoch 73/1000
 150 62s 125ms/step - loss: 0.6406 - acc: 0.8774 - val_loss: 0.6803 - val_acc: 0.8710
 151 Epoch 74/1000
 152 62s 125ms/step - loss: 0.6395 - acc: 0.8781 - val_loss: 0.6845 - val_acc: 0.8665
 153 Epoch 75/1000
 154 62s 125ms/step - loss: 0.6413 - acc: 0.8773 - val_loss: 0.7124 - val_acc: 0.8560
 155 Epoch 76/1000
 156 62s 125ms/step - loss: 0.6383 - acc: 0.8804 - val_loss: 0.7164 - val_acc: 0.8554
 157 Epoch 77/1000
 158 62s 125ms/step - loss: 0.6385 - acc: 0.8806 - val_loss: 0.6843 - val_acc: 0.8661
 159 Epoch 78/1000
 160 62s 124ms/step - loss: 0.6349 - acc: 0.8830 - val_loss: 0.7035 - val_acc: 0.8599
 161 Epoch 79/1000
 162 62s 124ms/step - loss: 0.6330 - acc: 0.8818 - val_loss: 0.6983 - val_acc: 0.8591
 163 Epoch 80/1000
 164 62s 125ms/step - loss: 0.6348 - acc: 0.8810 - val_loss: 0.6886 - val_acc: 0.8626
 165 Epoch 81/1000
 166 62s 125ms/step - loss: 0.6323 - acc: 0.8817 - val_loss: 0.6763 - val_acc: 0.8680
 167 Epoch 82/1000
 168 62s 125ms/step - loss: 0.6320 - acc: 0.8825 - val_loss: 0.6560 - val_acc: 0.8758
 169 Epoch 83/1000
 170 62s 125ms/step - loss: 0.6327 - acc: 0.8820 - val_loss: 0.6592 - val_acc: 0.8779
 171 Epoch 84/1000
 172 62s 124ms/step - loss: 0.6296 - acc: 0.8813 - val_loss: 0.6822 - val_acc: 0.8690
 173 Epoch 85/1000
 174 62s 125ms/step - loss: 0.6310 - acc: 0.8810 - val_loss: 0.6825 - val_acc: 0.8703
 175 Epoch 86/1000
 176 62s 125ms/step - loss: 0.6331 - acc: 0.8832 - val_loss: 0.6891 - val_acc: 0.8665
 177 Epoch 87/1000
 178 62s 125ms/step - loss: 0.6330 - acc: 0.8818 - val_loss: 0.6806 - val_acc: 0.8704
 179 Epoch 88/1000
 180 62s 125ms/step - loss: 0.6274 - acc: 0.8841 - val_loss: 0.6832 - val_acc: 0.8681
 181 Epoch 89/1000
 182 62s 125ms/step - loss: 0.6313 - acc: 0.8821 - val_loss: 0.6796 - val_acc: 0.8694
 183 Epoch 90/1000
 184 62s 125ms/step - loss: 0.6258 - acc: 0.8854 - val_loss: 0.6600 - val_acc: 0.8772
 185 Epoch 91/1000
 186 62s 125ms/step - loss: 0.6270 - acc: 0.8841 - val_loss: 0.6670 - val_acc: 0.8758
 187 Epoch 92/1000
 188 62s 125ms/step - loss: 0.6281 - acc: 0.8824 - val_loss: 0.6881 - val_acc: 0.8710
 189 Epoch 93/1000
 190 62s 124ms/step - loss: 0.6265 - acc: 0.8847 - val_loss: 0.6886 - val_acc: 0.8698
 191 Epoch 94/1000
 192 62s 125ms/step - loss: 0.6214 - acc: 0.8871 - val_loss: 0.6896 - val_acc: 0.8640
 193 Epoch 95/1000
 194 62s 125ms/step - loss: 0.6241 - acc: 0.8860 - val_loss: 0.6674 - val_acc: 0.8721
 195 Epoch 96/1000
 196 62s 125ms/step - loss: 0.6252 - acc: 0.8844 - val_loss: 0.6571 - val_acc: 0.8791
 197 Epoch 97/1000
 198 62s 125ms/step - loss: 0.6227 - acc: 0.8856 - val_loss: 0.6486 - val_acc: 0.8797
 199 Epoch 98/1000
 200 62s 125ms/step - loss: 0.6178 - acc: 0.8866 - val_loss: 0.6849 - val_acc: 0.8717
 201 Epoch 99/1000
 202 62s 125ms/step - loss: 0.6162 - acc: 0.8881 - val_loss: 0.6726 - val_acc: 0.8709
 203 Epoch 100/1000
 204 62s 124ms/step - loss: 0.6209 - acc: 0.8861 - val_loss: 0.6682 - val_acc: 0.8732
 205 Epoch 101/1000
 206 62s 125ms/step - loss: 0.6190 - acc: 0.8883 - val_loss: 0.6810 - val_acc: 0.8723
 207 Epoch 102/1000
 208 62s 125ms/step - loss: 0.6181 - acc: 0.8872 - val_loss: 0.6678 - val_acc: 0.8745
 209 Epoch 103/1000
 210 63s 125ms/step - loss: 0.6163 - acc: 0.8883 - val_loss: 0.6870 - val_acc: 0.8704
 211 Epoch 104/1000
 212 62s 125ms/step - loss: 0.6105 - acc: 0.8910 - val_loss: 0.6576 - val_acc: 0.8775
 213 Epoch 105/1000
 214 62s 125ms/step - loss: 0.6120 - acc: 0.8902 - val_loss: 0.6571 - val_acc: 0.8800
 215 Epoch 106/1000
 216 62s 125ms/step - loss: 0.6146 - acc: 0.8882 - val_loss: 0.6560 - val_acc: 0.8772
 217 Epoch 107/1000
 218 62s 125ms/step - loss: 0.6186 - acc: 0.8870 - val_loss: 0.6773 - val_acc: 0.8720
 219 Epoch 108/1000
 220 62s 125ms/step - loss: 0.6189 - acc: 0.8879 - val_loss: 0.6503 - val_acc: 0.8846
 221 Epoch 109/1000
 222 62s 125ms/step - loss: 0.6110 - acc: 0.8896 - val_loss: 0.6625 - val_acc: 0.8782
 223 Epoch 110/1000
 224 62s 125ms/step - loss: 0.6185 - acc: 0.8862 - val_loss: 0.6735 - val_acc: 0.8712
 225 Epoch 111/1000
 226 62s 125ms/step - loss: 0.6101 - acc: 0.8900 - val_loss: 0.6510 - val_acc: 0.8809
 227 Epoch 112/1000
 228 62s 125ms/step - loss: 0.6132 - acc: 0.8897 - val_loss: 0.6817 - val_acc: 0.8703
 229 Epoch 113/1000
 230 62s 125ms/step - loss: 0.6049 - acc: 0.8941 - val_loss: 0.6524 - val_acc: 0.8776
 231 Epoch 114/1000
 232 62s 125ms/step - loss: 0.6129 - acc: 0.8884 - val_loss: 0.6532 - val_acc: 0.8778
 233 Epoch 115/1000
 234 62s 125ms/step - loss: 0.6077 - acc: 0.8906 - val_loss: 0.6650 - val_acc: 0.8771
 235 Epoch 116/1000
 236 62s 124ms/step - loss: 0.6079 - acc: 0.8915 - val_loss: 0.6643 - val_acc: 0.8759
 237 Epoch 117/1000
 238 62s 125ms/step - loss: 0.6102 - acc: 0.8903 - val_loss: 0.6661 - val_acc: 0.8757
 239 Epoch 118/1000
 240 62s 124ms/step - loss: 0.6103 - acc: 0.8909 - val_loss: 0.6641 - val_acc: 0.8748
 241 Epoch 119/1000
 242 62s 125ms/step - loss: 0.6081 - acc: 0.8908 - val_loss: 0.6744 - val_acc: 0.8718
 243 Epoch 120/1000
 244 62s 125ms/step - loss: 0.6060 - acc: 0.8931 - val_loss: 0.6355 - val_acc: 0.8881
 245 Epoch 121/1000
 246 62s 125ms/step - loss: 0.6063 - acc: 0.8925 - val_loss: 0.6630 - val_acc: 0.8768
 247 Epoch 122/1000
 248 62s 125ms/step - loss: 0.6101 - acc: 0.8901 - val_loss: 0.6482 - val_acc: 0.8799
 249 Epoch 123/1000
 250 62s 125ms/step - loss: 0.6037 - acc: 0.8923 - val_loss: 0.6467 - val_acc: 0.8786
 251 Epoch 124/1000
 252 62s 124ms/step - loss: 0.6016 - acc: 0.8942 - val_loss: 0.6487 - val_acc: 0.8788
 253 Epoch 125/1000
 254 62s 125ms/step - loss: 0.6089 - acc: 0.8915 - val_loss: 0.6812 - val_acc: 0.8683
 255 Epoch 126/1000
 256 62s 125ms/step - loss: 0.6042 - acc: 0.8943 - val_loss: 0.6480 - val_acc: 0.8830
 257 Epoch 127/1000
 258 62s 125ms/step - loss: 0.6008 - acc: 0.8938 - val_loss: 0.6765 - val_acc: 0.8762
 259 Epoch 128/1000
 260 62s 125ms/step - loss: 0.6023 - acc: 0.8940 - val_loss: 0.6676 - val_acc: 0.8755
 261 Epoch 129/1000
 262 62s 125ms/step - loss: 0.6046 - acc: 0.8925 - val_loss: 0.6589 - val_acc: 0.8750
 263 Epoch 130/1000
 264 62s 125ms/step - loss: 0.6008 - acc: 0.8949 - val_loss: 0.6580 - val_acc: 0.8805
 265 Epoch 131/1000
 266 62s 125ms/step - loss: 0.6008 - acc: 0.8947 - val_loss: 0.6546 - val_acc: 0.8825
 267 Epoch 132/1000
 268 62s 125ms/step - loss: 0.6009 - acc: 0.8946 - val_loss: 0.6434 - val_acc: 0.8833
 269 Epoch 133/1000
 270 62s 125ms/step - loss: 0.6019 - acc: 0.8937 - val_loss: 0.6498 - val_acc: 0.8828
 271 Epoch 134/1000
 272 62s 125ms/step - loss: 0.5977 - acc: 0.8942 - val_loss: 0.6527 - val_acc: 0.8796
 273 Epoch 135/1000
 274 62s 125ms/step - loss: 0.6013 - acc: 0.8941 - val_loss: 0.6239 - val_acc: 0.8875
 275 Epoch 136/1000
 276 62s 125ms/step - loss: 0.5946 - acc: 0.8964 - val_loss: 0.6379 - val_acc: 0.8843
 277 Epoch 137/1000
 278 62s 125ms/step - loss: 0.5959 - acc: 0.8945 - val_loss: 0.6549 - val_acc: 0.8792
 279 Epoch 138/1000
 280 62s 125ms/step - loss: 0.5975 - acc: 0.8946 - val_loss: 0.6546 - val_acc: 0.8814
 281 Epoch 139/1000
 282 62s 125ms/step - loss: 0.5954 - acc: 0.8958 - val_loss: 0.6686 - val_acc: 0.8734
 283 Epoch 140/1000
 284 62s 125ms/step - loss: 0.5963 - acc: 0.8959 - val_loss: 0.6363 - val_acc: 0.8845
 285 Epoch 141/1000
 286 62s 125ms/step - loss: 0.5959 - acc: 0.8947 - val_loss: 0.6745 - val_acc: 0.8745
 287 Epoch 142/1000
 288 62s 124ms/step - loss: 0.5988 - acc: 0.8934 - val_loss: 0.6512 - val_acc: 0.8818
 289 Epoch 143/1000
 290 62s 125ms/step - loss: 0.5940 - acc: 0.8977 - val_loss: 0.6644 - val_acc: 0.8784
 291 Epoch 144/1000
 292 62s 125ms/step - loss: 0.5928 - acc: 0.8974 - val_loss: 0.6601 - val_acc: 0.8758
 293 Epoch 145/1000
 294 62s 124ms/step - loss: 0.6008 - acc: 0.8951 - val_loss: 0.6376 - val_acc: 0.8871
 295 Epoch 146/1000
 296 62s 125ms/step - loss: 0.5949 - acc: 0.8962 - val_loss: 0.6469 - val_acc: 0.8855
 297 Epoch 147/1000
 298 62s 125ms/step - loss: 0.5922 - acc: 0.8969 - val_loss: 0.6538 - val_acc: 0.8787
 299 Epoch 148/1000
 300 62s 125ms/step - loss: 0.5964 - acc: 0.8968 - val_loss: 0.6406 - val_acc: 0.8842
 301 Epoch 149/1000
 302 62s 124ms/step - loss: 0.5937 - acc: 0.8974 - val_loss: 0.6441 - val_acc: 0.8832
 303 Epoch 150/1000
 304 62s 125ms/step - loss: 0.5988 - acc: 0.8955 - val_loss: 0.6565 - val_acc: 0.8786
 305 Epoch 151/1000
 306 62s 125ms/step - loss: 0.5906 - acc: 0.8978 - val_loss: 0.6429 - val_acc: 0.8822
 307 Epoch 152/1000
 308 62s 125ms/step - loss: 0.5928 - acc: 0.8969 - val_loss: 0.6567 - val_acc: 0.8777
 309 Epoch 153/1000
 310 62s 125ms/step - loss: 0.5906 - acc: 0.8974 - val_loss: 0.6490 - val_acc: 0.8854
 311 Epoch 154/1000
 312 62s 125ms/step - loss: 0.5916 - acc: 0.8977 - val_loss: 0.6577 - val_acc: 0.8789
 313 Epoch 155/1000
 314 62s 125ms/step - loss: 0.5898 - acc: 0.8990 - val_loss: 0.6776 - val_acc: 0.8713
 315 Epoch 156/1000
 316 62s 125ms/step - loss: 0.5947 - acc: 0.8975 - val_loss: 0.6373 - val_acc: 0.8840
 317 Epoch 157/1000
 318 62s 125ms/step - loss: 0.5936 - acc: 0.8966 - val_loss: 0.6297 - val_acc: 0.8873
 319 Epoch 158/1000
 320 62s 125ms/step - loss: 0.5873 - acc: 0.8995 - val_loss: 0.6499 - val_acc: 0.8765
 321 Epoch 159/1000
 322 62s 125ms/step - loss: 0.5845 - acc: 0.8995 - val_loss: 0.6369 - val_acc: 0.8839
 323 Epoch 160/1000
 324 62s 125ms/step - loss: 0.5928 - acc: 0.8961 - val_loss: 0.6585 - val_acc: 0.8782
 325 Epoch 161/1000
 326 62s 125ms/step - loss: 0.5877 - acc: 0.8970 - val_loss: 0.6343 - val_acc: 0.8836
 327 Epoch 162/1000
 328 62s 125ms/step - loss: 0.5868 - acc: 0.9000 - val_loss: 0.6437 - val_acc: 0.8803
 329 Epoch 163/1000
 330 62s 125ms/step - loss: 0.5902 - acc: 0.8980 - val_loss: 0.6356 - val_acc: 0.8863
 331 Epoch 164/1000
 332 62s 125ms/step - loss: 0.5832 - acc: 0.9012 - val_loss: 0.6400 - val_acc: 0.8874
 333 Epoch 165/1000
 334 62s 125ms/step - loss: 0.5892 - acc: 0.8984 - val_loss: 0.6582 - val_acc: 0.8766
 335 Epoch 166/1000
 336 62s 124ms/step - loss: 0.5867 - acc: 0.9009 - val_loss: 0.6727 - val_acc: 0.8725
 337 Epoch 167/1000
 338 62s 125ms/step - loss: 0.5857 - acc: 0.9007 - val_loss: 0.6682 - val_acc: 0.8746
 339 Epoch 168/1000
 340 62s 125ms/step - loss: 0.5886 - acc: 0.8999 - val_loss: 0.6429 - val_acc: 0.8844
 341 Epoch 169/1000
 342 62s 125ms/step - loss: 0.5855 - acc: 0.8996 - val_loss: 0.6534 - val_acc: 0.8780
 343 Epoch 170/1000
 344 62s 125ms/step - loss: 0.5877 - acc: 0.8997 - val_loss: 0.6453 - val_acc: 0.8814
 345 Epoch 171/1000
 346 62s 125ms/step - loss: 0.5852 - acc: 0.8978 - val_loss: 0.6388 - val_acc: 0.8846
 347 Epoch 172/1000
 348 62s 125ms/step - loss: 0.5878 - acc: 0.8986 - val_loss: 0.6310 - val_acc: 0.8883
 349 Epoch 173/1000
 350 62s 125ms/step - loss: 0.5862 - acc: 0.9018 - val_loss: 0.6279 - val_acc: 0.8885
 351 Epoch 174/1000
 352 62s 125ms/step - loss: 0.5867 - acc: 0.8993 - val_loss: 0.6682 - val_acc: 0.8762
 353 Epoch 175/1000
 354 62s 125ms/step - loss: 0.5859 - acc: 0.8995 - val_loss: 0.6573 - val_acc: 0.8798
 355 Epoch 176/1000
 356 62s 125ms/step - loss: 0.5828 - acc: 0.9017 - val_loss: 0.6472 - val_acc: 0.8835
 357 Epoch 177/1000
 358 62s 125ms/step - loss: 0.5819 - acc: 0.9010 - val_loss: 0.6735 - val_acc: 0.8753
 359 Epoch 178/1000
 360 62s 125ms/step - loss: 0.5875 - acc: 0.8993 - val_loss: 0.6420 - val_acc: 0.8860
 361 Epoch 179/1000
 362 62s 125ms/step - loss: 0.5840 - acc: 0.9000 - val_loss: 0.6490 - val_acc: 0.8809
 363 Epoch 180/1000
 364 62s 125ms/step - loss: 0.5796 - acc: 0.9035 - val_loss: 0.6586 - val_acc: 0.8760
 365 Epoch 181/1000
 366 62s 125ms/step - loss: 0.5811 - acc: 0.9031 - val_loss: 0.6387 - val_acc: 0.8864
 367 Epoch 182/1000
 368 62s 125ms/step - loss: 0.5811 - acc: 0.9015 - val_loss: 0.6334 - val_acc: 0.8890
 369 Epoch 183/1000
 370 62s 125ms/step - loss: 0.5835 - acc: 0.9017 - val_loss: 0.6471 - val_acc: 0.8775
 371 Epoch 184/1000
 372 62s 124ms/step - loss: 0.5801 - acc: 0.9015 - val_loss: 0.6620 - val_acc: 0.8785
 373 Epoch 185/1000
 374 62s 125ms/step - loss: 0.5770 - acc: 0.9023 - val_loss: 0.6412 - val_acc: 0.8842
 375 Epoch 186/1000
 376 62s 125ms/step - loss: 0.5768 - acc: 0.9024 - val_loss: 0.6341 - val_acc: 0.8828
 377 Epoch 187/1000
 378 62s 124ms/step - loss: 0.5817 - acc: 0.9006 - val_loss: 0.6304 - val_acc: 0.8896
 379 Epoch 188/1000
 380 62s 125ms/step - loss: 0.5835 - acc: 0.9008 - val_loss: 0.6491 - val_acc: 0.8820
 381 Epoch 189/1000
 382 62s 125ms/step - loss: 0.5863 - acc: 0.8995 - val_loss: 0.6389 - val_acc: 0.8825
 383 Epoch 190/1000
 384 62s 125ms/step - loss: 0.5832 - acc: 0.9016 - val_loss: 0.6362 - val_acc: 0.8833
 385 Epoch 191/1000
 386 62s 125ms/step - loss: 0.5798 - acc: 0.9022 - val_loss: 0.6460 - val_acc: 0.8804
 387 Epoch 192/1000
 388 62s 125ms/step - loss: 0.5801 - acc: 0.9007 - val_loss: 0.6358 - val_acc: 0.8869
 389 Epoch 193/1000
 390 62s 125ms/step - loss: 0.5807 - acc: 0.9016 - val_loss: 0.6472 - val_acc: 0.8820
 391 Epoch 194/1000
 392 62s 125ms/step - loss: 0.5774 - acc: 0.9024 - val_loss: 0.6542 - val_acc: 0.8825
 393 Epoch 195/1000
 394 62s 125ms/step - loss: 0.5758 - acc: 0.9034 - val_loss: 0.6429 - val_acc: 0.8832
 395 Epoch 196/1000
 396 62s 125ms/step - loss: 0.5784 - acc: 0.9020 - val_loss: 0.6505 - val_acc: 0.8826
 397 Epoch 197/1000
 398 62s 125ms/step - loss: 0.5830 - acc: 0.9010 - val_loss: 0.6669 - val_acc: 0.8741
 399 Epoch 198/1000
 400 62s 125ms/step - loss: 0.5769 - acc: 0.9026 - val_loss: 0.6474 - val_acc: 0.8814
 401 Epoch 199/1000
 402 62s 125ms/step - loss: 0.5772 - acc: 0.9031 - val_loss: 0.6297 - val_acc: 0.8862
 403 Epoch 200/1000
 404 62s 125ms/step - loss: 0.5780 - acc: 0.9023 - val_loss: 0.6459 - val_acc: 0.8843
 405 Epoch 201/1000
 406 62s 125ms/step - loss: 0.5816 - acc: 0.9016 - val_loss: 0.6652 - val_acc: 0.8745
 407 Epoch 202/1000
 408 62s 125ms/step - loss: 0.5764 - acc: 0.9032 - val_loss: 0.6306 - val_acc: 0.8869
 409 Epoch 203/1000
 410 62s 125ms/step - loss: 0.5759 - acc: 0.9038 - val_loss: 0.6328 - val_acc: 0.8881
 411 Epoch 204/1000
 412 63s 125ms/step - loss: 0.5766 - acc: 0.9031 - val_loss: 0.6786 - val_acc: 0.8753
 413 Epoch 205/1000
 414 62s 125ms/step - loss: 0.5756 - acc: 0.9035 - val_loss: 0.6442 - val_acc: 0.8841
 415 Epoch 206/1000
 416 62s 125ms/step - loss: 0.5788 - acc: 0.9020 - val_loss: 0.6505 - val_acc: 0.8813
 417 Epoch 207/1000
 418 62s 125ms/step - loss: 0.5797 - acc: 0.9019 - val_loss: 0.6414 - val_acc: 0.8839
 419 Epoch 208/1000
 420 62s 125ms/step - loss: 0.5755 - acc: 0.9050 - val_loss: 0.6436 - val_acc: 0.8870
 421 Epoch 209/1000
 422 62s 125ms/step - loss: 0.5782 - acc: 0.9013 - val_loss: 0.6619 - val_acc: 0.8765
 423 Epoch 210/1000
 424 62s 125ms/step - loss: 0.5786 - acc: 0.9009 - val_loss: 0.6482 - val_acc: 0.8798
 425 Epoch 211/1000
 426 62s 125ms/step - loss: 0.5727 - acc: 0.9039 - val_loss: 0.6324 - val_acc: 0.8879
 427 Epoch 212/1000
 428 62s 125ms/step - loss: 0.5801 - acc: 0.9024 - val_loss: 0.6353 - val_acc: 0.8846
 429 Epoch 213/1000
 430 62s 124ms/step - loss: 0.5747 - acc: 0.9046 - val_loss: 0.6388 - val_acc: 0.8827
 431 Epoch 214/1000
 432 62s 125ms/step - loss: 0.5728 - acc: 0.9043 - val_loss: 0.6470 - val_acc: 0.8822
 433 Epoch 215/1000
 434 62s 125ms/step - loss: 0.5747 - acc: 0.9045 - val_loss: 0.6394 - val_acc: 0.8889
 435 Epoch 216/1000
 436 62s 125ms/step - loss: 0.5734 - acc: 0.9041 - val_loss: 0.6465 - val_acc: 0.8853
 437 Epoch 217/1000
 438 62s 125ms/step - loss: 0.5696 - acc: 0.9072 - val_loss: 0.6500 - val_acc: 0.8838
 439 Epoch 218/1000
 440 62s 124ms/step - loss: 0.5727 - acc: 0.9055 - val_loss: 0.6214 - val_acc: 0.8929
 441 Epoch 219/1000
 442 62s 125ms/step - loss: 0.5737 - acc: 0.9048 - val_loss: 0.6288 - val_acc: 0.8885
 443 Epoch 220/1000
 444 62s 125ms/step - loss: 0.5734 - acc: 0.9039 - val_loss: 0.6399 - val_acc: 0.8863
 445 Epoch 221/1000
 446 62s 124ms/step - loss: 0.5740 - acc: 0.9047 - val_loss: 0.6256 - val_acc: 0.8906
 447 Epoch 222/1000
 448 62s 125ms/step - loss: 0.5714 - acc: 0.9054 - val_loss: 0.6390 - val_acc: 0.8812
 449 Epoch 223/1000
 450 62s 125ms/step - loss: 0.5795 - acc: 0.9017 - val_loss: 0.6447 - val_acc: 0.8851
 451 Epoch 224/1000
 452 62s 125ms/step - loss: 0.5687 - acc: 0.9062 - val_loss: 0.6262 - val_acc: 0.8902
 453 Epoch 225/1000
 454 62s 125ms/step - loss: 0.5713 - acc: 0.9034 - val_loss: 0.6379 - val_acc: 0.8857
 455 Epoch 226/1000
 456 62s 125ms/step - loss: 0.5735 - acc: 0.9047 - val_loss: 0.6319 - val_acc: 0.8888
 457 Epoch 227/1000
 458 62s 125ms/step - loss: 0.5749 - acc: 0.9044 - val_loss: 0.6353 - val_acc: 0.8870
 459 Epoch 228/1000
 460 62s 125ms/step - loss: 0.5699 - acc: 0.9056 - val_loss: 0.6265 - val_acc: 0.8927
 461 Epoch 229/1000
 462 62s 125ms/step - loss: 0.5718 - acc: 0.9048 - val_loss: 0.6291 - val_acc: 0.8865
 463 Epoch 230/1000
 464 62s 125ms/step - loss: 0.5678 - acc: 0.9060 - val_loss: 0.6318 - val_acc: 0.8873
 465 Epoch 231/1000
 466 62s 125ms/step - loss: 0.5699 - acc: 0.9060 - val_loss: 0.6378 - val_acc: 0.8859
 467 Epoch 232/1000
 468 62s 125ms/step - loss: 0.5694 - acc: 0.9054 - val_loss: 0.6263 - val_acc: 0.8896
 469 Epoch 233/1000
 470 62s 125ms/step - loss: 0.5748 - acc: 0.9040 - val_loss: 0.6202 - val_acc: 0.8964
 471 Epoch 234/1000
 472 62s 125ms/step - loss: 0.5693 - acc: 0.9070 - val_loss: 0.6358 - val_acc: 0.8882
 473 Epoch 235/1000
 474 62s 125ms/step - loss: 0.5712 - acc: 0.9040 - val_loss: 0.6529 - val_acc: 0.8811
 475 Epoch 236/1000
 476 62s 125ms/step - loss: 0.5663 - acc: 0.9060 - val_loss: 0.6340 - val_acc: 0.8873
 477 Epoch 237/1000
 478 62s 124ms/step - loss: 0.5717 - acc: 0.9047 - val_loss: 0.6391 - val_acc: 0.8869
 479 Epoch 238/1000
 480 62s 125ms/step - loss: 0.5692 - acc: 0.9060 - val_loss: 0.6419 - val_acc: 0.8849
 481 Epoch 239/1000
 482 62s 125ms/step - loss: 0.5732 - acc: 0.9059 - val_loss: 0.6274 - val_acc: 0.8862
 483 Epoch 240/1000
 484 62s 125ms/step - loss: 0.5735 - acc: 0.9036 - val_loss: 0.6352 - val_acc: 0.8881
 485 Epoch 241/1000
 486 62s 125ms/step - loss: 0.5672 - acc: 0.9064 - val_loss: 0.6263 - val_acc: 0.8871
 487 Epoch 242/1000
 488 62s 125ms/step - loss: 0.5734 - acc: 0.9050 - val_loss: 0.6380 - val_acc: 0.8868
 489 Epoch 243/1000
 490 62s 125ms/step - loss: 0.5693 - acc: 0.9061 - val_loss: 0.6313 - val_acc: 0.8865
 491 Epoch 244/1000
 492 62s 124ms/step - loss: 0.5663 - acc: 0.9068 - val_loss: 0.6544 - val_acc: 0.8811
 493 Epoch 245/1000
 494 62s 125ms/step - loss: 0.5697 - acc: 0.9066 - val_loss: 0.6647 - val_acc: 0.8791
 495 Epoch 246/1000
 496 62s 124ms/step - loss: 0.5749 - acc: 0.9026 - val_loss: 0.6451 - val_acc: 0.8802
 497 Epoch 247/1000
 498 62s 125ms/step - loss: 0.5712 - acc: 0.9053 - val_loss: 0.6448 - val_acc: 0.8837
 499 Epoch 248/1000
 500 62s 125ms/step - loss: 0.5739 - acc: 0.9061 - val_loss: 0.6252 - val_acc: 0.8923
 501 Epoch 249/1000
 502 62s 125ms/step - loss: 0.5716 - acc: 0.9051 - val_loss: 0.6571 - val_acc: 0.8809
 503 Epoch 250/1000
 504 62s 125ms/step - loss: 0.5703 - acc: 0.9062 - val_loss: 0.6289 - val_acc: 0.8879
 505 Epoch 251/1000
 506 62s 125ms/step - loss: 0.5689 - acc: 0.9055 - val_loss: 0.6302 - val_acc: 0.8898
 507 Epoch 252/1000
 508 62s 124ms/step - loss: 0.5650 - acc: 0.9066 - val_loss: 0.6394 - val_acc: 0.8863
 509 Epoch 253/1000
 510 62s 124ms/step - loss: 0.5688 - acc: 0.9067 - val_loss: 0.6249 - val_acc: 0.8884
 511 Epoch 254/1000
 512 62s 125ms/step - loss: 0.5681 - acc: 0.9062 - val_loss: 0.6199 - val_acc: 0.8918
 513 Epoch 255/1000
 514 62s 125ms/step - loss: 0.5661 - acc: 0.9079 - val_loss: 0.6540 - val_acc: 0.8822
 515 Epoch 256/1000
 516 62s 125ms/step - loss: 0.5697 - acc: 0.9055 - val_loss: 0.6553 - val_acc: 0.8796
 517 Epoch 257/1000
 518 62s 125ms/step - loss: 0.5672 - acc: 0.9067 - val_loss: 0.6183 - val_acc: 0.8944
 519 Epoch 258/1000
 520 62s 125ms/step - loss: 0.5632 - acc: 0.9088 - val_loss: 0.6358 - val_acc: 0.8896
 521 Epoch 259/1000
 522 62s 125ms/step - loss: 0.5679 - acc: 0.9063 - val_loss: 0.6347 - val_acc: 0.8866
 523 Epoch 260/1000
 524 62s 124ms/step - loss: 0.5638 - acc: 0.9071 - val_loss: 0.6528 - val_acc: 0.8803
 525 Epoch 261/1000
 526 62s 124ms/step - loss: 0.5723 - acc: 0.9055 - val_loss: 0.6438 - val_acc: 0.8848
 527 Epoch 262/1000
 528 62s 125ms/step - loss: 0.5671 - acc: 0.9073 - val_loss: 0.6208 - val_acc: 0.8894
 529 Epoch 263/1000
 530 62s 125ms/step - loss: 0.5677 - acc: 0.9057 - val_loss: 0.6412 - val_acc: 0.8846
 531 Epoch 264/1000
 532 62s 124ms/step - loss: 0.5684 - acc: 0.9074 - val_loss: 0.6129 - val_acc: 0.8942
 533 Epoch 265/1000
 534 62s 125ms/step - loss: 0.5701 - acc: 0.9072 - val_loss: 0.6283 - val_acc: 0.8881
 535 Epoch 266/1000
 536 62s 125ms/step - loss: 0.5671 - acc: 0.9073 - val_loss: 0.6324 - val_acc: 0.8862
 537 Epoch 267/1000
 538 62s 125ms/step - loss: 0.5630 - acc: 0.9078 - val_loss: 0.6319 - val_acc: 0.8862
 539 Epoch 268/1000
 540 62s 125ms/step - loss: 0.5625 - acc: 0.9078 - val_loss: 0.6415 - val_acc: 0.8786
 541 Epoch 269/1000
 542 62s 125ms/step - loss: 0.5605 - acc: 0.9084 - val_loss: 0.6366 - val_acc: 0.8871
 543 Epoch 270/1000
 544 62s 125ms/step - loss: 0.5639 - acc: 0.9076 - val_loss: 0.6345 - val_acc: 0.8890
 545 Epoch 271/1000
 546 62s 124ms/step - loss: 0.5692 - acc: 0.9052 - val_loss: 0.6379 - val_acc: 0.8817
 547 Epoch 272/1000
 548 62s 125ms/step - loss: 0.5650 - acc: 0.9071 - val_loss: 0.6334 - val_acc: 0.8867
 549 Epoch 273/1000
 550 62s 125ms/step - loss: 0.5651 - acc: 0.9082 - val_loss: 0.6315 - val_acc: 0.8869
 551 Epoch 274/1000
 552 62s 125ms/step - loss: 0.5631 - acc: 0.9092 - val_loss: 0.6232 - val_acc: 0.8899
 553 Epoch 275/1000
 554 62s 125ms/step - loss: 0.5642 - acc: 0.9086 - val_loss: 0.6296 - val_acc: 0.8907
 555 Epoch 276/1000
 556 62s 124ms/step - loss: 0.5663 - acc: 0.9076 - val_loss: 0.6068 - val_acc: 0.8949
 557 Epoch 277/1000
 558 62s 125ms/step - loss: 0.5685 - acc: 0.9057 - val_loss: 0.6036 - val_acc: 0.8994
 559 Epoch 278/1000
 560 62s 125ms/step - loss: 0.5659 - acc: 0.9087 - val_loss: 0.6275 - val_acc: 0.8889
 561 Epoch 279/1000
 562 62s 124ms/step - loss: 0.5677 - acc: 0.9065 - val_loss: 0.6267 - val_acc: 0.8872
 563 Epoch 280/1000
 564 62s 124ms/step - loss: 0.5694 - acc: 0.9060 - val_loss: 0.6318 - val_acc: 0.8881
 565 Epoch 281/1000
 566 62s 125ms/step - loss: 0.5601 - acc: 0.9100 - val_loss: 0.6203 - val_acc: 0.8932
 567 Epoch 282/1000
 568 62s 125ms/step - loss: 0.5631 - acc: 0.9071 - val_loss: 0.6395 - val_acc: 0.8856
 569 Epoch 283/1000
 570 62s 125ms/step - loss: 0.5646 - acc: 0.9088 - val_loss: 0.6373 - val_acc: 0.8895
 571 Epoch 284/1000
 572 62s 124ms/step - loss: 0.5605 - acc: 0.9083 - val_loss: 0.6456 - val_acc: 0.8836
 573 Epoch 285/1000
 574 62s 124ms/step - loss: 0.5618 - acc: 0.9094 - val_loss: 0.6225 - val_acc: 0.8900
 575 Epoch 286/1000
 576 62s 124ms/step - loss: 0.5683 - acc: 0.9061 - val_loss: 0.6444 - val_acc: 0.8853
 577 Epoch 287/1000
 578 62s 124ms/step - loss: 0.5661 - acc: 0.9075 - val_loss: 0.6479 - val_acc: 0.8834
 579 Epoch 288/1000
 580 62s 125ms/step - loss: 0.5622 - acc: 0.9099 - val_loss: 0.6137 - val_acc: 0.8955
 581 Epoch 289/1000
 582 62s 125ms/step - loss: 0.5630 - acc: 0.9075 - val_loss: 0.6212 - val_acc: 0.8944
 583 Epoch 290/1000
 584 62s 125ms/step - loss: 0.5621 - acc: 0.9084 - val_loss: 0.6434 - val_acc: 0.8861
 585 Epoch 291/1000
 586 62s 125ms/step - loss: 0.5656 - acc: 0.9087 - val_loss: 0.6248 - val_acc: 0.8911
 587 Epoch 292/1000
 588 62s 124ms/step - loss: 0.5625 - acc: 0.9085 - val_loss: 0.6322 - val_acc: 0.8902
 589 Epoch 293/1000
 590 62s 125ms/step - loss: 0.5637 - acc: 0.9094 - val_loss: 0.6321 - val_acc: 0.8867
 591 Epoch 294/1000
 592 62s 125ms/step - loss: 0.5668 - acc: 0.9070 - val_loss: 0.6236 - val_acc: 0.8887
 593 Epoch 295/1000
 594 62s 125ms/step - loss: 0.5622 - acc: 0.9091 - val_loss: 0.6359 - val_acc: 0.8880
 595 Epoch 296/1000
 596 62s 125ms/step - loss: 0.5614 - acc: 0.9094 - val_loss: 0.6290 - val_acc: 0.8901
 597 Epoch 297/1000
 598 62s 125ms/step - loss: 0.5610 - acc: 0.9092 - val_loss: 0.6358 - val_acc: 0.8905
 599 Epoch 298/1000
 600 62s 125ms/step - loss: 0.5584 - acc: 0.9103 - val_loss: 0.6199 - val_acc: 0.8935
 601 Epoch 299/1000
 602 62s 125ms/step - loss: 0.5660 - acc: 0.9069 - val_loss: 0.6153 - val_acc: 0.8957
 603 Epoch 300/1000
 604 62s 124ms/step - loss: 0.5578 - acc: 0.9106 - val_loss: 0.6273 - val_acc: 0.8939
 605 Epoch 301/1000
 606 lr changed to 0.010000000149011612
 607 62s 124ms/step - loss: 0.4654 - acc: 0.9431 - val_loss: 0.5402 - val_acc: 0.9195
 608 Epoch 302/1000
 609 63s 125ms/step - loss: 0.4160 - acc: 0.9576 - val_loss: 0.5281 - val_acc: 0.9208
 610 Epoch 303/1000
 611 63s 125ms/step - loss: 0.3942 - acc: 0.9640 - val_loss: 0.5227 - val_acc: 0.9234
 612 Epoch 304/1000
 613 62s 125ms/step - loss: 0.3791 - acc: 0.9677 - val_loss: 0.5185 - val_acc: 0.9257
 614 Epoch 305/1000
 615 62s 125ms/step - loss: 0.3685 - acc: 0.9689 - val_loss: 0.5151 - val_acc: 0.9273
 616 Epoch 306/1000
 617 62s 125ms/step - loss: 0.3548 - acc: 0.9717 - val_loss: 0.5098 - val_acc: 0.9268
 618 Epoch 307/1000
 619 62s 125ms/step - loss: 0.3455 - acc: 0.9737 - val_loss: 0.5064 - val_acc: 0.9260
 620 Epoch 308/1000
 621 62s 124ms/step - loss: 0.3382 - acc: 0.9758 - val_loss: 0.5038 - val_acc: 0.9268
 622 Epoch 309/1000
 623 62s 125ms/step - loss: 0.3281 - acc: 0.9766 - val_loss: 0.5063 - val_acc: 0.9248
 624 Epoch 310/1000
 625 62s 125ms/step - loss: 0.3208 - acc: 0.9779 - val_loss: 0.5018 - val_acc: 0.9242
 626 Epoch 311/1000
 627 62s 125ms/step - loss: 0.3133 - acc: 0.9792 - val_loss: 0.5024 - val_acc: 0.9248
 628 Epoch 312/1000
 629 62s 125ms/step - loss: 0.3078 - acc: 0.9790 - val_loss: 0.4962 - val_acc: 0.9250
 630 Epoch 313/1000
 631 63s 125ms/step - loss: 0.2999 - acc: 0.9810 - val_loss: 0.5008 - val_acc: 0.9234
 632 Epoch 314/1000
 633 62s 125ms/step - loss: 0.2930 - acc: 0.9817 - val_loss: 0.4988 - val_acc: 0.9227
 634 Epoch 315/1000
 635 62s 125ms/step - loss: 0.2868 - acc: 0.9824 - val_loss: 0.4896 - val_acc: 0.9221
 636 Epoch 316/1000
 637 62s 125ms/step - loss: 0.2815 - acc: 0.9827 - val_loss: 0.4896 - val_acc: 0.9255
 638 Epoch 317/1000
 639 62s 125ms/step - loss: 0.2752 - acc: 0.9834 - val_loss: 0.4882 - val_acc: 0.9233
 640 Epoch 318/1000
 641 62s 125ms/step - loss: 0.2719 - acc: 0.9836 - val_loss: 0.4935 - val_acc: 0.9225
 642 Epoch 319/1000
 643 62s 125ms/step - loss: 0.2659 - acc: 0.9839 - val_loss: 0.4843 - val_acc: 0.9230
 644 Epoch 320/1000
 645 62s 125ms/step - loss: 0.2607 - acc: 0.9845 - val_loss: 0.4881 - val_acc: 0.9221
 646 Epoch 321/1000
 647 62s 125ms/step - loss: 0.2561 - acc: 0.9850 - val_loss: 0.4871 - val_acc: 0.9200
 648 Epoch 322/1000
 649 62s 125ms/step - loss: 0.2543 - acc: 0.9846 - val_loss: 0.4793 - val_acc: 0.9227
 650 Epoch 323/1000
 651 62s 125ms/step - loss: 0.2500 - acc: 0.9852 - val_loss: 0.4661 - val_acc: 0.9221
 652 Epoch 324/1000
 653 62s 125ms/step - loss: 0.2459 - acc: 0.9851 - val_loss: 0.4621 - val_acc: 0.9260
 654 Epoch 325/1000
 655 62s 125ms/step - loss: 0.2410 - acc: 0.9855 - val_loss: 0.4690 - val_acc: 0.9236
 656 Epoch 326/1000
 657 62s 125ms/step - loss: 0.2352 - acc: 0.9866 - val_loss: 0.4689 - val_acc: 0.9227
 658 Epoch 327/1000
 659 62s 125ms/step - loss: 0.2334 - acc: 0.9860 - val_loss: 0.4711 - val_acc: 0.9205
 660 Epoch 328/1000
 661 62s 125ms/step - loss: 0.2296 - acc: 0.9863 - val_loss: 0.4718 - val_acc: 0.9231
 662 Epoch 329/1000
 663 62s 125ms/step - loss: 0.2259 - acc: 0.9869 - val_loss: 0.4648 - val_acc: 0.9212
 664 Epoch 330/1000
 665 62s 125ms/step - loss: 0.2211 - acc: 0.9875 - val_loss: 0.4697 - val_acc: 0.9229
 666 Epoch 331/1000
 667 62s 125ms/step - loss: 0.2228 - acc: 0.9861 - val_loss: 0.4697 - val_acc: 0.9200
 668 Epoch 332/1000
 669 62s 124ms/step - loss: 0.2175 - acc: 0.9862 - val_loss: 0.4546 - val_acc: 0.9224
 670 Epoch 333/1000
 671 62s 125ms/step - loss: 0.2143 - acc: 0.9872 - val_loss: 0.4580 - val_acc: 0.9229
 672 Epoch 334/1000
 673 62s 124ms/step - loss: 0.2107 - acc: 0.9878 - val_loss: 0.4492 - val_acc: 0.9197
 674 Epoch 335/1000
 675 62s 125ms/step - loss: 0.2080 - acc: 0.9875 - val_loss: 0.4626 - val_acc: 0.9184
 676 Epoch 336/1000
 677 62s 125ms/step - loss: 0.2066 - acc: 0.9870 - val_loss: 0.4614 - val_acc: 0.9180
 678 Epoch 337/1000
 679 62s 125ms/step - loss: 0.2045 - acc: 0.9871 - val_loss: 0.4447 - val_acc: 0.9210
 680 Epoch 338/1000
 681 62s 125ms/step - loss: 0.2001 - acc: 0.9874 - val_loss: 0.4554 - val_acc: 0.9207
 682 Epoch 339/1000
 683 62s 125ms/step - loss: 0.1991 - acc: 0.9877 - val_loss: 0.4527 - val_acc: 0.9206
 684 Epoch 340/1000
 685 62s 125ms/step - loss: 0.1958 - acc: 0.9878 - val_loss: 0.4630 - val_acc: 0.9157
 686 Epoch 341/1000
 687 62s 125ms/step - loss: 0.1957 - acc: 0.9868 - val_loss: 0.4447 - val_acc: 0.9225
 688 Epoch 342/1000
 689 62s 125ms/step - loss: 0.1939 - acc: 0.9870 - val_loss: 0.4558 - val_acc: 0.9160
 690 Epoch 343/1000
 691 63s 125ms/step - loss: 0.1921 - acc: 0.9866 - val_loss: 0.4451 - val_acc: 0.9195
 692 Epoch 344/1000
 693 62s 125ms/step - loss: 0.1929 - acc: 0.9860 - val_loss: 0.4431 - val_acc: 0.9213
 694 Epoch 345/1000
 695 62s 125ms/step - loss: 0.1889 - acc: 0.9864 - val_loss: 0.4386 - val_acc: 0.9213
 696 Epoch 346/1000
 697 62s 125ms/step - loss: 0.1865 - acc: 0.9869 - val_loss: 0.4504 - val_acc: 0.9167
 698 Epoch 347/1000
 699 62s 125ms/step - loss: 0.1847 - acc: 0.9870 - val_loss: 0.4285 - val_acc: 0.9196
 700 Epoch 348/1000
 701 62s 125ms/step - loss: 0.1836 - acc: 0.9865 - val_loss: 0.4252 - val_acc: 0.9220
 702 Epoch 349/1000
 703 62s 124ms/step - loss: 0.1827 - acc: 0.9864 - val_loss: 0.4364 - val_acc: 0.9205
 704 Epoch 350/1000
 705 62s 125ms/step - loss: 0.1800 - acc: 0.9870 - val_loss: 0.4379 - val_acc: 0.9214
 706 Epoch 351/1000
 707 62s 125ms/step - loss: 0.1793 - acc: 0.9869 - val_loss: 0.4343 - val_acc: 0.9193
 708 Epoch 352/1000
 709 62s 125ms/step - loss: 0.1768 - acc: 0.9873 - val_loss: 0.4342 - val_acc: 0.9216
 710 Epoch 353/1000
 711 62s 125ms/step - loss: 0.1784 - acc: 0.9855 - val_loss: 0.4390 - val_acc: 0.9192
 712 Epoch 354/1000
 713 63s 125ms/step - loss: 0.1763 - acc: 0.9860 - val_loss: 0.4257 - val_acc: 0.9197
 714 Epoch 355/1000
 715 62s 125ms/step - loss: 0.1724 - acc: 0.9867 - val_loss: 0.4276 - val_acc: 0.9191
 716 Epoch 356/1000
 717 62s 124ms/step - loss: 0.1727 - acc: 0.9867 - val_loss: 0.4395 - val_acc: 0.9202
 718 Epoch 357/1000
 719 62s 124ms/step - loss: 0.1710 - acc: 0.9860 - val_loss: 0.4386 - val_acc: 0.9174
 720 Epoch 358/1000
 721 62s 125ms/step - loss: 0.1722 - acc: 0.9852 - val_loss: 0.4284 - val_acc: 0.9179
 722 Epoch 359/1000
 723 62s 125ms/step - loss: 0.1691 - acc: 0.9870 - val_loss: 0.4245 - val_acc: 0.9213
 724 Epoch 360/1000
 725 62s 125ms/step - loss: 0.1687 - acc: 0.9859 - val_loss: 0.4181 - val_acc: 0.9153
 726 Epoch 361/1000
 727 62s 125ms/step - loss: 0.1684 - acc: 0.9861 - val_loss: 0.4114 - val_acc: 0.9186
 728 Epoch 362/1000
 729 62s 125ms/step - loss: 0.1650 - acc: 0.9867 - val_loss: 0.4036 - val_acc: 0.9195
 730 Epoch 363/1000
 731 62s 125ms/step - loss: 0.1690 - acc: 0.9853 - val_loss: 0.4161 - val_acc: 0.9183
 732 Epoch 364/1000
 733 62s 125ms/step - loss: 0.1651 - acc: 0.9859 - val_loss: 0.4265 - val_acc: 0.9159
 734 Epoch 365/1000
 735 62s 124ms/step - loss: 0.1666 - acc: 0.9853 - val_loss: 0.4090 - val_acc: 0.9208
 736 Epoch 366/1000
 737 62s 124ms/step - loss: 0.1650 - acc: 0.9854 - val_loss: 0.4077 - val_acc: 0.9237
 738 Epoch 367/1000
 739 62s 124ms/step - loss: 0.1656 - acc: 0.9850 - val_loss: 0.4051 - val_acc: 0.9215
 740 Epoch 368/1000
 741 62s 124ms/step - loss: 0.1616 - acc: 0.9860 - val_loss: 0.4279 - val_acc: 0.9154
 742 Epoch 369/1000
 743 62s 124ms/step - loss: 0.1657 - acc: 0.9844 - val_loss: 0.4328 - val_acc: 0.9136
 744 Epoch 370/1000
 745 62s 124ms/step - loss: 0.1624 - acc: 0.9851 - val_loss: 0.4312 - val_acc: 0.9144
 746 Epoch 371/1000
 747 62s 124ms/step - loss: 0.1648 - acc: 0.9842 - val_loss: 0.4086 - val_acc: 0.9181
 748 Epoch 372/1000
 749 62s 124ms/step - loss: 0.1615 - acc: 0.9853 - val_loss: 0.4178 - val_acc: 0.9195
 750 Epoch 373/1000
 751 62s 123ms/step - loss: 0.1620 - acc: 0.9846 - val_loss: 0.3955 - val_acc: 0.9195
 752 Epoch 374/1000
 753 62s 123ms/step - loss: 0.1602 - acc: 0.9853 - val_loss: 0.4074 - val_acc: 0.9197
 754 Epoch 375/1000
 755 62s 124ms/step - loss: 0.1612 - acc: 0.9842 - val_loss: 0.4081 - val_acc: 0.9187
 756 Epoch 376/1000
 757 62s 124ms/step - loss: 0.1611 - acc: 0.9845 - val_loss: 0.4138 - val_acc: 0.9174
 758 Epoch 377/1000
 759 62s 124ms/step - loss: 0.1603 - acc: 0.9840 - val_loss: 0.4135 - val_acc: 0.9168
 760 Epoch 378/1000
 761 62s 124ms/step - loss: 0.1597 - acc: 0.9840 - val_loss: 0.4254 - val_acc: 0.9158
 762 Epoch 379/1000
 763 62s 123ms/step - loss: 0.1610 - acc: 0.9845 - val_loss: 0.4306 - val_acc: 0.9155
 764 Epoch 380/1000
 765 62s 124ms/step - loss: 0.1590 - acc: 0.9842 - val_loss: 0.4183 - val_acc: 0.9137
 766 Epoch 381/1000
 767 62s 124ms/step - loss: 0.1600 - acc: 0.9835 - val_loss: 0.4124 - val_acc: 0.9180
 768 Epoch 382/1000
 769 62s 123ms/step - loss: 0.1574 - acc: 0.9840 - val_loss: 0.4224 - val_acc: 0.9173
 770 Epoch 383/1000
 771 62s 124ms/step - loss: 0.1564 - acc: 0.9850 - val_loss: 0.4164 - val_acc: 0.9182
 772 Epoch 384/1000
 773 62s 124ms/step - loss: 0.1535 - acc: 0.9855 - val_loss: 0.4009 - val_acc: 0.9218
 774 Epoch 385/1000
 775 62s 124ms/step - loss: 0.1577 - acc: 0.9830 - val_loss: 0.4206 - val_acc: 0.9150
 776 Epoch 386/1000
 777 62s 125ms/step - loss: 0.1576 - acc: 0.9838 - val_loss: 0.4181 - val_acc: 0.9158
 778 Epoch 387/1000
 779 62s 125ms/step - loss: 0.1567 - acc: 0.9843 - val_loss: 0.4205 - val_acc: 0.9139
 780 Epoch 388/1000
 781 62s 124ms/step - loss: 0.1548 - acc: 0.9845 - val_loss: 0.4168 - val_acc: 0.9160
 782 Epoch 389/1000
 783 62s 124ms/step - loss: 0.1570 - acc: 0.9832 - val_loss: 0.4166 - val_acc: 0.9178
 784 Epoch 390/1000
 785 62s 124ms/step - loss: 0.1557 - acc: 0.9842 - val_loss: 0.4142 - val_acc: 0.9159
 786 Epoch 391/1000
 787 62s 125ms/step - loss: 0.1592 - acc: 0.9826 - val_loss: 0.4110 - val_acc: 0.9186
 788 Epoch 392/1000
 789 62s 125ms/step - loss: 0.1577 - acc: 0.9832 - val_loss: 0.4116 - val_acc: 0.9180
 790 Epoch 393/1000
 791 62s 125ms/step - loss: 0.1570 - acc: 0.9838 - val_loss: 0.4033 - val_acc: 0.9168
 792 Epoch 394/1000
 793 62s 124ms/step - loss: 0.1564 - acc: 0.9838 - val_loss: 0.4234 - val_acc: 0.9134
 794 Epoch 395/1000
 795 62s 125ms/step - loss: 0.1587 - acc: 0.9825 - val_loss: 0.3980 - val_acc: 0.9216
 796 Epoch 396/1000
 797 62s 124ms/step - loss: 0.1562 - acc: 0.9833 - val_loss: 0.4011 - val_acc: 0.9188
 798 Epoch 397/1000
 799 62s 124ms/step - loss: 0.1553 - acc: 0.9838 - val_loss: 0.4025 - val_acc: 0.9161
 800 Epoch 398/1000
 801 62s 124ms/step - loss: 0.1531 - acc: 0.9845 - val_loss: 0.3951 - val_acc: 0.9195
 802 Epoch 399/1000
 803 62s 124ms/step - loss: 0.1521 - acc: 0.9848 - val_loss: 0.4025 - val_acc: 0.9188
 804 Epoch 400/1000
 805 62s 125ms/step - loss: 0.1554 - acc: 0.9833 - val_loss: 0.4085 - val_acc: 0.9161
 806 Epoch 401/1000
 807 62s 125ms/step - loss: 0.1542 - acc: 0.9841 - val_loss: 0.4103 - val_acc: 0.9202
 808 Epoch 402/1000
 809 62s 125ms/step - loss: 0.1528 - acc: 0.9844 - val_loss: 0.4119 - val_acc: 0.9168
 810 Epoch 403/1000
 811 62s 125ms/step - loss: 0.1566 - acc: 0.9825 - val_loss: 0.4014 - val_acc: 0.9186
 812 Epoch 404/1000
 813 62s 125ms/step - loss: 0.1523 - acc: 0.9843 - val_loss: 0.4243 - val_acc: 0.9147
 814 Epoch 405/1000
 815 62s 124ms/step - loss: 0.1520 - acc: 0.9850 - val_loss: 0.4182 - val_acc: 0.9159
 816 Epoch 406/1000
 817 62s 124ms/step - loss: 0.1521 - acc: 0.9835 - val_loss: 0.4021 - val_acc: 0.9178
 818 Epoch 407/1000
 819 62s 125ms/step - loss: 0.1566 - acc: 0.9823 - val_loss: 0.4143 - val_acc: 0.9150
 820 Epoch 408/1000
 821 62s 124ms/step - loss: 0.1520 - acc: 0.9846 - val_loss: 0.3987 - val_acc: 0.9200
 822 Epoch 409/1000
 823 62s 125ms/step - loss: 0.1538 - acc: 0.9837 - val_loss: 0.4051 - val_acc: 0.9160
 824 Epoch 410/1000
 825 62s 124ms/step - loss: 0.1541 - acc: 0.9824 - val_loss: 0.4159 - val_acc: 0.9133
 826 Epoch 411/1000
 827 62s 124ms/step - loss: 0.1540 - acc: 0.9834 - val_loss: 0.4171 - val_acc: 0.9119
 828 Epoch 412/1000
 829 62s 123ms/step - loss: 0.1523 - acc: 0.9843 - val_loss: 0.4103 - val_acc: 0.9154
 830 Epoch 413/1000
 831 62s 124ms/step - loss: 0.1502 - acc: 0.9845 - val_loss: 0.4164 - val_acc: 0.9148
 832 Epoch 414/1000
 833 62s 125ms/step - loss: 0.1585 - acc: 0.9811 - val_loss: 0.4115 - val_acc: 0.9166
 834 Epoch 415/1000
 835 62s 125ms/step - loss: 0.1508 - acc: 0.9844 - val_loss: 0.4210 - val_acc: 0.9119
 836 Epoch 416/1000
 837 62s 125ms/step - loss: 0.1531 - acc: 0.9835 - val_loss: 0.4125 - val_acc: 0.9169
 838 Epoch 417/1000
 839 62s 125ms/step - loss: 0.1502 - acc: 0.9838 - val_loss: 0.4174 - val_acc: 0.9139
 840 Epoch 418/1000
 841 62s 125ms/step - loss: 0.1477 - acc: 0.9853 - val_loss: 0.4091 - val_acc: 0.9173
 842 Epoch 419/1000
 843 62s 125ms/step - loss: 0.1529 - acc: 0.9831 - val_loss: 0.4144 - val_acc: 0.9151
 844 Epoch 420/1000
 845 62s 124ms/step - loss: 0.1546 - acc: 0.9824 - val_loss: 0.3990 - val_acc: 0.9175
 846 Epoch 421/1000
 847 62s 125ms/step - loss: 0.1502 - acc: 0.9843 - val_loss: 0.3900 - val_acc: 0.9187
 848 Epoch 422/1000
 849 62s 125ms/step - loss: 0.1500 - acc: 0.9843 - val_loss: 0.4025 - val_acc: 0.9168
 850 Epoch 423/1000
 851 62s 125ms/step - loss: 0.1551 - acc: 0.9824 - val_loss: 0.4141 - val_acc: 0.9173
 852 Epoch 424/1000
 853 62s 125ms/step - loss: 0.1507 - acc: 0.9841 - val_loss: 0.4181 - val_acc: 0.9132
 854 Epoch 425/1000
 855 62s 125ms/step - loss: 0.1517 - acc: 0.9835 - val_loss: 0.4149 - val_acc: 0.9178
 856 Epoch 426/1000
 857 62s 125ms/step - loss: 0.1533 - acc: 0.9830 - val_loss: 0.4080 - val_acc: 0.9159
 858 Epoch 427/1000
 859 62s 125ms/step - loss: 0.1500 - acc: 0.9840 - val_loss: 0.4227 - val_acc: 0.9111
 860 Epoch 428/1000
 861 62s 125ms/step - loss: 0.1515 - acc: 0.9840 - val_loss: 0.4033 - val_acc: 0.9161
 862 Epoch 429/1000
 863 62s 125ms/step - loss: 0.1487 - acc: 0.9846 - val_loss: 0.4152 - val_acc: 0.9151
 864 Epoch 430/1000
 865 62s 124ms/step - loss: 0.1504 - acc: 0.9838 - val_loss: 0.4093 - val_acc: 0.9144
 866 Epoch 431/1000
 867 62s 125ms/step - loss: 0.1479 - acc: 0.9849 - val_loss: 0.3920 - val_acc: 0.9185
 868 Epoch 432/1000
 869 62s 125ms/step - loss: 0.1486 - acc: 0.9847 - val_loss: 0.4119 - val_acc: 0.9136
 870 Epoch 433/1000
 871 62s 124ms/step - loss: 0.1518 - acc: 0.9839 - val_loss: 0.4071 - val_acc: 0.9161
 872 Epoch 434/1000
 873 62s 125ms/step - loss: 0.1526 - acc: 0.9830 - val_loss: 0.4031 - val_acc: 0.9162
 874 Epoch 435/1000
 875 62s 125ms/step - loss: 0.1519 - acc: 0.9835 - val_loss: 0.3861 - val_acc: 0.9210
 876 Epoch 436/1000
 877 62s 125ms/step - loss: 0.1487 - acc: 0.9845 - val_loss: 0.4099 - val_acc: 0.9184
 878 Epoch 437/1000
 879 62s 124ms/step - loss: 0.1531 - acc: 0.9823 - val_loss: 0.4020 - val_acc: 0.9205
 880 Epoch 438/1000
 881 62s 125ms/step - loss: 0.1519 - acc: 0.9833 - val_loss: 0.4032 - val_acc: 0.9179
 882 Epoch 439/1000
 883 62s 125ms/step - loss: 0.1523 - acc: 0.9832 - val_loss: 0.3972 - val_acc: 0.9171
 884 Epoch 440/1000
 885 62s 124ms/step - loss: 0.1512 - acc: 0.9840 - val_loss: 0.3920 - val_acc: 0.9195
 886 Epoch 441/1000
 887 62s 125ms/step - loss: 0.1503 - acc: 0.9835 - val_loss: 0.4231 - val_acc: 0.9130
 888 Epoch 442/1000
 889 62s 125ms/step - loss: 0.1464 - acc: 0.9849 - val_loss: 0.4230 - val_acc: 0.9133
 890 Epoch 443/1000
 891 62s 125ms/step - loss: 0.1509 - acc: 0.9837 - val_loss: 0.4154 - val_acc: 0.9127
 892 Epoch 444/1000
 893 62s 124ms/step - loss: 0.1488 - acc: 0.9846 - val_loss: 0.4284 - val_acc: 0.9120
 894 Epoch 445/1000
 895 62s 124ms/step - loss: 0.1468 - acc: 0.9853 - val_loss: 0.4246 - val_acc: 0.9143
 896 Epoch 446/1000
 897 62s 125ms/step - loss: 0.1499 - acc: 0.9839 - val_loss: 0.4191 - val_acc: 0.9150
 898 Epoch 447/1000
 899 62s 124ms/step - loss: 0.1494 - acc: 0.9841 - val_loss: 0.4146 - val_acc: 0.9183
 900 Epoch 448/1000
 901 62s 125ms/step - loss: 0.1527 - acc: 0.9829 - val_loss: 0.4059 - val_acc: 0.9157
 902 Epoch 449/1000
 903 62s 125ms/step - loss: 0.1491 - acc: 0.9845 - val_loss: 0.4126 - val_acc: 0.9151
 904 Epoch 450/1000
 905 62s 125ms/step - loss: 0.1494 - acc: 0.9842 - val_loss: 0.4150 - val_acc: 0.9152
 906 Epoch 451/1000
 907 62s 124ms/step - loss: 0.1521 - acc: 0.9838 - val_loss: 0.3997 - val_acc: 0.9176
 908 Epoch 452/1000
 909 62s 124ms/step - loss: 0.1518 - acc: 0.9834 - val_loss: 0.4082 - val_acc: 0.9126
 910 Epoch 453/1000
 911 62s 124ms/step - loss: 0.1514 - acc: 0.9835 - val_loss: 0.4053 - val_acc: 0.9177
 912 Epoch 454/1000
 913 62s 124ms/step - loss: 0.1506 - acc: 0.9836 - val_loss: 0.4148 - val_acc: 0.9166
 914 Epoch 455/1000
 915 62s 125ms/step - loss: 0.1493 - acc: 0.9845 - val_loss: 0.3997 - val_acc: 0.9207
 916 Epoch 456/1000
 917 62s 124ms/step - loss: 0.1486 - acc: 0.9842 - val_loss: 0.4132 - val_acc: 0.9148
 918 Epoch 457/1000
 919 62s 125ms/step - loss: 0.1479 - acc: 0.9856 - val_loss: 0.3984 - val_acc: 0.9181
 920 Epoch 458/1000
 921 62s 125ms/step - loss: 0.1515 - acc: 0.9826 - val_loss: 0.4026 - val_acc: 0.9162
 922 Epoch 459/1000
 923 62s 125ms/step - loss: 0.1515 - acc: 0.9838 - val_loss: 0.4098 - val_acc: 0.9137
 924 Epoch 460/1000
 925 62s 124ms/step - loss: 0.1513 - acc: 0.9828 - val_loss: 0.4047 - val_acc: 0.9180
 926 Epoch 461/1000
 927 62s 125ms/step - loss: 0.1466 - acc: 0.9855 - val_loss: 0.3965 - val_acc: 0.9193
 928 Epoch 462/1000
 929 62s 124ms/step - loss: 0.1518 - acc: 0.9835 - val_loss: 0.4046 - val_acc: 0.9151
 930 Epoch 463/1000
 931 62s 124ms/step - loss: 0.1506 - acc: 0.9839 - val_loss: 0.4084 - val_acc: 0.9140
 932 Epoch 464/1000
 933 62s 125ms/step - loss: 0.1503 - acc: 0.9840 - val_loss: 0.4086 - val_acc: 0.9145
 934 Epoch 465/1000
 935 62s 125ms/step - loss: 0.1523 - acc: 0.9837 - val_loss: 0.3983 - val_acc: 0.9184
 936 Epoch 466/1000
 937 62s 125ms/step - loss: 0.1493 - acc: 0.9839 - val_loss: 0.3957 - val_acc: 0.9187
 938 Epoch 467/1000
 939 62s 124ms/step - loss: 0.1492 - acc: 0.9840 - val_loss: 0.4008 - val_acc: 0.9212
 940 Epoch 468/1000
 941 62s 124ms/step - loss: 0.1474 - acc: 0.9851 - val_loss: 0.4152 - val_acc: 0.9135
 942 Epoch 469/1000
 943 62s 124ms/step - loss: 0.1466 - acc: 0.9851 - val_loss: 0.4033 - val_acc: 0.9179
 944 Epoch 470/1000
 945 62s 124ms/step - loss: 0.1471 - acc: 0.9856 - val_loss: 0.3983 - val_acc: 0.9182
 946 Epoch 471/1000
 947 62s 124ms/step - loss: 0.1473 - acc: 0.9846 - val_loss: 0.4316 - val_acc: 0.9089
 948 Epoch 472/1000
 949 62s 124ms/step - loss: 0.1539 - acc: 0.9828 - val_loss: 0.3924 - val_acc: 0.9199
 950 Epoch 473/1000
 951 62s 125ms/step - loss: 0.1483 - acc: 0.9847 - val_loss: 0.3929 - val_acc: 0.9197
 952 Epoch 474/1000
 953 62s 124ms/step - loss: 0.1465 - acc: 0.9856 - val_loss: 0.4017 - val_acc: 0.9178
 954 Epoch 475/1000
 955 62s 124ms/step - loss: 0.1541 - acc: 0.9831 - val_loss: 0.4030 - val_acc: 0.9145
 956 Epoch 476/1000
 957 62s 124ms/step - loss: 0.1472 - acc: 0.9853 - val_loss: 0.4143 - val_acc: 0.9160
 958 Epoch 477/1000
 959 62s 124ms/step - loss: 0.1498 - acc: 0.9842 - val_loss: 0.4277 - val_acc: 0.9121
 960 Epoch 478/1000
 961 62s 124ms/step - loss: 0.1473 - acc: 0.9851 - val_loss: 0.4091 - val_acc: 0.9135
 962 Epoch 479/1000
 963 62s 124ms/step - loss: 0.1513 - acc: 0.9838 - val_loss: 0.4148 - val_acc: 0.9163
 964 Epoch 480/1000
 965 62s 124ms/step - loss: 0.1487 - acc: 0.9844 - val_loss: 0.4073 - val_acc: 0.9137
 966 Epoch 481/1000
 967 62s 124ms/step - loss: 0.1446 - acc: 0.9862 - val_loss: 0.4056 - val_acc: 0.9184
 968 Epoch 482/1000
 969 62s 125ms/step - loss: 0.1469 - acc: 0.9847 - val_loss: 0.4189 - val_acc: 0.9152
 970 Epoch 483/1000
 971 62s 125ms/step - loss: 0.1447 - acc: 0.9855 - val_loss: 0.4097 - val_acc: 0.9169
 972 Epoch 484/1000
 973 62s 125ms/step - loss: 0.1490 - acc: 0.9849 - val_loss: 0.3920 - val_acc: 0.9207
 974 Epoch 485/1000
 975 62s 125ms/step - loss: 0.1466 - acc: 0.9853 - val_loss: 0.4131 - val_acc: 0.9162
 976 Epoch 486/1000
 977 62s 124ms/step - loss: 0.1461 - acc: 0.9854 - val_loss: 0.4065 - val_acc: 0.9142
 978 Epoch 487/1000
 979 62s 125ms/step - loss: 0.1479 - acc: 0.9849 - val_loss: 0.4088 - val_acc: 0.9159
 980 Epoch 488/1000
 981 62s 124ms/step - loss: 0.1481 - acc: 0.9842 - val_loss: 0.4170 - val_acc: 0.9150
 982 Epoch 489/1000
 983 62s 125ms/step - loss: 0.1480 - acc: 0.9843 - val_loss: 0.4196 - val_acc: 0.9114
 984 Epoch 490/1000
 985 62s 124ms/step - loss: 0.1471 - acc: 0.9854 - val_loss: 0.4182 - val_acc: 0.9145
 986 Epoch 491/1000
 987 62s 124ms/step - loss: 0.1457 - acc: 0.9859 - val_loss: 0.4128 - val_acc: 0.9173
 988 Epoch 492/1000
 989 62s 124ms/step - loss: 0.1491 - acc: 0.9833 - val_loss: 0.4115 - val_acc: 0.9145
 990 Epoch 493/1000
 991 62s 125ms/step - loss: 0.1483 - acc: 0.9845 - val_loss: 0.4246 - val_acc: 0.9138
 992 Epoch 494/1000
 993 62s 124ms/step - loss: 0.1510 - acc: 0.9837 - val_loss: 0.4197 - val_acc: 0.9140
 994 Epoch 495/1000
 995 62s 125ms/step - loss: 0.1441 - acc: 0.9863 - val_loss: 0.4147 - val_acc: 0.9143
 996 Epoch 496/1000
 997 62s 125ms/step - loss: 0.1445 - acc: 0.9862 - val_loss: 0.4187 - val_acc: 0.9116
 998 Epoch 497/1000
 999 62s 124ms/step - loss: 0.1469 - acc: 0.9854 - val_loss: 0.4090 - val_acc: 0.9181
1000 Epoch 498/1000
1001 62s 125ms/step - loss: 0.1505 - acc: 0.9837 - val_loss: 0.3973 - val_acc: 0.9184
1002 Epoch 499/1000
1003 62s 125ms/step - loss: 0.1479 - acc: 0.9847 - val_loss: 0.4087 - val_acc: 0.9166
1004 Epoch 500/1000
1005 62s 124ms/step - loss: 0.1477 - acc: 0.9848 - val_loss: 0.4203 - val_acc: 0.9136
1006 Epoch 501/1000
1007 62s 124ms/step - loss: 0.1492 - acc: 0.9845 - val_loss: 0.4222 - val_acc: 0.9131
1008 Epoch 502/1000
1009 62s 125ms/step - loss: 0.1470 - acc: 0.9850 - val_loss: 0.4172 - val_acc: 0.9138
1010 Epoch 503/1000
1011 62s 124ms/step - loss: 0.1497 - acc: 0.9839 - val_loss: 0.4288 - val_acc: 0.9133
1012 Epoch 504/1000
1013 62s 125ms/step - loss: 0.1488 - acc: 0.9844 - val_loss: 0.4096 - val_acc: 0.9167
1014 Epoch 505/1000
1015 62s 125ms/step - loss: 0.1467 - acc: 0.9854 - val_loss: 0.4186 - val_acc: 0.9139
1016 Epoch 506/1000
1017 62s 124ms/step - loss: 0.1456 - acc: 0.9856 - val_loss: 0.4107 - val_acc: 0.9167
1018 Epoch 507/1000
1019 62s 125ms/step - loss: 0.1488 - acc: 0.9847 - val_loss: 0.4200 - val_acc: 0.9152
1020 Epoch 508/1000
1021 62s 124ms/step - loss: 0.1480 - acc: 0.9842 - val_loss: 0.4129 - val_acc: 0.9133
1022 Epoch 509/1000
1023 62s 125ms/step - loss: 0.1497 - acc: 0.9838 - val_loss: 0.4048 - val_acc: 0.9136
1024 Epoch 510/1000
1025 62s 124ms/step - loss: 0.1509 - acc: 0.9840 - val_loss: 0.3958 - val_acc: 0.9167
1026 Epoch 511/1000
1027 62s 125ms/step - loss: 0.1478 - acc: 0.9850 - val_loss: 0.4136 - val_acc: 0.9157
1028 Epoch 512/1000
1029 62s 125ms/step - loss: 0.1435 - acc: 0.9863 - val_loss: 0.4051 - val_acc: 0.9185
1030 Epoch 513/1000
1031 62s 125ms/step - loss: 0.1458 - acc: 0.9856 - val_loss: 0.4088 - val_acc: 0.9154
1032 Epoch 514/1000
1033 62s 125ms/step - loss: 0.1483 - acc: 0.9851 - val_loss: 0.4187 - val_acc: 0.9142
1034 Epoch 515/1000
1035 62s 125ms/step - loss: 0.1463 - acc: 0.9853 - val_loss: 0.4347 - val_acc: 0.9096
1036 Epoch 516/1000
1037 62s 125ms/step - loss: 0.1467 - acc: 0.9856 - val_loss: 0.4235 - val_acc: 0.9148
1038 Epoch 517/1000
1039 62s 123ms/step - loss: 0.1452 - acc: 0.9854 - val_loss: 0.4250 - val_acc: 0.9135
1040 Epoch 518/1000
1041 62s 123ms/step - loss: 0.1499 - acc: 0.9844 - val_loss: 0.4048 - val_acc: 0.9174
1042 Epoch 519/1000
1043 62s 123ms/step - loss: 0.1482 - acc: 0.9843 - val_loss: 0.4159 - val_acc: 0.9140
1044 Epoch 520/1000
1045 62s 123ms/step - loss: 0.1483 - acc: 0.9841 - val_loss: 0.4160 - val_acc: 0.9153
1046 Epoch 521/1000
1047 62s 123ms/step - loss: 0.1464 - acc: 0.9858 - val_loss: 0.4009 - val_acc: 0.9165
1048 Epoch 522/1000
1049 62s 123ms/step - loss: 0.1454 - acc: 0.9852 - val_loss: 0.4126 - val_acc: 0.9164
1050 Epoch 523/1000
1051 62s 123ms/step - loss: 0.1435 - acc: 0.9866 - val_loss: 0.4130 - val_acc: 0.9201
1052 Epoch 524/1000
1053 62s 124ms/step - loss: 0.1478 - acc: 0.9848 - val_loss: 0.4095 - val_acc: 0.9172
1054 Epoch 525/1000
1055 62s 125ms/step - loss: 0.1511 - acc: 0.9837 - val_loss: 0.4036 - val_acc: 0.9179
1056 Epoch 526/1000
1057 62s 125ms/step - loss: 0.1491 - acc: 0.9842 - val_loss: 0.4036 - val_acc: 0.9173
1058 Epoch 527/1000
1059 62s 125ms/step - loss: 0.1433 - acc: 0.9865 - val_loss: 0.4225 - val_acc: 0.9123
1060 Epoch 528/1000
1061 62s 125ms/step - loss: 0.1453 - acc: 0.9854 - val_loss: 0.4156 - val_acc: 0.9156
1062 Epoch 529/1000
1063 62s 125ms/step - loss: 0.1462 - acc: 0.9856 - val_loss: 0.4154 - val_acc: 0.9143
1064 Epoch 530/1000
1065 62s 125ms/step - loss: 0.1500 - acc: 0.9840 - val_loss: 0.4176 - val_acc: 0.9120
1066 Epoch 531/1000
1067 62s 125ms/step - loss: 0.1489 - acc: 0.9845 - val_loss: 0.4087 - val_acc: 0.9158
1068 Epoch 532/1000
1069 62s 125ms/step - loss: 0.1480 - acc: 0.9849 - val_loss: 0.4042 - val_acc: 0.9154
1070 Epoch 533/1000
1071 62s 124ms/step - loss: 0.1483 - acc: 0.9850 - val_loss: 0.4191 - val_acc: 0.9155
1072 Epoch 534/1000
1073 62s 125ms/step - loss: 0.1432 - acc: 0.9865 - val_loss: 0.4054 - val_acc: 0.9173
1074 Epoch 535/1000
1075 62s 125ms/step - loss: 0.1471 - acc: 0.9854 - val_loss: 0.4200 - val_acc: 0.9133
1076 Epoch 536/1000
1077 62s 125ms/step - loss: 0.1518 - acc: 0.9835 - val_loss: 0.4052 - val_acc: 0.9160
1078 Epoch 537/1000
1079 62s 125ms/step - loss: 0.1442 - acc: 0.9860 - val_loss: 0.4331 - val_acc: 0.9126
1080 Epoch 538/1000
1081 62s 124ms/step - loss: 0.1466 - acc: 0.9848 - val_loss: 0.4207 - val_acc: 0.9173
1082 Epoch 539/1000
1083 62s 125ms/step - loss: 0.1469 - acc: 0.9851 - val_loss: 0.4202 - val_acc: 0.9109
1084 Epoch 540/1000
1085 62s 124ms/step - loss: 0.1496 - acc: 0.9838 - val_loss: 0.4297 - val_acc: 0.9126
1086 Epoch 541/1000
1087 62s 124ms/step - loss: 0.1453 - acc: 0.9863 - val_loss: 0.4219 - val_acc: 0.9139
1088 Epoch 542/1000
1089 62s 125ms/step - loss: 0.1469 - acc: 0.9854 - val_loss: 0.4203 - val_acc: 0.9137
1090 Epoch 543/1000
1091 62s 124ms/step - loss: 0.1471 - acc: 0.9852 - val_loss: 0.4216 - val_acc: 0.9152
1092 Epoch 544/1000
1093 62s 125ms/step - loss: 0.1514 - acc: 0.9840 - val_loss: 0.4178 - val_acc: 0.9138
1094 Epoch 545/1000
1095 62s 125ms/step - loss: 0.1459 - acc: 0.9853 - val_loss: 0.4266 - val_acc: 0.9126
1096 Epoch 546/1000
1097 62s 125ms/step - loss: 0.1480 - acc: 0.9845 - val_loss: 0.4147 - val_acc: 0.9176
1098 Epoch 547/1000
1099 62s 125ms/step - loss: 0.1464 - acc: 0.9856 - val_loss: 0.4306 - val_acc: 0.9130
1100 Epoch 548/1000
1101 62s 124ms/step - loss: 0.1490 - acc: 0.9840 - val_loss: 0.4259 - val_acc: 0.9143
1102 Epoch 549/1000
1103 62s 124ms/step - loss: 0.1485 - acc: 0.9847 - val_loss: 0.4349 - val_acc: 0.9109
1104 Epoch 550/1000
1105 62s 124ms/step - loss: 0.1487 - acc: 0.9844 - val_loss: 0.3984 - val_acc: 0.9189
1106 Epoch 551/1000
1107 62s 125ms/step - loss: 0.1489 - acc: 0.9850 - val_loss: 0.4091 - val_acc: 0.9185
1108 Epoch 552/1000
1109 62s 124ms/step - loss: 0.1486 - acc: 0.9845 - val_loss: 0.4143 - val_acc: 0.9138
1110 Epoch 553/1000
1111 62s 125ms/step - loss: 0.1453 - acc: 0.9859 - val_loss: 0.4008 - val_acc: 0.9154
1112 Epoch 554/1000
1113 62s 124ms/step - loss: 0.1445 - acc: 0.9864 - val_loss: 0.4308 - val_acc: 0.9134
1114 Epoch 555/1000
1115 62s 125ms/step - loss: 0.1484 - acc: 0.9844 - val_loss: 0.4206 - val_acc: 0.9143
1116 Epoch 556/1000
1117 62s 124ms/step - loss: 0.1455 - acc: 0.9856 - val_loss: 0.3994 - val_acc: 0.9197
1118 Epoch 557/1000
1119 62s 124ms/step - loss: 0.1446 - acc: 0.9860 - val_loss: 0.3963 - val_acc: 0.9194
1120 Epoch 558/1000
1121 62s 124ms/step - loss: 0.1460 - acc: 0.9852 - val_loss: 0.4125 - val_acc: 0.9140
1122 Epoch 559/1000
1123 62s 125ms/step - loss: 0.1446 - acc: 0.9859 - val_loss: 0.4092 - val_acc: 0.9165
1124 Epoch 560/1000
1125 62s 124ms/step - loss: 0.1442 - acc: 0.9865 - val_loss: 0.3911 - val_acc: 0.9212
1126 Epoch 561/1000
1127 62s 124ms/step - loss: 0.1459 - acc: 0.9852 - val_loss: 0.3984 - val_acc: 0.9185
1128 Epoch 562/1000
1129 62s 124ms/step - loss: 0.1473 - acc: 0.9851 - val_loss: 0.4080 - val_acc: 0.9196
1130 Epoch 563/1000
1131 62s 124ms/step - loss: 0.1465 - acc: 0.9860 - val_loss: 0.4058 - val_acc: 0.9166
1132 Epoch 564/1000
1133 62s 124ms/step - loss: 0.1423 - acc: 0.9870 - val_loss: 0.4046 - val_acc: 0.9180
1134 Epoch 565/1000
1135 62s 124ms/step - loss: 0.1486 - acc: 0.9851 - val_loss: 0.4022 - val_acc: 0.9184
1136 Epoch 566/1000
1137 62s 124ms/step - loss: 0.1478 - acc: 0.9853 - val_loss: 0.3896 - val_acc: 0.9224
1138 Epoch 567/1000
1139 62s 124ms/step - loss: 0.1470 - acc: 0.9850 - val_loss: 0.4141 - val_acc: 0.9151
1140 Epoch 568/1000
1141 62s 124ms/step - loss: 0.1438 - acc: 0.9862 - val_loss: 0.4139 - val_acc: 0.9197
1142 Epoch 569/1000
1143 62s 125ms/step - loss: 0.1470 - acc: 0.9851 - val_loss: 0.4143 - val_acc: 0.9156
1144 Epoch 570/1000
1145 62s 125ms/step - loss: 0.1484 - acc: 0.9845 - val_loss: 0.4151 - val_acc: 0.9148
1146 Epoch 571/1000
1147 62s 125ms/step - loss: 0.1479 - acc: 0.9849 - val_loss: 0.4206 - val_acc: 0.9136
1148 Epoch 572/1000
1149 62s 124ms/step - loss: 0.1458 - acc: 0.9855 - val_loss: 0.4172 - val_acc: 0.9147
1150 Epoch 573/1000
1151 62s 124ms/step - loss: 0.1450 - acc: 0.9860 - val_loss: 0.4267 - val_acc: 0.9156
1152 Epoch 574/1000
1153 62s 124ms/step - loss: 0.1514 - acc: 0.9834 - val_loss: 0.4357 - val_acc: 0.9127
1154 Epoch 575/1000
1155 62s 124ms/step - loss: 0.1475 - acc: 0.9851 - val_loss: 0.4212 - val_acc: 0.9142
1156 Epoch 576/1000
1157 62s 125ms/step - loss: 0.1464 - acc: 0.9858 - val_loss: 0.4141 - val_acc: 0.9162
1158 Epoch 577/1000
1159 62s 125ms/step - loss: 0.1478 - acc: 0.9846 - val_loss: 0.4065 - val_acc: 0.9151
1160 Epoch 578/1000
1161 62s 125ms/step - loss: 0.1418 - acc: 0.9868 - val_loss: 0.4090 - val_acc: 0.9145
1162 Epoch 579/1000
1163 62s 125ms/step - loss: 0.1456 - acc: 0.9852 - val_loss: 0.4350 - val_acc: 0.9101
1164 Epoch 580/1000
1165 62s 125ms/step - loss: 0.1422 - acc: 0.9870 - val_loss: 0.4116 - val_acc: 0.9185
1166 Epoch 581/1000
1167 62s 125ms/step - loss: 0.1449 - acc: 0.9858 - val_loss: 0.4245 - val_acc: 0.9127
1168 Epoch 582/1000
1169 62s 125ms/step - loss: 0.1429 - acc: 0.9863 - val_loss: 0.4157 - val_acc: 0.9163
1170 Epoch 583/1000
1171 62s 125ms/step - loss: 0.1473 - acc: 0.9851 - val_loss: 0.4094 - val_acc: 0.9165
1172 Epoch 584/1000
1173 62s 125ms/step - loss: 0.1500 - acc: 0.9845 - val_loss: 0.4269 - val_acc: 0.9115
1174 Epoch 585/1000
1175 62s 125ms/step - loss: 0.1450 - acc: 0.9860 - val_loss: 0.4189 - val_acc: 0.9165
1176 Epoch 586/1000
1177 62s 125ms/step - loss: 0.1450 - acc: 0.9859 - val_loss: 0.4153 - val_acc: 0.9153
1178 Epoch 587/1000
1179 62s 125ms/step - loss: 0.1453 - acc: 0.9859 - val_loss: 0.4166 - val_acc: 0.9155
1180 Epoch 588/1000
1181 62s 125ms/step - loss: 0.1409 - acc: 0.9875 - val_loss: 0.4088 - val_acc: 0.9193
1182 Epoch 589/1000
1183 62s 125ms/step - loss: 0.1455 - acc: 0.9854 - val_loss: 0.4220 - val_acc: 0.9149
1184 Epoch 590/1000
1185 62s 125ms/step - loss: 0.1466 - acc: 0.9848 - val_loss: 0.4264 - val_acc: 0.9136
1186 Epoch 591/1000
1187 62s 125ms/step - loss: 0.1424 - acc: 0.9868 - val_loss: 0.4212 - val_acc: 0.9178
1188 Epoch 592/1000
1189 62s 125ms/step - loss: 0.1441 - acc: 0.9862 - val_loss: 0.4271 - val_acc: 0.9127
1190 Epoch 593/1000
1191 62s 124ms/step - loss: 0.1469 - acc: 0.9852 - val_loss: 0.4247 - val_acc: 0.9170
1192 Epoch 594/1000
1193 62s 125ms/step - loss: 0.1468 - acc: 0.9845 - val_loss: 0.4080 - val_acc: 0.9192
1194 Epoch 595/1000
1195 62s 125ms/step - loss: 0.1437 - acc: 0.9857 - val_loss: 0.4111 - val_acc: 0.9174
1196 Epoch 596/1000
1197 62s 125ms/step - loss: 0.1451 - acc: 0.9852 - val_loss: 0.4290 - val_acc: 0.9124
1198 Epoch 597/1000
1199 62s 124ms/step - loss: 0.1465 - acc: 0.9856 - val_loss: 0.4203 - val_acc: 0.9167
1200 Epoch 598/1000
1201 62s 125ms/step - loss: 0.1451 - acc: 0.9855 - val_loss: 0.4203 - val_acc: 0.9136
1202 Epoch 599/1000
1203 62s 125ms/step - loss: 0.1460 - acc: 0.9857 - val_loss: 0.4248 - val_acc: 0.9161
1204 Epoch 600/1000
1205 62s 124ms/step - loss: 0.1466 - acc: 0.9856 - val_loss: 0.4286 - val_acc: 0.9143
1206 Epoch 601/1000
1207 lr changed to 0.0009999999776482583
1208 62s 125ms/step - loss: 0.1318 - acc: 0.9907 - val_loss: 0.3912 - val_acc: 0.9255
1209 Epoch 602/1000
1210 62s 124ms/step - loss: 0.1212 - acc: 0.9945 - val_loss: 0.3822 - val_acc: 0.9269
1211 Epoch 603/1000
1212 62s 125ms/step - loss: 0.1176 - acc: 0.9960 - val_loss: 0.3786 - val_acc: 0.9289
1213 Epoch 604/1000
1214 62s 125ms/step - loss: 0.1168 - acc: 0.9959 - val_loss: 0.3779 - val_acc: 0.9286
1215 Epoch 605/1000
1216 62s 125ms/step - loss: 0.1146 - acc: 0.9965 - val_loss: 0.3782 - val_acc: 0.9295
1217 Epoch 606/1000
1218 62s 125ms/step - loss: 0.1130 - acc: 0.9973 - val_loss: 0.3791 - val_acc: 0.9294
1219 Epoch 607/1000
1220 62s 125ms/step - loss: 0.1127 - acc: 0.9974 - val_loss: 0.3780 - val_acc: 0.9301
1221 Epoch 608/1000
1222 62s 125ms/step - loss: 0.1118 - acc: 0.9976 - val_loss: 0.3777 - val_acc: 0.9300
1223 Epoch 609/1000
1224 62s 125ms/step - loss: 0.1112 - acc: 0.9975 - val_loss: 0.3760 - val_acc: 0.9298
1225 Epoch 610/1000
1226 62s 125ms/step - loss: 0.1102 - acc: 0.9978 - val_loss: 0.3769 - val_acc: 0.9301
1227 Epoch 611/1000
1228 62s 125ms/step - loss: 0.1106 - acc: 0.9977 - val_loss: 0.3775 - val_acc: 0.9309
1229 Epoch 612/1000
1230 62s 125ms/step - loss: 0.1092 - acc: 0.9979 - val_loss: 0.3781 - val_acc: 0.9304
1231 Epoch 613/1000
1232 62s 124ms/step - loss: 0.1096 - acc: 0.9979 - val_loss: 0.3768 - val_acc: 0.9297
1233 Epoch 614/1000
1234 62s 125ms/step - loss: 0.1092 - acc: 0.9979 - val_loss: 0.3770 - val_acc: 0.9302
1235 Epoch 615/1000
1236 62s 125ms/step - loss: 0.1084 - acc: 0.9982 - val_loss: 0.3779 - val_acc: 0.9309
1237 Epoch 616/1000
1238 62s 125ms/step - loss: 0.1077 - acc: 0.9983 - val_loss: 0.3804 - val_acc: 0.9299
1239 Epoch 617/1000
1240 62s 125ms/step - loss: 0.1073 - acc: 0.9983 - val_loss: 0.3799 - val_acc: 0.9302
1241 Epoch 618/1000
1242 62s 125ms/step - loss: 0.1069 - acc: 0.9985 - val_loss: 0.3816 - val_acc: 0.9305
1243 Epoch 619/1000
1244 62s 125ms/step - loss: 0.1063 - acc: 0.9985 - val_loss: 0.3814 - val_acc: 0.9303
1245 Epoch 620/1000
1246 62s 125ms/step - loss: 0.1066 - acc: 0.9983 - val_loss: 0.3817 - val_acc: 0.9301
1247 Epoch 621/1000
1248 62s 125ms/step - loss: 0.1060 - acc: 0.9987 - val_loss: 0.3811 - val_acc: 0.9303
1249 Epoch 622/1000
1250 62s 124ms/step - loss: 0.1058 - acc: 0.9985 - val_loss: 0.3815 - val_acc: 0.9298
1251 Epoch 623/1000
1252 62s 124ms/step - loss: 0.1051 - acc: 0.9986 - val_loss: 0.3810 - val_acc: 0.9302
1253 Epoch 624/1000
1254 62s 124ms/step - loss: 0.1050 - acc: 0.9986 - val_loss: 0.3825 - val_acc: 0.9303
1255 Epoch 625/1000
1256 62s 124ms/step - loss: 0.1048 - acc: 0.9987 - val_loss: 0.3845 - val_acc: 0.9294
1257 Epoch 626/1000
1258 62s 125ms/step - loss: 0.1040 - acc: 0.9988 - val_loss: 0.3842 - val_acc: 0.9296
1259 Epoch 627/1000
1260 62s 125ms/step - loss: 0.1037 - acc: 0.9988 - val_loss: 0.3833 - val_acc: 0.9304
1261 Epoch 628/1000
1262 62s 124ms/step - loss: 0.1048 - acc: 0.9982 - val_loss: 0.3844 - val_acc: 0.9303
1263 Epoch 629/1000
1264 62s 124ms/step - loss: 0.1045 - acc: 0.9984 - val_loss: 0.3829 - val_acc: 0.9289
1265 Epoch 630/1000
1266 62s 125ms/step - loss: 0.1032 - acc: 0.9988 - val_loss: 0.3823 - val_acc: 0.9302
1267 Epoch 631/1000
1268 62s 125ms/step - loss: 0.1034 - acc: 0.9987 - val_loss: 0.3809 - val_acc: 0.9314
1269 Epoch 632/1000
1270 62s 125ms/step - loss: 0.1029 - acc: 0.9987 - val_loss: 0.3812 - val_acc: 0.9309
1271 Epoch 633/1000
1272 62s 125ms/step - loss: 0.1023 - acc: 0.9990 - val_loss: 0.3815 - val_acc: 0.9303
1273 Epoch 634/1000
1274 62s 124ms/step - loss: 0.1025 - acc: 0.9987 - val_loss: 0.3854 - val_acc: 0.9303
1275 Epoch 635/1000
1276 62s 124ms/step - loss: 0.1022 - acc: 0.9988 - val_loss: 0.3849 - val_acc: 0.9305
1277 Epoch 636/1000
1278 62s 124ms/step - loss: 0.1015 - acc: 0.9989 - val_loss: 0.3840 - val_acc: 0.9312
1279 Epoch 637/1000
1280 62s 124ms/step - loss: 0.1012 - acc: 0.9991 - val_loss: 0.3831 - val_acc: 0.9308
1281 Epoch 638/1000
1282 62s 125ms/step - loss: 0.1012 - acc: 0.9990 - val_loss: 0.3830 - val_acc: 0.9315
1283 Epoch 639/1000
1284 62s 124ms/step - loss: 0.1012 - acc: 0.9989 - val_loss: 0.3826 - val_acc: 0.9309
1285 Epoch 640/1000
1286 62s 125ms/step - loss: 0.1006 - acc: 0.9990 - val_loss: 0.3838 - val_acc: 0.9309
1287 Epoch 641/1000
1288 62s 125ms/step - loss: 0.1004 - acc: 0.9989 - val_loss: 0.3843 - val_acc: 0.9315
1289 Epoch 642/1000
1290 62s 125ms/step - loss: 0.0998 - acc: 0.9992 - val_loss: 0.3852 - val_acc: 0.9311
1291 Epoch 643/1000
1292 62s 124ms/step - loss: 0.1001 - acc: 0.9991 - val_loss: 0.3846 - val_acc: 0.9309
1293 Epoch 644/1000
1294 62s 124ms/step - loss: 0.1000 - acc: 0.9990 - val_loss: 0.3847 - val_acc: 0.9304
1295 Epoch 645/1000
1296 62s 124ms/step - loss: 0.0993 - acc: 0.9992 - val_loss: 0.3843 - val_acc: 0.9314
1297 Epoch 646/1000
1298 62s 125ms/step - loss: 0.0993 - acc: 0.9991 - val_loss: 0.3823 - val_acc: 0.9313
1299 Epoch 647/1000
1300 62s 125ms/step - loss: 0.0989 - acc: 0.9992 - val_loss: 0.3840 - val_acc: 0.9305
1301 Epoch 648/1000
1302 62s 125ms/step - loss: 0.0991 - acc: 0.9990 - val_loss: 0.3843 - val_acc: 0.9313
1303 Epoch 649/1000
1304 62s 125ms/step - loss: 0.0984 - acc: 0.9992 - val_loss: 0.3854 - val_acc: 0.9315
1305 Epoch 650/1000
1306 62s 125ms/step - loss: 0.0982 - acc: 0.9991 - val_loss: 0.3853 - val_acc: 0.9322
1307 Epoch 651/1000
1308 62s 125ms/step - loss: 0.0981 - acc: 0.9993 - val_loss: 0.3860 - val_acc: 0.9312
1309 Epoch 652/1000
1310 62s 124ms/step - loss: 0.0982 - acc: 0.9990 - val_loss: 0.3880 - val_acc: 0.9307
1311 Epoch 653/1000
1312 62s 125ms/step - loss: 0.0980 - acc: 0.9990 - val_loss: 0.3868 - val_acc: 0.9314
1313 Epoch 654/1000
1314 62s 125ms/step - loss: 0.0973 - acc: 0.9993 - val_loss: 0.3846 - val_acc: 0.9319
1315 Epoch 655/1000
1316 62s 125ms/step - loss: 0.0971 - acc: 0.9994 - val_loss: 0.3826 - val_acc: 0.9309
1317 Epoch 656/1000
1318 62s 125ms/step - loss: 0.0970 - acc: 0.9990 - val_loss: 0.3831 - val_acc: 0.9319
1319 Epoch 657/1000
1320 62s 125ms/step - loss: 0.0970 - acc: 0.9991 - val_loss: 0.3833 - val_acc: 0.9310
1321 Epoch 658/1000
1322 62s 124ms/step - loss: 0.0966 - acc: 0.9992 - val_loss: 0.3810 - val_acc: 0.9314
1323 Epoch 659/1000
1324 62s 125ms/step - loss: 0.0967 - acc: 0.9990 - val_loss: 0.3795 - val_acc: 0.9330
1325 Epoch 660/1000
1326 62s 125ms/step - loss: 0.0962 - acc: 0.9992 - val_loss: 0.3816 - val_acc: 0.9329
1327 Epoch 661/1000
1328 62s 124ms/step - loss: 0.0964 - acc: 0.9991 - val_loss: 0.3839 - val_acc: 0.9321
1329 Epoch 662/1000
1330 62s 125ms/step - loss: 0.0956 - acc: 0.9992 - val_loss: 0.3841 - val_acc: 0.9321
1331 Epoch 663/1000
1332 62s 125ms/step - loss: 0.0953 - acc: 0.9993 - val_loss: 0.3820 - val_acc: 0.9320
1333 Epoch 664/1000
1334 62s 125ms/step - loss: 0.0949 - acc: 0.9994 - val_loss: 0.3808 - val_acc: 0.9326
1335 Epoch 665/1000
1336 63s 125ms/step - loss: 0.0957 - acc: 0.9990 - val_loss: 0.3808 - val_acc: 0.9332
1337 Epoch 666/1000
1338 62s 125ms/step - loss: 0.0955 - acc: 0.9989 - val_loss: 0.3811 - val_acc: 0.9320
1339 Epoch 667/1000
1340 62s 125ms/step - loss: 0.0948 - acc: 0.9993 - val_loss: 0.3811 - val_acc: 0.9330
1341 Epoch 668/1000
1342 62s 124ms/step - loss: 0.0949 - acc: 0.9991 - val_loss: 0.3822 - val_acc: 0.9320
1343 Epoch 669/1000
1344 62s 125ms/step - loss: 0.0947 - acc: 0.9992 - val_loss: 0.3843 - val_acc: 0.9321
1345 Epoch 670/1000
1346 62s 124ms/step - loss: 0.0943 - acc: 0.9992 - val_loss: 0.3810 - val_acc: 0.9314
1347 Epoch 671/1000
1348 62s 125ms/step - loss: 0.0941 - acc: 0.9991 - val_loss: 0.3823 - val_acc: 0.9324
1349 Epoch 672/1000
1350 62s 125ms/step - loss: 0.0937 - acc: 0.9993 - val_loss: 0.3826 - val_acc: 0.9328
1351 Epoch 673/1000
1352 62s 125ms/step - loss: 0.0936 - acc: 0.9992 - val_loss: 0.3816 - val_acc: 0.9330
1353 Epoch 674/1000
1354 62s 125ms/step - loss: 0.0935 - acc: 0.9991 - val_loss: 0.3806 - val_acc: 0.9328
1355 Epoch 675/1000
1356 62s 125ms/step - loss: 0.0931 - acc: 0.9992 - val_loss: 0.3828 - val_acc: 0.9331
1357 Epoch 676/1000
1358 62s 124ms/step - loss: 0.0935 - acc: 0.9989 - val_loss: 0.3859 - val_acc: 0.9327
1359 Epoch 677/1000
1360 62s 124ms/step - loss: 0.0927 - acc: 0.9993 - val_loss: 0.3836 - val_acc: 0.9329
1361 Epoch 678/1000
1362 62s 125ms/step - loss: 0.0925 - acc: 0.9994 - val_loss: 0.3829 - val_acc: 0.9325
1363 Epoch 679/1000
1364 62s 125ms/step - loss: 0.0924 - acc: 0.9993 - val_loss: 0.3823 - val_acc: 0.9341
1365 Epoch 680/1000
1366 62s 125ms/step - loss: 0.0920 - acc: 0.9993 - val_loss: 0.3843 - val_acc: 0.9326
1367 Epoch 681/1000
1368 62s 125ms/step - loss: 0.0922 - acc: 0.9993 - val_loss: 0.3855 - val_acc: 0.9315
1369 Epoch 682/1000
1370 62s 125ms/step - loss: 0.0918 - acc: 0.9993 - val_loss: 0.3850 - val_acc: 0.9314
1371 Epoch 683/1000
1372 62s 125ms/step - loss: 0.0915 - acc: 0.9994 - val_loss: 0.3850 - val_acc: 0.9312
1373 Epoch 684/1000
1374 62s 125ms/step - loss: 0.0911 - acc: 0.9995 - val_loss: 0.3848 - val_acc: 0.9313
1375 Epoch 685/1000
1376 62s 125ms/step - loss: 0.0908 - acc: 0.9996 - val_loss: 0.3854 - val_acc: 0.9322
1377 Epoch 686/1000
1378 62s 125ms/step - loss: 0.0912 - acc: 0.9992 - val_loss: 0.3836 - val_acc: 0.9325
1379 Epoch 687/1000
1380 62s 125ms/step - loss: 0.0909 - acc: 0.9993 - val_loss: 0.3848 - val_acc: 0.9320
1381 Epoch 688/1000
1382 63s 125ms/step - loss: 0.0904 - acc: 0.9995 - val_loss: 0.3857 - val_acc: 0.9316
1383 Epoch 689/1000
1384 62s 125ms/step - loss: 0.0904 - acc: 0.9994 - val_loss: 0.3858 - val_acc: 0.9318
1385 Epoch 690/1000
1386 62s 125ms/step - loss: 0.0901 - acc: 0.9995 - val_loss: 0.3829 - val_acc: 0.9320
1387 Epoch 691/1000
1388 62s 124ms/step - loss: 0.0906 - acc: 0.9993 - val_loss: 0.3816 - val_acc: 0.9324
1389 Epoch 692/1000
1390 62s 124ms/step - loss: 0.0900 - acc: 0.9992 - val_loss: 0.3827 - val_acc: 0.9308
1391 Epoch 693/1000
1392 62s 124ms/step - loss: 0.0898 - acc: 0.9994 - val_loss: 0.3814 - val_acc: 0.9324
1393 Epoch 694/1000
1394 62s 124ms/step - loss: 0.0898 - acc: 0.9994 - val_loss: 0.3820 - val_acc: 0.9329
1395 Epoch 695/1000
1396 63s 125ms/step - loss: 0.0892 - acc: 0.9995 - val_loss: 0.3823 - val_acc: 0.9335
1397 Epoch 696/1000
1398 62s 125ms/step - loss: 0.0893 - acc: 0.9992 - val_loss: 0.3825 - val_acc: 0.9326
1399 Epoch 697/1000
1400 62s 123ms/step - loss: 0.0891 - acc: 0.9994 - val_loss: 0.3827 - val_acc: 0.9324
1401 Epoch 698/1000
1402 62s 123ms/step - loss: 0.0892 - acc: 0.9992 - val_loss: 0.3812 - val_acc: 0.9331
1403 Epoch 699/1000
1404 62s 124ms/step - loss: 0.0888 - acc: 0.9992 - val_loss: 0.3822 - val_acc: 0.9310
1405 Epoch 700/1000
1406 62s 125ms/step - loss: 0.0883 - acc: 0.9995 - val_loss: 0.3825 - val_acc: 0.9317
1407 Epoch 755/1000
1408 62s 124ms/step - loss: 0.0792 - acc: 0.9996 - val_loss: 0.3819 - val_acc: 0.9316
1409 Epoch 756/1000
1410 62s 123ms/step - loss: 0.0791 - acc: 0.9996 - val_loss: 0.3808 - val_acc: 0.9312
1411 Epoch 757/1000
1412 62s 123ms/step - loss: 0.0787 - acc: 0.9996 - val_loss: 0.3823 - val_acc: 0.9318
1413 Epoch 758/1000
1414 62s 124ms/step - loss: 0.0788 - acc: 0.9995 - val_loss: 0.3813 - val_acc: 0.9322
1415 Epoch 759/1000
1416 62s 124ms/step - loss: 0.0783 - acc: 0.9997 - val_loss: 0.3816 - val_acc: 0.9330
1417 Epoch 760/1000
1418 62s 124ms/step - loss: 0.0782 - acc: 0.9997 - val_loss: 0.3780 - val_acc: 0.9331
1419 Epoch 761/1000
1420 62s 124ms/step - loss: 0.0783 - acc: 0.9995 - val_loss: 0.3778 - val_acc: 0.9325
1421 Epoch 762/1000
1422 62s 124ms/step - loss: 0.0780 - acc: 0.9996 - val_loss: 0.3769 - val_acc: 0.9319
1423 Epoch 763/1000
1424 62s 124ms/step - loss: 0.0780 - acc: 0.9996 - val_loss: 0.3781 - val_acc: 0.9324
1425 Epoch 764/1000
1426 62s 124ms/step - loss: 0.0779 - acc: 0.9996 - val_loss: 0.3798 - val_acc: 0.9331
1427 Epoch 765/1000
1428 62s 123ms/step - loss: 0.0778 - acc: 0.9995 - val_loss: 0.3799 - val_acc: 0.9323
1429 Epoch 766/1000
1430 62s 124ms/step - loss: 0.0777 - acc: 0.9996 - val_loss: 0.3809 - val_acc: 0.9311
1431 Epoch 767/1000
1432 62s 124ms/step - loss: 0.0772 - acc: 0.9996 - val_loss: 0.3797 - val_acc: 0.9322
1433 Epoch 768/1000
1434 62s 124ms/step - loss: 0.0772 - acc: 0.9996 - val_loss: 0.3814 - val_acc: 0.9323
1435 Epoch 769/1000
1436 62s 124ms/step - loss: 0.0770 - acc: 0.9996 - val_loss: 0.3811 - val_acc: 0.9330
1437 Epoch 770/1000
1438 62s 124ms/step - loss: 0.0763 - acc: 0.9999 - val_loss: 0.3816 - val_acc: 0.9331
1439 Epoch 771/1000
1440 62s 124ms/step - loss: 0.0764 - acc: 0.9997 - val_loss: 0.3830 - val_acc: 0.9317
1441 Epoch 772/1000
1442 62s 124ms/step - loss: 0.0767 - acc: 0.9995 - val_loss: 0.3810 - val_acc: 0.9320
1443 Epoch 773/1000
1444 62s 123ms/step - loss: 0.0768 - acc: 0.9995 - val_loss: 0.3831 - val_acc: 0.9325
1445 Epoch 774/1000
1446 62s 123ms/step - loss: 0.0764 - acc: 0.9995 - val_loss: 0.3824 - val_acc: 0.9334
1447 Epoch 775/1000
1448 62s 124ms/step - loss: 0.0761 - acc: 0.9997 - val_loss: 0.3816 - val_acc: 0.9329
1449 Epoch 776/1000
1450 62s 124ms/step - loss: 0.0758 - acc: 0.9997 - val_loss: 0.3815 - val_acc: 0.9340
1451 Epoch 777/1000
1452 62s 124ms/step - loss: 0.0756 - acc: 0.9998 - val_loss: 0.3810 - val_acc: 0.9333
1453 Epoch 778/1000
1454 62s 124ms/step - loss: 0.0756 - acc: 0.9996 - val_loss: 0.3838 - val_acc: 0.9322
1455 Epoch 779/1000
1456 62s 124ms/step - loss: 0.0755 - acc: 0.9996 - val_loss: 0.3831 - val_acc: 0.9329
1457 Epoch 780/1000
1458 62s 124ms/step - loss: 0.0756 - acc: 0.9994 - val_loss: 0.3833 - val_acc: 0.9313
1459 Epoch 781/1000
1460 62s 124ms/step - loss: 0.0756 - acc: 0.9995 - val_loss: 0.3831 - val_acc: 0.9325
1461 Epoch 782/1000
1462 62s 124ms/step - loss: 0.0751 - acc: 0.9996 - val_loss: 0.3820 - val_acc: 0.9340
1463 Epoch 783/1000
1464 62s 123ms/step - loss: 0.0749 - acc: 0.9996 - val_loss: 0.3818 - val_acc: 0.9328
1465 Epoch 784/1000
1466 62s 124ms/step - loss: 0.0749 - acc: 0.9995 - val_loss: 0.3788 - val_acc: 0.9331
1467 Epoch 785/1000
1468 62s 124ms/step - loss: 0.0747 - acc: 0.9995 - val_loss: 0.3800 - val_acc: 0.9335
1469 Epoch 786/1000
1470 62s 124ms/step - loss: 0.0745 - acc: 0.9996 - val_loss: 0.3790 - val_acc: 0.9333
1471 Epoch 787/1000
1472 62s 123ms/step - loss: 0.0745 - acc: 0.9995 - val_loss: 0.3798 - val_acc: 0.9348
1473 Epoch 788/1000
1474 62s 123ms/step - loss: 0.0746 - acc: 0.9995 - val_loss: 0.3808 - val_acc: 0.9340
1475 Epoch 789/1000
1476 62s 124ms/step - loss: 0.0741 - acc: 0.9996 - val_loss: 0.3784 - val_acc: 0.9347
1477 Epoch 790/1000
1478 62s 124ms/step - loss: 0.0741 - acc: 0.9996 - val_loss: 0.3764 - val_acc: 0.9334
1479 Epoch 791/1000
1480 62s 124ms/step - loss: 0.0739 - acc: 0.9996 - val_loss: 0.3747 - val_acc: 0.9340
1481 Epoch 792/1000
1482 62s 124ms/step - loss: 0.0736 - acc: 0.9996 - val_loss: 0.3762 - val_acc: 0.9326
1483 Epoch 793/1000
1484 62s 124ms/step - loss: 0.0737 - acc: 0.9996 - val_loss: 0.3747 - val_acc: 0.9341
1485 Epoch 794/1000
1486 62s 124ms/step - loss: 0.0736 - acc: 0.9996 - val_loss: 0.3778 - val_acc: 0.9322
1487 Epoch 795/1000
1488 62s 123ms/step - loss: 0.0732 - acc: 0.9997 - val_loss: 0.3804 - val_acc: 0.9324
1489 Epoch 796/1000
1490 62s 123ms/step - loss: 0.0732 - acc: 0.9996 - val_loss: 0.3771 - val_acc: 0.9336
1491 Epoch 797/1000
1492 62s 124ms/step - loss: 0.0733 - acc: 0.9994 - val_loss: 0.3761 - val_acc: 0.9325
1493 Epoch 798/1000
1494 62s 123ms/step - loss: 0.0729 - acc: 0.9996 - val_loss: 0.3770 - val_acc: 0.9323
1495 Epoch 799/1000
1496 62s 124ms/step - loss: 0.0727 - acc: 0.9996 - val_loss: 0.3768 - val_acc: 0.9324
1497 Epoch 800/1000
1498 62s 124ms/step - loss: 0.0725 - acc: 0.9997 - val_loss: 0.3784 - val_acc: 0.9329
1499 Epoch 801/1000
1500 62s 123ms/step - loss: 0.0723 - acc: 0.9997 - val_loss: 0.3741 - val_acc: 0.9342
1501 Epoch 802/1000
1502 62s 124ms/step - loss: 0.0723 - acc: 0.9996 - val_loss: 0.3772 - val_acc: 0.9332
1503 Epoch 803/1000
1504 62s 123ms/step - loss: 0.0721 - acc: 0.9996 - val_loss: 0.3778 - val_acc: 0.9329
1505 Epoch 804/1000
1506 62s 123ms/step - loss: 0.0722 - acc: 0.9995 - val_loss: 0.3759 - val_acc: 0.9337
1507 Epoch 805/1000
1508 62s 123ms/step - loss: 0.0719 - acc: 0.9996 - val_loss: 0.3788 - val_acc: 0.9335
1509 Epoch 806/1000
1510 62s 123ms/step - loss: 0.0719 - acc: 0.9995 - val_loss: 0.3815 - val_acc: 0.9332
1511 Epoch 807/1000
1512 62s 123ms/step - loss: 0.0716 - acc: 0.9997 - val_loss: 0.3774 - val_acc: 0.9321
1513 Epoch 808/1000
1514 62s 124ms/step - loss: 0.0714 - acc: 0.9997 - val_loss: 0.3774 - val_acc: 0.9337
1515 Epoch 809/1000
1516 62s 124ms/step - loss: 0.0714 - acc: 0.9997 - val_loss: 0.3786 - val_acc: 0.9320
1517 Epoch 810/1000
1518 62s 124ms/step - loss: 0.0713 - acc: 0.9996 - val_loss: 0.3776 - val_acc: 0.9322
1519 Epoch 811/1000
1520 62s 123ms/step - loss: 0.0712 - acc: 0.9996 - val_loss: 0.3782 - val_acc: 0.9332
1521 Epoch 812/1000
1522 62s 123ms/step - loss: 0.0709 - acc: 0.9997 - val_loss: 0.3837 - val_acc: 0.9322
1523 Epoch 813/1000
1524 62s 123ms/step - loss: 0.0705 - acc: 0.9998 - val_loss: 0.3839 - val_acc: 0.9322
1525 Epoch 814/1000
1526 62s 123ms/step - loss: 0.0707 - acc: 0.9996 - val_loss: 0.3820 - val_acc: 0.9318
1527 Epoch 815/1000
1528 62s 123ms/step - loss: 0.0705 - acc: 0.9997 - val_loss: 0.3829 - val_acc: 0.9309
1529 Epoch 816/1000
1530 62s 124ms/step - loss: 0.0703 - acc: 0.9996 - val_loss: 0.3810 - val_acc: 0.9318
1531 Epoch 817/1000
1532 62s 124ms/step - loss: 0.0700 - acc: 0.9998 - val_loss: 0.3799 - val_acc: 0.9316
1533 Epoch 818/1000
1534 62s 124ms/step - loss: 0.0701 - acc: 0.9996 - val_loss: 0.3789 - val_acc: 0.9314
1535 Epoch 819/1000
1536 62s 124ms/step - loss: 0.0698 - acc: 0.9997 - val_loss: 0.3802 - val_acc: 0.9326
1537 Epoch 820/1000
1538 62s 123ms/step - loss: 0.0699 - acc: 0.9996 - val_loss: 0.3837 - val_acc: 0.9301
1539 Epoch 821/1000
1540 62s 123ms/step - loss: 0.0697 - acc: 0.9996 - val_loss: 0.3833 - val_acc: 0.9317
1541 Epoch 822/1000
1542 62s 124ms/step - loss: 0.0698 - acc: 0.9995 - val_loss: 0.3851 - val_acc: 0.9305
1543 Epoch 823/1000
1544 62s 124ms/step - loss: 0.0694 - acc: 0.9997 - val_loss: 0.3824 - val_acc: 0.9311
1545 Epoch 824/1000
1546 62s 124ms/step - loss: 0.0693 - acc: 0.9995 - val_loss: 0.3830 - val_acc: 0.9303
1547 Epoch 825/1000
1548 62s 124ms/step - loss: 0.0690 - acc: 0.9998 - val_loss: 0.3802 - val_acc: 0.9298
1549 Epoch 826/1000
1550 62s 124ms/step - loss: 0.0689 - acc: 0.9996 - val_loss: 0.3810 - val_acc: 0.9305
1551 Epoch 827/1000
1552 62s 124ms/step - loss: 0.0689 - acc: 0.9997 - val_loss: 0.3813 - val_acc: 0.9309
1553 Epoch 828/1000
1554 62s 124ms/step - loss: 0.0688 - acc: 0.9996 - val_loss: 0.3799 - val_acc: 0.9316
1555 Epoch 829/1000
1556 62s 123ms/step - loss: 0.0687 - acc: 0.9996 - val_loss: 0.3766 - val_acc: 0.9322
1557 Epoch 830/1000
1558 62s 124ms/step - loss: 0.0688 - acc: 0.9995 - val_loss: 0.3764 - val_acc: 0.9329
1559 Epoch 831/1000
1560 62s 124ms/step - loss: 0.0684 - acc: 0.9996 - val_loss: 0.3750 - val_acc: 0.9324
1561 Epoch 832/1000
1562 62s 124ms/step - loss: 0.0685 - acc: 0.9996 - val_loss: 0.3781 - val_acc: 0.9314
1563 Epoch 833/1000
1564 62s 124ms/step - loss: 0.0682 - acc: 0.9997 - val_loss: 0.3741 - val_acc: 0.9313
1565 Epoch 834/1000
1566 62s 124ms/step - loss: 0.0682 - acc: 0.9996 - val_loss: 0.3738 - val_acc: 0.9312
1567 Epoch 835/1000
1568 62s 124ms/step - loss: 0.0680 - acc: 0.9996 - val_loss: 0.3753 - val_acc: 0.9319
1569 Epoch 836/1000
1570 62s 124ms/step - loss: 0.0679 - acc: 0.9997 - val_loss: 0.3753 - val_acc: 0.9317
1571 Epoch 837/1000
1572 62s 123ms/step - loss: 0.0677 - acc: 0.9996 - val_loss: 0.3769 - val_acc: 0.9320
1573 Epoch 838/1000
1574 62s 123ms/step - loss: 0.0674 - acc: 0.9997 - val_loss: 0.3775 - val_acc: 0.9317
1575 Epoch 839/1000
1576 62s 124ms/step - loss: 0.0678 - acc: 0.9995 - val_loss: 0.3779 - val_acc: 0.9327
1577 Epoch 840/1000
1578 62s 124ms/step - loss: 0.0673 - acc: 0.9995 - val_loss: 0.3773 - val_acc: 0.9319
1579 Epoch 841/1000
1580 62s 124ms/step - loss: 0.0672 - acc: 0.9996 - val_loss: 0.3764 - val_acc: 0.9333
1581 Epoch 842/1000
1582 62s 124ms/step - loss: 0.0670 - acc: 0.9997 - val_loss: 0.3741 - val_acc: 0.9323
1583 Epoch 843/1000
1584 62s 124ms/step - loss: 0.0667 - acc: 0.9997 - val_loss: 0.3723 - val_acc: 0.9321
1585 Epoch 844/1000
1586 62s 124ms/step - loss: 0.0667 - acc: 0.9997 - val_loss: 0.3731 - val_acc: 0.9315
1587 Epoch 845/1000
1588 62s 123ms/step - loss: 0.0668 - acc: 0.9996 - val_loss: 0.3733 - val_acc: 0.9320
1589 Epoch 846/1000
1590 62s 124ms/step - loss: 0.0666 - acc: 0.9996 - val_loss: 0.3722 - val_acc: 0.9315
1591 Epoch 847/1000
1592 62s 123ms/step - loss: 0.0664 - acc: 0.9996 - val_loss: 0.3719 - val_acc: 0.9327
1593 Epoch 848/1000
1594 62s 124ms/step - loss: 0.0662 - acc: 0.9997 - val_loss: 0.3720 - val_acc: 0.9309
1595 Epoch 849/1000
1596 62s 124ms/step - loss: 0.0660 - acc: 0.9996 - val_loss: 0.3716 - val_acc: 0.9316
1597 Epoch 850/1000
1598 62s 124ms/step - loss: 0.0660 - acc: 0.9996 - val_loss: 0.3732 - val_acc: 0.9310
1599 Epoch 851/1000
1600 62s 123ms/step - loss: 0.0661 - acc: 0.9995 - val_loss: 0.3718 - val_acc: 0.9314
1601 Epoch 852/1000
1602 62s 123ms/step - loss: 0.0657 - acc: 0.9997 - val_loss: 0.3748 - val_acc: 0.9300
1603 Epoch 853/1000
1604 62s 124ms/step - loss: 0.0654 - acc: 0.9997 - val_loss: 0.3724 - val_acc: 0.9314
1605 Epoch 854/1000
1606 62s 124ms/step - loss: 0.0658 - acc: 0.9995 - val_loss: 0.3750 - val_acc: 0.9283
1607 Epoch 855/1000
1608 62s 124ms/step - loss: 0.0652 - acc: 0.9998 - val_loss: 0.3719 - val_acc: 0.9314
1609 Epoch 856/1000
1610 62s 124ms/step - loss: 0.0652 - acc: 0.9998 - val_loss: 0.3724 - val_acc: 0.9314
1611 Epoch 857/1000
1612 62s 123ms/step - loss: 0.0652 - acc: 0.9995 - val_loss: 0.3732 - val_acc: 0.9300
1613 Epoch 858/1000
1614 62s 124ms/step - loss: 0.0648 - acc: 0.9997 - val_loss: 0.3714 - val_acc: 0.9307
1615 Epoch 859/1000
1616 62s 123ms/step - loss: 0.0654 - acc: 0.9994 - val_loss: 0.3719 - val_acc: 0.9315
1617 Epoch 860/1000
1618 62s 124ms/step - loss: 0.0645 - acc: 0.9997 - val_loss: 0.3726 - val_acc: 0.9308
1619 Epoch 861/1000
1620 62s 123ms/step - loss: 0.0648 - acc: 0.9996 - val_loss: 0.3725 - val_acc: 0.9308
1621 Epoch 862/1000
1622 62s 123ms/step - loss: 0.0645 - acc: 0.9997 - val_loss: 0.3698 - val_acc: 0.9312
1623 Epoch 863/1000
1624 62s 124ms/step - loss: 0.0642 - acc: 0.9997 - val_loss: 0.3715 - val_acc: 0.9305
1625 Epoch 864/1000
1626 62s 124ms/step - loss: 0.0643 - acc: 0.9997 - val_loss: 0.3724 - val_acc: 0.9302
1627 Epoch 865/1000
1628 62s 124ms/step - loss: 0.0639 - acc: 0.9998 - val_loss: 0.3748 - val_acc: 0.9304
1629 Epoch 866/1000
1630 62s 124ms/step - loss: 0.0641 - acc: 0.9995 - val_loss: 0.3751 - val_acc: 0.9315
1631 Epoch 867/1000
1632 62s 124ms/step - loss: 0.0638 - acc: 0.9997 - val_loss: 0.3729 - val_acc: 0.9325
1633 Epoch 868/1000
1634 62s 123ms/step - loss: 0.0637 - acc: 0.9997 - val_loss: 0.3750 - val_acc: 0.9320
1635 Epoch 869/1000
1636 62s 124ms/step - loss: 0.0634 - acc: 0.9997 - val_loss: 0.3738 - val_acc: 0.9312
1637 Epoch 870/1000
1638 62s 123ms/step - loss: 0.0634 - acc: 0.9996 - val_loss: 0.3731 - val_acc: 0.9313
1639 Epoch 871/1000
1640 62s 124ms/step - loss: 0.0632 - acc: 0.9998 - val_loss: 0.3750 - val_acc: 0.9311
1641 Epoch 872/1000
1642 62s 124ms/step - loss: 0.0633 - acc: 0.9997 - val_loss: 0.3784 - val_acc: 0.9313
1643 Epoch 873/1000
1644 62s 124ms/step - loss: 0.0635 - acc: 0.9995 - val_loss: 0.3719 - val_acc: 0.9312
1645 Epoch 874/1000
1646 62s 123ms/step - loss: 0.0631 - acc: 0.9996 - val_loss: 0.3706 - val_acc: 0.9330
1647 Epoch 875/1000
1648 62s 124ms/step - loss: 0.0626 - acc: 0.9997 - val_loss: 0.3711 - val_acc: 0.9331
1649 Epoch 876/1000
1650 62s 123ms/step - loss: 0.0628 - acc: 0.9996 - val_loss: 0.3730 - val_acc: 0.9332
1651 Epoch 877/1000
1652 62s 123ms/step - loss: 0.0626 - acc: 0.9997 - val_loss: 0.3744 - val_acc: 0.9323
1653 Epoch 878/1000
1654 62s 124ms/step - loss: 0.0623 - acc: 0.9997 - val_loss: 0.3724 - val_acc: 0.9321
1655 Epoch 879/1000
1656 62s 124ms/step - loss: 0.0624 - acc: 0.9996 - val_loss: 0.3749 - val_acc: 0.9312
1657 Epoch 880/1000
1658 62s 124ms/step - loss: 0.0621 - acc: 0.9997 - val_loss: 0.3728 - val_acc: 0.9314
1659 Epoch 881/1000
1660 62s 123ms/step - loss: 0.0620 - acc: 0.9997 - val_loss: 0.3733 - val_acc: 0.9317
1661 Epoch 882/1000
1662 62s 124ms/step - loss: 0.0623 - acc: 0.9996 - val_loss: 0.3779 - val_acc: 0.9298
1663 Epoch 883/1000
1664 62s 124ms/step - loss: 0.0621 - acc: 0.9996 - val_loss: 0.3733 - val_acc: 0.9309
1665 Epoch 884/1000
1666 62s 123ms/step - loss: 0.0617 - acc: 0.9997 - val_loss: 0.3714 - val_acc: 0.9312
1667 Epoch 885/1000
1668 62s 124ms/step - loss: 0.0615 - acc: 0.9997 - val_loss: 0.3708 - val_acc: 0.9313
1669 Epoch 886/1000
1670 62s 124ms/step - loss: 0.0615 - acc: 0.9997 - val_loss: 0.3727 - val_acc: 0.9305
1671 Epoch 887/1000
1672 62s 123ms/step - loss: 0.0616 - acc: 0.9996 - val_loss: 0.3699 - val_acc: 0.9313
1673 Epoch 888/1000
1674 62s 124ms/step - loss: 0.0611 - acc: 0.9997 - val_loss: 0.3709 - val_acc: 0.9310
1675 Epoch 889/1000
1676 62s 124ms/step - loss: 0.0611 - acc: 0.9997 - val_loss: 0.3718 - val_acc: 0.9309
1677 Epoch 890/1000
1678 62s 124ms/step - loss: 0.0611 - acc: 0.9997 - val_loss: 0.3721 - val_acc: 0.9315
1679 Epoch 891/1000
1680 62s 124ms/step - loss: 0.0606 - acc: 0.9998 - val_loss: 0.3726 - val_acc: 0.9324
1681 Epoch 892/1000
1682 62s 123ms/step - loss: 0.0606 - acc: 0.9997 - val_loss: 0.3737 - val_acc: 0.9321
1683 Epoch 893/1000
1684 62s 123ms/step - loss: 0.0607 - acc: 0.9996 - val_loss: 0.3709 - val_acc: 0.9325
1685 Epoch 894/1000
1686 62s 123ms/step - loss: 0.0602 - acc: 0.9999 - val_loss: 0.3701 - val_acc: 0.9325
1687 Epoch 895/1000
1688 62s 124ms/step - loss: 0.0604 - acc: 0.9997 - val_loss: 0.3670 - val_acc: 0.9327
1689 Epoch 896/1000
1690 62s 124ms/step - loss: 0.0606 - acc: 0.9995 - val_loss: 0.3646 - val_acc: 0.9325
1691 Epoch 897/1000
1692 62s 124ms/step - loss: 0.0603 - acc: 0.9997 - val_loss: 0.3693 - val_acc: 0.9315
1693 Epoch 898/1000
1694 62s 123ms/step - loss: 0.0602 - acc: 0.9996 - val_loss: 0.3705 - val_acc: 0.9312
1695 Epoch 899/1000
1696 62s 124ms/step - loss: 0.0599 - acc: 0.9997 - val_loss: 0.3697 - val_acc: 0.9309
1697 Epoch 900/1000
1698 62s 123ms/step - loss: 0.0600 - acc: 0.9997 - val_loss: 0.3694 - val_acc: 0.9313
1699 Epoch 901/1000
1700 lr changed to 9.999999310821295e-05
1701 62s 123ms/step - loss: 0.0597 - acc: 0.9998 - val_loss: 0.3694 - val_acc: 0.9313
1702 Epoch 902/1000
1703 62s 124ms/step - loss: 0.0595 - acc: 0.9998 - val_loss: 0.3685 - val_acc: 0.9316
1704 Epoch 903/1000
1705 62s 124ms/step - loss: 0.0597 - acc: 0.9998 - val_loss: 0.3685 - val_acc: 0.9314
1706 Epoch 904/1000
1707 62s 124ms/step - loss: 0.0599 - acc: 0.9997 - val_loss: 0.3686 - val_acc: 0.9316
1708 Epoch 905/1000
1709 62s 124ms/step - loss: 0.0598 - acc: 0.9997 - val_loss: 0.3684 - val_acc: 0.9316
1710 Epoch 906/1000
1711 62s 123ms/step - loss: 0.0596 - acc: 0.9998 - val_loss: 0.3683 - val_acc: 0.9313
1712 Epoch 907/1000
1713 62s 123ms/step - loss: 0.0596 - acc: 0.9998 - val_loss: 0.3681 - val_acc: 0.9314
1714 Epoch 908/1000
1715 62s 123ms/step - loss: 0.0594 - acc: 0.9998 - val_loss: 0.3679 - val_acc: 0.9311
1716 Epoch 909/1000
1717 62s 124ms/step - loss: 0.0597 - acc: 0.9997 - val_loss: 0.3676 - val_acc: 0.9309
1718 Epoch 910/1000
1719 62s 123ms/step - loss: 0.0596 - acc: 0.9997 - val_loss: 0.3673 - val_acc: 0.9311
1720 Epoch 911/1000
1721 62s 123ms/step - loss: 0.0597 - acc: 0.9997 - val_loss: 0.3675 - val_acc: 0.9311
1722 Epoch 912/1000
1723 62s 124ms/step - loss: 0.0596 - acc: 0.9997 - val_loss: 0.3671 - val_acc: 0.9311
1724 Epoch 913/1000
1725 62s 123ms/step - loss: 0.0595 - acc: 0.9997 - val_loss: 0.3666 - val_acc: 0.9314
1726 Epoch 914/1000
1727 62s 124ms/step - loss: 0.0594 - acc: 0.9998 - val_loss: 0.3663 - val_acc: 0.9317
1728 Epoch 915/1000
1729 62s 123ms/step - loss: 0.0598 - acc: 0.9996 - val_loss: 0.3660 - val_acc: 0.9318
1730 Epoch 916/1000
1731 62s 124ms/step - loss: 0.0596 - acc: 0.9997 - val_loss: 0.3658 - val_acc: 0.9320
1732 Epoch 917/1000
1733 62s 124ms/step - loss: 0.0595 - acc: 0.9997 - val_loss: 0.3658 - val_acc: 0.9318
1734 Epoch 918/1000
1735 62s 124ms/step - loss: 0.0596 - acc: 0.9997 - val_loss: 0.3656 - val_acc: 0.9316
1736 Epoch 919/1000
1737 62s 124ms/step - loss: 0.0595 - acc: 0.9997 - val_loss: 0.3654 - val_acc: 0.9316
1738 Epoch 920/1000
1739 62s 123ms/step - loss: 0.0596 - acc: 0.9997 - val_loss: 0.3651 - val_acc: 0.9314
1740 Epoch 921/1000
1741 62s 124ms/step - loss: 0.0593 - acc: 0.9998 - val_loss: 0.3650 - val_acc: 0.9314
1742 Epoch 922/1000
1743 62s 123ms/step - loss: 0.0594 - acc: 0.9998 - val_loss: 0.3650 - val_acc: 0.9316
1744 Epoch 923/1000
1745 62s 124ms/step - loss: 0.0593 - acc: 0.9998 - val_loss: 0.3645 - val_acc: 0.9316
1746 Epoch 924/1000
1747 62s 124ms/step - loss: 0.0594 - acc: 0.9998 - val_loss: 0.3644 - val_acc: 0.9315
1748 Epoch 925/1000
1749 62s 124ms/step - loss: 0.0592 - acc: 0.9998 - val_loss: 0.3648 - val_acc: 0.9315
1750 Epoch 926/1000
1751 62s 124ms/step - loss: 0.0593 - acc: 0.9997 - val_loss: 0.3647 - val_acc: 0.9318
1752 Epoch 927/1000
1753 62s 123ms/step - loss: 0.0594 - acc: 0.9998 - val_loss: 0.3646 - val_acc: 0.9313
1754 Epoch 928/1000
1755 62s 124ms/step - loss: 0.0594 - acc: 0.9998 - val_loss: 0.3649 - val_acc: 0.9317
1756 Epoch 929/1000
1757 62s 123ms/step - loss: 0.0592 - acc: 0.9998 - val_loss: 0.3649 - val_acc: 0.9320
1758 Epoch 930/1000
1759 62s 124ms/step - loss: 0.0591 - acc: 0.9998 - val_loss: 0.3649 - val_acc: 0.9322
1760 Epoch 931/1000
1761 62s 124ms/step - loss: 0.0592 - acc: 0.9999 - val_loss: 0.3647 - val_acc: 0.9318
1762 Epoch 932/1000
1763 62s 124ms/step - loss: 0.0592 - acc: 0.9998 - val_loss: 0.3644 - val_acc: 0.9319
1764 Epoch 933/1000
1765 62s 124ms/step - loss: 0.0594 - acc: 0.9997 - val_loss: 0.3642 - val_acc: 0.9319
1766 Epoch 934/1000
1767 62s 123ms/step - loss: 0.0594 - acc: 0.9998 - val_loss: 0.3646 - val_acc: 0.9318
1768 Epoch 935/1000
1769 62s 124ms/step - loss: 0.0594 - acc: 0.9997 - val_loss: 0.3641 - val_acc: 0.9318
1770 Epoch 936/1000
1771 62s 124ms/step - loss: 0.0592 - acc: 0.9998 - val_loss: 0.3639 - val_acc: 0.9313
1772 Epoch 937/1000
1773 62s 124ms/step - loss: 0.0592 - acc: 0.9998 - val_loss: 0.3637 - val_acc: 0.9321
1774 Epoch 938/1000
1775 62s 124ms/step - loss: 0.0591 - acc: 0.9998 - val_loss: 0.3638 - val_acc: 0.9320
1776 Epoch 939/1000
1777 62s 123ms/step - loss: 0.0590 - acc: 0.9999 - val_loss: 0.3638 - val_acc: 0.9320
1778 Epoch 940/1000
1779 62s 124ms/step - loss: 0.0591 - acc: 0.9998 - val_loss: 0.3633 - val_acc: 0.9324
1780 Epoch 941/1000
1781 62s 123ms/step - loss: 0.0591 - acc: 0.9998 - val_loss: 0.3635 - val_acc: 0.9325
1782 Epoch 942/1000
1783 62s 123ms/step - loss: 0.0592 - acc: 0.9997 - val_loss: 0.3632 - val_acc: 0.9324
1784 Epoch 943/1000
1785 62s 124ms/step - loss: 0.0591 - acc: 0.9997 - val_loss: 0.3637 - val_acc: 0.9330
1786 Epoch 944/1000
1787 62s 124ms/step - loss: 0.0590 - acc: 0.9998 - val_loss: 0.3638 - val_acc: 0.9327
1788 Epoch 945/1000
1789 62s 124ms/step - loss: 0.0591 - acc: 0.9997 - val_loss: 0.3641 - val_acc: 0.9329
1790 Epoch 946/1000
1791 62s 124ms/step - loss: 0.0590 - acc: 0.9998 - val_loss: 0.3641 - val_acc: 0.9328
1792 Epoch 947/1000
1793 62s 124ms/step - loss: 0.0590 - acc: 0.9997 - val_loss: 0.3645 - val_acc: 0.9328
1794 Epoch 948/1000
1795 62s 123ms/step - loss: 0.0591 - acc: 0.9997 - val_loss: 0.3643 - val_acc: 0.9329
1796 Epoch 949/1000
1797 62s 124ms/step - loss: 0.0591 - acc: 0.9997 - val_loss: 0.3643 - val_acc: 0.9328
1798 Epoch 950/1000
1799 62s 124ms/step - loss: 0.0591 - acc: 0.9998 - val_loss: 0.3647 - val_acc: 0.9329
1800 Epoch 951/1000
1801 62s 124ms/step - loss: 0.0589 - acc: 0.9998 - val_loss: 0.3646 - val_acc: 0.9330
1802 Epoch 952/1000
1803 62s 123ms/step - loss: 0.0588 - acc: 0.9998 - val_loss: 0.3645 - val_acc: 0.9326
1804 Epoch 953/1000
1805 62s 124ms/step - loss: 0.0591 - acc: 0.9997 - val_loss: 0.3645 - val_acc: 0.9326
1806 Epoch 954/1000
1807 62s 123ms/step - loss: 0.0589 - acc: 0.9998 - val_loss: 0.3648 - val_acc: 0.9329
1808 Epoch 955/1000
1809 62s 124ms/step - loss: 0.0587 - acc: 0.9999 - val_loss: 0.3648 - val_acc: 0.9327
1810 Epoch 956/1000
1811 62s 124ms/step - loss: 0.0588 - acc: 0.9998 - val_loss: 0.3651 - val_acc: 0.9325
1812 Epoch 957/1000
1813 62s 124ms/step - loss: 0.0589 - acc: 0.9998 - val_loss: 0.3649 - val_acc: 0.9324
1814 Epoch 958/1000
1815 62s 123ms/step - loss: 0.0588 - acc: 0.9998 - val_loss: 0.3650 - val_acc: 0.9320
1816 Epoch 959/1000
1817 62s 124ms/step - loss: 0.0588 - acc: 0.9998 - val_loss: 0.3649 - val_acc: 0.9322
1818 Epoch 960/1000
1819 62s 124ms/step - loss: 0.0590 - acc: 0.9998 - val_loss: 0.3651 - val_acc: 0.9320
1820 Epoch 961/1000
1821 62s 124ms/step - loss: 0.0588 - acc: 0.9998 - val_loss: 0.3652 - val_acc: 0.9325
1822 Epoch 962/1000
1823 62s 124ms/step - loss: 0.0590 - acc: 0.9998 - val_loss: 0.3646 - val_acc: 0.9322
1824 Epoch 963/1000
1825 62s 124ms/step - loss: 0.0590 - acc: 0.9997 - val_loss: 0.3649 - val_acc: 0.9324
1826 Epoch 964/1000
1827 62s 124ms/step - loss: 0.0590 - acc: 0.9998 - val_loss: 0.3651 - val_acc: 0.9322
1828 Epoch 965/1000
1829 62s 123ms/step - loss: 0.0588 - acc: 0.9998 - val_loss: 0.3649 - val_acc: 0.9324
1830 Epoch 966/1000
1831 62s 124ms/step - loss: 0.0589 - acc: 0.9997 - val_loss: 0.3649 - val_acc: 0.9322
1832 Epoch 967/1000
1833 62s 124ms/step - loss: 0.0590 - acc: 0.9996 - val_loss: 0.3649 - val_acc: 0.9328
1834 Epoch 968/1000
1835 62s 124ms/step - loss: 0.0588 - acc: 0.9997 - val_loss: 0.3648 - val_acc: 0.9328
1836 Epoch 969/1000
1837 62s 124ms/step - loss: 0.0589 - acc: 0.9998 - val_loss: 0.3647 - val_acc: 0.9325
1838 Epoch 970/1000
1839 62s 124ms/step - loss: 0.0587 - acc: 0.9998 - val_loss: 0.3645 - val_acc: 0.9325
1840 Epoch 971/1000
1841 62s 123ms/step - loss: 0.0587 - acc: 0.9997 - val_loss: 0.3646 - val_acc: 0.9328
1842 Epoch 972/1000
1843 62s 124ms/step - loss: 0.0588 - acc: 0.9998 - val_loss: 0.3650 - val_acc: 0.9331
1844 Epoch 973/1000
1845 62s 124ms/step - loss: 0.0586 - acc: 0.9998 - val_loss: 0.3649 - val_acc: 0.9331
1846 Epoch 974/1000
1847 62s 124ms/step - loss: 0.0587 - acc: 0.9998 - val_loss: 0.3646 - val_acc: 0.9326
1848 Epoch 975/1000
1849 62s 124ms/step - loss: 0.0586 - acc: 0.9998 - val_loss: 0.3645 - val_acc: 0.9322
1850 Epoch 976/1000
1851 62s 124ms/step - loss: 0.0585 - acc: 0.9998 - val_loss: 0.3645 - val_acc: 0.9326
1852 Epoch 977/1000
1853 62s 124ms/step - loss: 0.0591 - acc: 0.9996 - val_loss: 0.3646 - val_acc: 0.9323
1854 Epoch 978/1000
1855 62s 124ms/step - loss: 0.0587 - acc: 0.9998 - val_loss: 0.3645 - val_acc: 0.9326
1856 Epoch 979/1000
1857 62s 124ms/step - loss: 0.0588 - acc: 0.9997 - val_loss: 0.3646 - val_acc: 0.9320
1858 Epoch 980/1000
1859 62s 124ms/step - loss: 0.0588 - acc: 0.9997 - val_loss: 0.3648 - val_acc: 0.9319
1860 Epoch 981/1000
1861 62s 123ms/step - loss: 0.0587 - acc: 0.9998 - val_loss: 0.3650 - val_acc: 0.9317
1862 Epoch 982/1000
1863 62s 124ms/step - loss: 0.0584 - acc: 0.9999 - val_loss: 0.3650 - val_acc: 0.9317
1864 Epoch 983/1000
1865 62s 124ms/step - loss: 0.0586 - acc: 0.9998 - val_loss: 0.3650 - val_acc: 0.9323
1866 Epoch 984/1000
1867 62s 124ms/step - loss: 0.0586 - acc: 0.9998 - val_loss: 0.3651 - val_acc: 0.9322
1868 Epoch 985/1000
1869 62s 124ms/step - loss: 0.0587 - acc: 0.9997 - val_loss: 0.3651 - val_acc: 0.9321
1870 Epoch 986/1000
1871 62s 124ms/step - loss: 0.0588 - acc: 0.9998 - val_loss: 0.3648 - val_acc: 0.9323
1872 Epoch 987/1000
1873 62s 124ms/step - loss: 0.0586 - acc: 0.9997 - val_loss: 0.3644 - val_acc: 0.9318
1874 Epoch 988/1000
1875 62s 124ms/step - loss: 0.0586 - acc: 0.9997 - val_loss: 0.3648 - val_acc: 0.9322
1876 Epoch 989/1000
1877 62s 124ms/step - loss: 0.0585 - acc: 0.9998 - val_loss: 0.3650 - val_acc: 0.9322
1878 Epoch 990/1000
1879 62s 124ms/step - loss: 0.0586 - acc: 0.9998 - val_loss: 0.3646 - val_acc: 0.9319
1880 Epoch 991/1000
1881 62s 124ms/step - loss: 0.0584 - acc: 0.9998 - val_loss: 0.3647 - val_acc: 0.9323
1882 Epoch 992/1000
1883 62s 124ms/step - loss: 0.0585 - acc: 0.9997 - val_loss: 0.3647 - val_acc: 0.9320
1884 Epoch 993/1000
1885 62s 124ms/step - loss: 0.0587 - acc: 0.9997 - val_loss: 0.3646 - val_acc: 0.9318
1886 Epoch 994/1000
1887 62s 124ms/step - loss: 0.0584 - acc: 0.9999 - val_loss: 0.3650 - val_acc: 0.9320
1888 Epoch 995/1000
1889 62s 124ms/step - loss: 0.0586 - acc: 0.9997 - val_loss: 0.3650 - val_acc: 0.9315
1890 Epoch 996/1000
1891 62s 123ms/step - loss: 0.0585 - acc: 0.9998 - val_loss: 0.3649 - val_acc: 0.9319
1892 Epoch 997/1000
1893 62s 123ms/step - loss: 0.0585 - acc: 0.9998 - val_loss: 0.3645 - val_acc: 0.9318
1894 Epoch 998/1000
1895 62s 124ms/step - loss: 0.0584 - acc: 0.9998 - val_loss: 0.3648 - val_acc: 0.9320
1896 Epoch 999/1000
1897 62s 124ms/step - loss: 0.0584 - acc: 0.9999 - val_loss: 0.3646 - val_acc: 0.9320
1898 Epoch 1000/1000
1899 62s 124ms/step - loss: 0.0581 - acc: 0.9999 - val_loss: 0.3646 - val_acc: 0.9323
1900 Train loss: 0.062079589650034905
1901 Train accuracy: 0.9986200013160705
1902 Test loss: 0.3645842906832695
1903 Test accuracy: 0.9323000019788742

中间有一部分epoch（从第701到754个epoch）的结果没记录下来。

Minghang Zhao, Shisheng Zhong, Xuyun Fu, Baoping Tang, Shaojiang Dong, Michael Pecht, Deep Residual Networks with Adaptively Parametric Rectifier Linear Units for Fault Diagnosis, IEEE Transactions on Industrial Electronics, 2020, DOI: 10.1109/TIE.2020.2972458

https://ieeexplore.ieee.org/document/8998530

posted @ 2020-05-17 22:42 世俗杂念阅读(1034) 评论(0) 编辑收藏举报

会员力量，点亮园子希望

刷新页面返回顶部

深度残差网络+自适应参数化ReLU激活函数（调参记录6）

公告