Feature Extractor[VGG]


0. AlexNet


1. VGG

VGG网络相对来说,结构简单,通俗易懂,作者通过分析2013年imagenet的比赛的最好模型,并发现感受野还是小的好,然后再加上《network in network》中的\(1*1\)卷积核,使得全文只在卷积网络的深度上做文章,从而得出了网络还是越深越好的结论
VGG


2. Inception

与VGG同期出来的有googlenet,该网络通过关注减少模型参数,而不降低模型性能的角度出发,设计出了inception结构,提出了googlenet

然后google人员发现因为网络在训练过程中,下层参数的变化会导致下层输出的数据分布不断变化,从而当前层乃至后面层都需要不断地去拟合新的分布,这个想法也是来自于迁移学习中的covariate shift问题(即,训练样本和测试样本其实不一致)。
而借鉴迁移学习的这个概念,如果细化到网络内部的每一层的话,该现象又叫做internal covariate shift。
那么通过白化角度,的确可以解决这个问题,可是如果白化独立于网络的优化算法而存在,会导致网络没什么改变,而且会因为两者不相关而让其中的网络参数不断变大,即使loss稳定了,也还是会变大。所以就需要设计出一种即放入优化算法,而又能够对每层网络的输入进行类白化的效果。而如果基于mini-batch进行0均值和1方差,本身又会引入新的问题:
从概率论角度出发:后验概率=先验概率*联合概率。
如果只是做类白化,那么就相当于引入了联合概率,从而数据的分布其实还是会变化的,而且拿sigmoid举例的话,也会导致网络根本达不到其非线性部分,而一直处在线性部分。
那么解决该问题的角度就是,引入修复变量,进行线性变换,使得变换后的数据分布能够修复之前0均值1方差带来的损失,从而提出了BN(被大家认为是inception v2)

inception v2
inception V3
inception V4


3. HighwayNet


4. ResNet

ResNet v1
ResNet v2


5. DenseNet

DenseNet


6. MobileNet

7. ShuffleNet

8. SENet

SENet


9. NASNet

参考文献:

  1. [原理] Zeiler M D, Fergus R. Visualizing and understanding convolutional networks[C]//European conference on computer vision. Springer, Cham, 2014: 818-833.
  2. [alexnet] Krizhevsky A, Sutskever I, Hinton G E. Imagenet classification with deep convolutional neural networks[C]//Advances in neural information processing systems. 2012: 1097-1105.
  3. [vgg] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[J]. arXiv preprint arXiv:1409.1556, 2014.
  4. [googlenet] Szegedy C, Liu W, Jia Y, et al. Going deeper with convolutions[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2015: 1-9.
    .. [bn&inception v2] Ioffe S, Szegedy C. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift[C]//Proceedings of the 32nd International Conference on Machine Learning (ICML-15). 2015: 448-456.
    .. [BRN] Ioffe S. Batch Renormalization: Towards Reducing Minibatch Dependence in Batch-Normalized Models[J]. arXiv preprint arXiv:1702.03275, 2017
    .. [inception v3] Szegedy C, Vanhoucke V, Ioffe S, et al. Rethinking the inception architecture for computer vision[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016: 2818-2826.
    .. [inception v4] Szegedy C, Ioffe S, Vanhoucke V, et al. Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning[C]//AAAI. 2017: 4278-4284.
  5. [highway net] R. K. Srivastava, K. Greff, and J. Schmidhuber. Highway networks. arXiv:1505.00387, 2015.
    .. [highway net v2] Srivastava R K, Greff K, Schmidhuber J. Training very deep networks[C]//Advances in neural information processing systems. 2015: 2377-2385.
  6. [resnet v1] He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 770-778.
    .. [resnet v2] He K, Zhang X, Ren S, et al. Identity mappings in deep residual networks[C]//European Conference on Computer Vision. Springer International Publishing, 2016: 630-645.
    .. [wider or deeper] Wu Z, Shen C, Hengel A. Wider or deeper: Revisiting the resnet model for visual recognition[J]. arXiv preprint arXiv:1611.10080, 2016.
    .. [WRN] Zagoruyko S, Komodakis N. Wide residual networks[J]. arXiv preprint arXiv:1605.07146, 2016.
    ..[ResNext] Xie S, Girshick R, Dollár P, et al. Aggregated residual transformations for deep neural networks[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2017: 5987-5995.
  7. [Dense Net] Huang G, Liu Z, Weinberger K Q, et al. Densely connected convolutional networks[J]. arXiv preprint arXiv:1608.06993, 2016.
    .. [Dense Net] Pleiss G, Chen D, Huang G, et al. Memory-Efficient Implementation of DenseNets[J]. arXiv preprint arXiv:1707.06990, 2017.
    .. [why&how] .DenseNet 的“what”、“why”和“how”
    .. [多尺度DenseNet] Huang G, Chen D, Li T, et al. Multi-Scale Dense Convolutional Networks for Efficient Prediction[J]. arXiv preprint arXiv:1703.09844, 2017.
  8. [MobileNet] Howard A G, Zhu M, Chen B, et al. Mobilenets: Efficient convolutional neural networks for mobile vision applications[J]. arXiv preprint arXiv:1704.04861, 2017.
    ..[MobileNetV2] Sandler M, Howard A, Zhu M, et al. MobileNetV2: Inverted Residuals and Linear Bottlenecks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018: 4510-4520.
  9. [SENet] Hu J, Shen L, Sun G. Squeeze-and-Excitation Networks[J]. arXiv preprint arXiv:1709.01507, 2017.
  10. [xception] Chollet F. Xception: Deep Learning with Depthwise Separable Convolutions[J]. arXiv preprint arXiv:1610.02357, 2016.
  11. [NASNet] B. Zoph and Q. V. Le. Neural architecture search with reinforcement learning. In International Conference on Learning Representations, 2017.
    .. [NASNet] Zoph B, Vasudevan V, Shlens J, et al. Learning transferable architectures for scalable image recognition[J]. arXiv preprint arXiv:1707.07012, 2017. .
posted @ 2017-10-23 15:14  仙守  阅读(1826)  评论(0编辑  收藏  举报