2019/11/20 ReLU论文学习

论文地址： Deep Sparse Rectifier Neural Networks

ReLU的好处

1. 增加了网络的稀疏性，而稀疏性被证明对网络的表达有好处。

如何善用ReLU

1. rectifying activation allows deep networks to achieve their best performance without unsupervised pre-training.
2. Still, rectifier networks can benefit from unsupervised pre-training in the context of semi-supervised learning where large amounts of unlabeled data are provided.

资料汇总

博客1