deep learning 的综述

从13年11月初开始接触DL,奈何boss忙or 各种问题,对DL理解没有CSDN大神 比如 zouxy09等 深刻,主要是自己觉得没啥进展,感觉荒废时日(丢脸啊,这么久。。。。)开始开文,即为记录自己是怎么一步一个逗比的走过的路的,也为了自己思维更有条理。请看客,轻拍,(如果有错,我会立马改正,谢谢大家的指正。==!其实有人看没人看都是个问题。哈哈)

 推荐 tornadomeet 的博客园学习资料

 zouxy09 的csdn学习资料



Rachel-Zhang 浙大DL女神



2009  Learning Deep Architectures for AI

2010 Deep Machine Learning – A New Frontier in Artificial Intelligence Research

2011 An Introduction to Deep Learning

2012  Representation Learning: A Review and New Perspectives

2012 深度学习研究综述

2014  Deep Learning in Neural Networks: An Overview

 2014 Object Detection with Deep Learning CVPR 2014 Tutorial




2014  A Tutorial Survey of Architectures, Algorithms, and Applications for Deep Learning

      其实很多比如PPT什么的就很好,比如hinton的,andrew ng的 ,Yann LeCun的,Yoshua Bengio的,从他们的home page上就可以找到很多有用的文章。他们作为大神,没话说,而且难能可贵的 作为老师,他们也出了视频,或者很多对于我们菜鸟的浅显入门的东西。还有吴立德,吴老爷子的教学视频(优酷就有,但是杂音太多)。公开课可下载资源汇总  (这里有很全的视频学习资料,比如ng的机器学习,hinton的机器学习,自然语言处理各种。)

读书列表,有列的,也有Yoshua Bengio推荐的书单(有些链接失效的。我这个月才发现这个,如果太老,或者什么请忽略我。)

Reading lists for new LISA students
Research in General
● How to write a great research paper
Basics of machine learning
Basics of deep learning
● Learning deep architectures for AI
● Practical recommendations for gradientbased
training of deep architectures
● Quick’n’dirty introduction to deep learning: Advances in Deep Learning
● A fast learning algorithm for deep belief nets
● Greedy LayerWise
Training of Deep Networks
● Stacked denoising autoencoders: Learning useful representations in a deep network with
a local denoising criterion
● Contractive autoencoders:
Explicit invariance during feature extraction
● Why does unsupervised pretraining
help deep learning?
● An Analysis of Single Layer Networks in Unsupervised Feature Learning
● The importance of Encoding Versus Training With Sparse Coding and Vector
● Representation Learning: A Review and New Perspectives
● Deep Learning of Representations: Looking Forward
● Measuring Invariances in Deep Networks
● Neural networks course at USherbrooke [youtube]
Feedforward nets
● “Improving Neural Nets with Dropout” by Nitish Srivastava
● “Deep Sparse Rectifier Neural Networks”
● “What is the best multistage
architecture for object recognition?”
● “Maxout Networks”
● Iain Murray’s MLSS slides
● Radford Neal’s Review Paper (old but still very comprehensive)
● Better Mixing via Deep Representations
Restricted Boltzmann Machines
● Unsupervised learning of distributions of binary vectors using 2layer
● A practical guide to training restricted Boltzmann machines
● Training restricted Boltzmann machines using approximations to the likelihood gradient
● Tempered Markov Chain Monte Carlo for training of Restricted Boltzmann Machine
● How to Center Binary Restricted Boltzmann Machines
● Enhanced Gradient for Training Restricted Boltzmann Machines
● Using fast weights to improve persistent contrastive divergence
● Training Products of Experts by Minimizing Contrastive Divergence
Boltzmann Machines
● Deep Boltzmann Machines (Salakhutdinov & Hinton)
● Multimodal Learning with Deep Boltzmann Machines
● MultiPrediction
Deep Boltzmann Machines
● A Twostage
Pretraining Algorithm for Deep Boltzmann Machines
Regularized Auto-Encoders
● The Manifold Tangent Classifier
Stochastic Nets & GSNs
● Estimating or Propagating Gradients Through Stochastic Neurons for Conditional
● Learning Stochastic Feedforward Neural Networks
● Generalized Denoising AutoEncoders
as Generative Models
● Deep Generative Stochastic Networks Trainable by Backprop
● Slow, Decorrelated Features for Pretraining Complex Celllike
● What Regularized AutoEncoders
Learn from the Data Generating Distribution
● Generalized Denoising AutoEncoders
as Generative Models
● Why the logistic function?
Recurrent Nets
● Learning longterm
dependencies with gradient descent is difficult
● Advances in Optimizing Recurrent Networks
● Learning recurrent neural networks with Hessianfree
● On the importance of momentum and initialization in deep learning,
● Long shortterm
memory (Hochreiter & Schmidhuber)
● Generating Sequences With Recurrent Neural Networks
● Long ShortTerm
Memory in Echo State Networks: Details of a Simulation Study
● The "echo state" approach to analysing and training recurrent neural networks
● BackpropagationDecorrelation:
online recurrent learning with O(N) complexity
● New results on recurrent network training:Unifying the algorithms and accelerating
● Audio Chord Recognition with Recurrent Neural Networks
● Modeling Temporal Dependencies in HighDimensional
Sequences: Application to
Polyphonic Music Generation and Transcription
Convolutional Nets
● Generalization and Network Design Strategies (LeCun)
● ImageNet Classification with Deep Convolutional Neural Networks, Alex Krizhevsky, Ilya
Sutskever, Geoffrey E Hinton, NIPS 2012.
● On Random Weights and Unsupervised Feature Learning
Optimization issues with DL
● Curriculum Learning
● Evolving Culture vs Local Minima
● Knowledge Matters: Importance of Prior Information for Optimization
● Efficient Backprop
● Practical recommendations for gradientbased
training of deep architectures
● Natural Gradient Works Efficiently (Amari 1998)
● Hessian Free
● Natural Gradient (TONGA)
● Revisiting Natural Gradient
● Natural Language Processing (Almost) from Scratch
● DeViSE: A Deep VisualSemantic
Embedding Model
● Distributed Representations of Words and Phrases and their Compositionality
● Dynamic Pooling and Unfolding Recursive Autoencoders for Paraphrase Detection
● Fields of Experts
● What makes a good model of natural images?
● Phone Recognition with the meancovariance
restricted Boltzmann machine
● Unsupervised Models of Images by SpikeandSlab
● Imagenet classification with deep convolutional neural networks
● Learning to relate images
Scaling Up
● Large Scale Distributed Deep Networks
● Random search for hyperparameter
● Practical Bayesian Optimization of Machine Learning Algorithms
DL + Reinforcement learning
● Playing Atari with Deep Reinforcement Learning (paper not officially released yet!)
Graphical Models Background
● An Introduction to Graphical Models (Mike Jordan, brief course notes)
● A View of the EM Algorithm that Justifies Incremental, Sparse and Other Variants (Neal &
Hinton, important paper to the modern understanding of ExpectationMaximization)
● A Unifying Review of Linear Gaussian Models (Roweis & Ghahramani, ties together PCA,
factor analysis, hidden Markov models, Gaussian mixtures, kmeans,
linear dynamical
● An Introduction to Variational Methods for Graphical Models (Jordan et al, meanfield,
● Writing a great research paper (video of the presentation)
Software documentation
● Python, Theano, Pylearn2, Linux (bash) (at least the 5 first sections), git (5 first sections),
github/contributing to it (Theano doc), vim tutorial or emacs tutorial
Software lists of built-in commands/functions
● Bash commands
● List of Builtin
Python Functions
● vim commands
Other Software stuff to know about:
● screen
● ssh
● ipython
● matplotlib

posted @ 2014-08-25 16:51  仙守  阅读(1570)  评论(0编辑  收藏  举报