Deep Learning lectures

This post highlights some great lectures about deep learning

Stanford CS229

  • Only 1 [slide] which gives an intro about deep learning.

Quoc Le’s Lectures on Deep Learning

  • [Original lectures]
  • If you are interested in high level overview of deep learning, these lectures will be a great for your needs. The lectures only have 3 videos of 1 hour each.

Update:  Dr. Le has posted tutorials on this topic: Part 1 and Part 2.

Dr. Quoc Le from the Google Brain project team (yes, the one that made headlines for creating a cat recognizer) presented a series of lectures at the Machine Learning Summer School (MLSS ’14) in Pittsburgh this week. This is my favorite lecture series from the event till now and I was glad to be able to attend them.

The good news is that the organizers have made available the entire set of video lectures in 4K for you to watch. But since Dr. Le did most of them on the board and did not provide any accompanying slides, I decided to put the contents of the lectures along with the videos here.

In this post I posted Dr. Le’s lecture videos and added content links with short descriptions to help you navigate them better.

Event TypeDescriptionCourse Materials
Lecture Neural Networks Review [Video]
[deep-learning-tutorial-1]
Lecture NNs in Practice [Video]
Lecture Deep NN Architectures [Video]
[deep-learning-tutorial-2]

CS231n: Convolutional Neural Networks for Visual Recognition

Syllabus

Event TypeDateDescriptionCourse Materials
Lecture Jan 4 Intro to Computer Vision, historical context. [slides] [video]
Lecture Jan 6 Image classification and the data-driven approach
k-nearest neighbor
Linear classification I
[slides] [video]
[python/numpy tutorial]
[image classification notes]
[linear classification notes]
Lecture Jan 11 Linear classification II
Higher-level representations, image features
Optimization, stochastic gradient descent
[slides] [video]
[linear classification notes]
[optimization notes]
Lecture Jan 13 Backpropagation
Introduction to neural networks
[slides] [video]
[backprop notes]
[Efficient BackProp] (optional)
related: [1], [2], [3] (optional)
Lecture Jan 18 Holiday; No class.
A1 Due Jan 20 Assignment #1 (kNN/SVM/Softmax/2-Layer Net) Due date [Assignment #1]
Lecture Jan 20 Training Neural Networks Part 1
activation functions, weight initialization, gradient flow, batch normalization
babysitting the learning process, hyperparameter optimization
[slides] [video]
Neural Nets notes 1
Neural Nets notes 2
Neural Nets notes 3
tips/tricks: [1], [2], [3] (optional)
Deep Learning [Nature] (optional)
Lecture Jan 25 Training Neural Networks Part 2: parameter updates, ensembles, dropout
Convolutional Neural Networks: intro
[slides] [video]
Neural Nets notes 3
Lecture Jan 27 Convolutional Neural Networks: architectures, convolution / pooling layers
Case study of ImageNet challenge winning ConvNets
[slides] [video]
ConvNet notes
Proposal due Jan 30 Couse Project Proposal due [proposal description]
Lecture Feb 1 ConvNets for spatial localization
Object detection
[slides] [video]
Lecture Feb 3 Understanding and visualizing Convolutional Neural Networks
Backprop into image: Visualizations, deep dream, artistic style transfer
Adversarial fooling examples
[slides] [video]
A2 Due Feb 5 Assignment #2 (Neural Nets) Due date [Assignment #2]
Lecture Feb 8 Recurrent Neural Networks (RNN), Long Short Term Memory (LSTM)
RNN language models
Image captioning
[slides] [video]
DL book RNN chapter (optional)
min-char-rnn, char-rnn, neuraltalk2
Midterm Feb 10 In-class midterm
Lecture Feb 15 Holiday; No class.
Milestone Feb 17 Course Project Milestone
Lecture Feb 17 Training ConvNets in practice
Data augmentation, transfer learning
Distributed training, CPU/GPU bottlenecks
Efficient convolutions
[slides] [video]
Lecture Feb 22 Overview of Caffe/Torch/Theano/TensorFlow [slides] [video]
A3 Due Feb 24 Assignment #3 (ConvNets) Due date [Assignment #3]
Lecture Feb 24 Segmentation
Soft attention models
Spatial transformer networks
[slides] [video]
Lecture Feb 29 ConvNets for videos
Unsupervised learning
[slides] [video]
Lecture Mar 2 Invited Talk: Jeff Dean [video]
Lecture Mar 7 Student spotlight talks, conclusions [slides]
Poster Presentation Mar 9
Final Project Due Mar 13 Final course project due date [reports]