Description:

This course is about deep learning. Students taking this course will learn the theories, models, algorithms, and recent progress of deep learning. The course starts with machine learning basics and classical neural network models, followed by deep convolutional neural networks, recurrent neural networks, reinforcement learning, etc., and their applications. Students are expected to have good knowledge of calculus, linear algebra, probability and statistics as prerequisites.

This website contains only information of Part 1 (the first half of the course).

---

Textbooks (The textbooks are used loosely):

• Deep Learning, Ian Goodfellow and Yoshua Bengio and Aaron Courville.
• Dive into Deep Learning, Zhang et al.

Instructor: Joey Tianyi Zhou

---

Teaching Schedule:

The following schedule will not be strictly followed.

Date	Topic	Notes
Week 1	1. Introduction	History of Deep Learning. Machine Learning Settings.
Week 2	2. Machine Learning Basics	Data representation. Linear Regression/Classification. K-NN. SVM.
Week 3	3. Machine Learning Basics II	Linear Regression/Classification, Nonlinear Classification, Overfitting, Regularization, Hypermeters.
Week 4	4. Fully Connected Neural Networks	Learning Visual Features. Training Neural Networks. Loss Optimization, Gradient Descent. Regularization.
Week 5	5. Autoencoder & CNN	Variants of Autoencoder CNN Pipeline. Adversarial examples/training.
Week 6	6. Why Deep? Why Small?	Deeper vs wider? Model Compression for DNN.

---

Syllabus:

• CNN, architectures, image classification, object detection, domain adaptation
• Deep Generative Networks: Autoencoders, UNets, variational autoencoders (VAE), generative adversarial networks (GAN), pix2pix, cycleGAN, recent developments and applications
• RNN, LSTM, Attention, Transformer, Vision Transformer, recent developments and applications
• Deep Reinforcement Learning, Q-Learning, DQN, Policy Gradient, Value Function, Actor-Critics, applications
• Bayesian Neural Network (BNN), Bayesian deep learning, applications
• Advanced topics: few shot, imbalanced classes, out of distribution, etc.

---