Description:
This course is about deep learning.
Students taking this course will learn the
theories, models, algorithms, and recent progress of deep
learning. The course starts
with machine learning basics and classical neural network
models, followed by deep convolutional neural networks,
recurrent neural networks, reinforcement learning, etc., and
their applications.
Students are expected to have good knowledge of calculus,
linear algebra, probability and statistics as prerequisites.
This website contains only information of Part 1 (the first
half of the course).
Textbooks (The textbooks are used loosely):
Instructor:
Assoc. Prof. (adj.) Joey Tianyi
Zhou (Part 1)
Lecturer Yuecong Xu (Part 2)
Teaching Schedule:
The following schedule will not be strictly followed.
Syllabus:
- 1. Introduction to deep learning (Week 1):
a) Relation and difference between deep learning and machine learning;
b) History of deep learning and recent developments;
c) Overview of popular models and applications;
d) Basic terminology and concepts: e.g., common learning methods, common network structures, etc.
- 2. Neural networks, training pipeline and learning strategies (Week 2):
a) Introduction classic (and basic) multi-layer perceptron neural network (MLP);
b) Introduction the training pipeline of MLP: learning with backpropagation;
c) Explain the optimization algorithm: SGD (with momentum), Adam, Adagrad, Second-order optimization, Hessian-free optimization, theoretical analysis of deep neural network model performance;
d) Explain the different learning strategies: supervised, semi-supervised and unsupervised learning with examples. Include more specific but common learning strategies such as self-supervised learning (including contrastive learning) and transfer learning;
e) Explain the possible shortcomings of the training pipeline with BP: overfitting, and the solution to mitigate overfitting via regularization (network dropout, data augmentation, etc.)
- 3. Autoencoder, CNN and RNN and their Variants (Week 3):
a) Introduction to convolutional / recurrent neural networks: structure, training, and characteristics, including the classic and recent variants;
b) Introduction to the two basic tasks in computer vision and natural language processing: image and text classification, including the tasks and dataset for the tasks;
c) Introduction to recent advances of CNNs/RNNs, including the combination of CNNs/RNNs for image captioning, action recognition, etc.;
d) Explain the reason for the rise and fall of CNNs and RNNs.
- 4. Transformers (Week 4):
a) Introduction to self-attention and Transformer model, including advantages over CNNs and RNNs;
b) Explain the expansion of Transformers to the vision tasks: the non-local block and the ViT. Further introduce the expansion of Transformers to regression-based task (object detection): DETR;
c) Introduce the recent advances of Transformer models (focus on advances in architecture, leave applications such as to videos and LL(V)Ms to following).
- 5. Prompting (Week 5):
a) Introduction to prompting;
b) Explain the usage of prompting in both NLP and CV domains.
- 6. Model and Dataset Compression (Week 6)
a) Introduction to Model Compression including pruning, distillation, and quantization.
b) Introduction to Dataset Distillation and Pruning.