CS760, Fall 2022
Department of Computer Sciences
University of Wisconsin–Madison
This schedule is tentative and subject to change. Please check back often. In particular, the deadlines for the homework sets/project may change; please see Canvas for the actual deadlines.
The reading is not required but strongly recommended for all students. Those explicitly noted as optional are for students interested in that specific topic. "A, B; C; D" means to read (A OR B) AND C AND D. Text in red means a link to the reading material. Abbreviations for textbooks:
| Date | Lecture | Readings | Homework Released | Homework Due | |
|---|---|---|---|---|---|
| Thursday Sept. 8 | Course Overview : [Slides] | Jordan and Mitchell, Science, 2015 | [Homework 1] | ||
| Tuesday Sept. 13 | ML Overview: Supervised/Unsupervised/RL, Classification/Regression, General Approach [Slides] | ||||
| Thursday Sept. 15 | Supervised Learning I: Setup, Examples. Instance-Based Learning, Decision Trees [Slides] | ||||
| Tuesday Sept. 20 | Supervised Learning II: Setup + Examples. Decision Trees [Slides] | Murphy Chapter 16.2 / Shalev-Shwartz and Ben-David Chapter 18 | |||
| Thursday Sept. 22 | Evaluation: Bias, Cross-Validation, Precision/Recall, ROC Curves [Slides] | ||||
| Tuesday Sept. 27 | Regression I: Linear Regression, Logistic Regression, Normal equations, GD [Slides] | Murphy 8.1-3 and 8.6; Maximum Likelihood, Logistic Regression, and Stochastic Gradient Training | [Homework 2] | ||
| Thursday Sept. 29 | Regression II: Logistic Regression, Gradient Descent Analysis, SGD [Slides] | Convergence Theorems for Gradient Descent | |||
| Tuesday Oct. 4 | Naive Bayes: Generative vs Discriminative Models, ML vs MAP [Slides] | Mitchell 6.1-6.10, Murphy 3 | Homework 2 due | ||
| Thursday Oct. 6 | Neural Networks I: Perceptron, Training, MLPs [Slides] | Mitchell chapter 4, Murphy 16.5 and 28, Bishop 5.1-5.3 LeCun et al., Nature, 2015 | [Homework 3] | ||
| Tuesday Oct. 11 | Neural Networks II: Training, Optimization, SGD, Backpropagation [Slides] | ||||
| Thursday Oct. 13 | Neural Networks III: Regularization, Data Augmentation [Slides] | ||||
| Tuesday Oct. 18 | Neural Networks IV: CNNs [Slides] | Goodfellow-Bengio-Courville chapter 9; CNN papers 1. LeNet 2. AlexNet 3. ResNet | |||
| Thursday Oct. 20 | Neural Networks IV: RNNs [Slides] | Goodfellow-Bengio-Courville chapter 10; Optional papers to read for part 4: 1. LSTM 2. GRU | [Homework 4] | ||
| Tuesday Oct. 25 | Practical Aspects of Training + Review [Slides] | ||||
| Tuesday Nov. 1 | Generative Models: Autoregressive, Flows, GANs [Slides] | Optional tutorial: Goodfellow's GAN tutorial; Optional papers: 1. Normalizing Flows for Probabilistic Modeling and Inference 2. Generative Adversarial Networks (GANs) | |||
| Thursday Nov. 3 | Kernels + SVMs:Margins, Support Vectors, Kernels [Slides] | Andrew Ng's note on SVM, Ben-Hur and Weston's note; Mohri-Rostamizadeh-Talwalkar Appendix B (Convex Optimization), Bishop Appendix E (Lagrange Multipliers) | |||
| Tuesday Nov. 8 | Graphical Models I: Bayesian Networks, Training, Structure Learning [Slides] | ||||
| Thursday Nov. 10 | Graphical Models II: Undirected Models, Markov Random Fields [Slides] | Mitchell chapter 6, Bishop chapter 8.1, Shalev-Shwartz and Ben-David chapter 24; Heckerman Tutorial, Wainwright and Jordan Chapters 2, 3 | |||
| Tuesday Nov. 15 | Unsupervised Learning I: Clustering, GMM models, EM [Slides] | ||||
| Thursday Nov. 17 | Unsupervised Learning II: Dimensionality Reduction, PCA [Slides] | ||||
| Tuesday Nov. 22 | Learning Theory: Generalization, PAC [Slides] | Mohri-Rostamizadeh-Talwalkar Chapter 2, Mitchell Chapter 7 | |||
| Tuesday Nov. 29 | Reinforcement Learning I: MDPs, Value Iteration, Policy Iteration [Slides] | Mitchell Chapter 13 | |||
| Tuesday Dec. 6 | Reinforcement Learning II: Q-learning, Approximation [Slides] | ||||
| Thursday Dec. 8 | Reinforcement Learning III: Function Approximation, Policy Search, Reinforce |