This section provides links to lecture notes for the course, along with corresponding reading assignments and supplementary materials. Some of the reading assignments refer to chapters from a draft textbook which is not currently available on this site.
|
|
|
|
|
|
|
|
LEC # |
|
|
|
LECTURE |
|
|
|
READINGS |
|
|
OTHER MATERIALS |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
|
|
|
Introduction (PDF) |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2 |
|
|
|
Linear Regression (PDF) |
|
|
|
(Jordan & Bishop: Chapters 5-5.3) |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
3 |
|
|
|
Additive Models, Maximum Likelihood (PDF) |
|
|
|
(Jordan & Bishop: Chapter 4 (up to eq 4.20) Chapters 5-5.3 & 5.6) |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
4 |
|
|
|
Active Learning (PDF) |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
5 |
|
|
|
Classification (PDF) |
|
|
|
(Jordan & Bishop: Chapter 4 Mixture Models; Chapter 6-6.3.1) |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
6 |
|
|
|
Logistic Regression, Regularization (PDF) |
|
|
|
|
|
|
Additional Notes on Regularization (PDF) |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
7 |
|
|
|
Regularization, Support Vector Machines (PDF) |
|
|
|
Optional Paper: Burges, C. "A Tutorial on Support Vector Machines for Pattern Recognition," Kluwer Academic Publishers. |
|
|
Tutorial on Lagrange Multipliers (PDF) |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
8 |
|
|
|
Support Vector Machines, Text Classification (PDF) |
|
|
|
Optional Paper: Schapire, R.E. "A Brief Introduction to Boosting," Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence, 1999.
Optional Paper: Friedman, J., Trevor, H., and Tibshirani, R. "Additive Logistic Regression: A Stastical View Of Boosting," 1999. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
9 |
|
|
|
Feature Selection, Combination of Methods (PDF) |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
10 |
|
|
|
Boosting, Complexity (PDF) |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
11 |
|
|
|
Structural Risk Minimization, Description Length
(+ Midterm Discussion) (PDF) |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
MIDTERM EXAM |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
12 |
|
|
|
Mixture Models, EM (PDF) |
|
|
|
(Jordan & Bishop: Chapter 4 Statistical Concepts, Mixture Models; Chapter 9 up to 9.2) |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
13 |
|
|
|
EM, Regularization, Conditional Mixtures (PDF) |
|
|
|
(Jordan & Bishop: Chapter 9 up to 9.2.3) |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
14 |
|
|
|
Non-Parametric Density Estimation, Clustering (PDF) |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
15 |
|
|
|
Clustering, Markov Models (PDF) |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
16 |
|
|
|
Markov and Hidden Markov Models (PDF) |
|
|
|
(Jordan & Bishop: Chapter 11)
Optional Paper: Rabiner, L.R. "A Tutorial on Hidden Markov Model and Selected Applications in Speech Recognition," Proceedings of the IEEE, Vol. 77, No. 2, February 1989. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
17 |
|
|
|
Hidden Markov Models (PDF) |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
18 |
|
|
|
Viterbi, Graphical Models (PDF) |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
19 |
|
|
|
Bayesian Networks (PDF) |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
20 |
|
|
|
Medical Diagnosis Example, Influence Diagrams(PDF) |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
21 |
|
|
|
Influence Diagrams, Exact Inference (PDF) |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
22 |
|
|
|
Belief Propagation (PDF) |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
23 |
|
|
|
Learning Graphical Models (Guest Lecture) (PDF) |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
FINAL EXAM |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|