The lecture notes for this course were prepared by Alexander Rakhlin, a student in the class.
Lecture files.
| Lec # |
Topics |
| 1 |
Introduction |
| 2 |
Voting Classifiers, Training Error of Boosting (PDF) |
| 3 |
Support Vector Machines (SVM) (PDF) |
| 4 |
Generalization Error of SVM (PDF) |
| 5 |
One Dimensional Concentration Inequalities; Bennett's Inequality (PDF) |
| 6 |
Bernstein's Inequality (PDF) |
| 7 |
Hoeffding's Inequality, Hoeffding-Chernoff Inequality (PDF) |
| 8 |
Vapnik-Chervonenkis Classes of Sets (PDF) |
| 9 |
Properties of VC Classes of Sets (PDF) |
| 10 |
Symmetrization; Pessimistic VC Inequality (PDF) |
| 11 |
Optimistic VC Inequality (PDF) |
| 12 |
VC Subgraph Classes of Functions; Packing and Covering Numbers (PDF) |
| 13 |
Covering Numbers of the VC Subgraph Classes (PDF) |
| 14 |
Kolmogorov's Chaining Method; Dudley's Entropy Integral (PDF) |
| 15 |
More Symmetrization; Generalized VC Inequality (PDF) |
| 16 |
Consequences of the Generalized VC Inequality (PDF) |
| 17 |
Covering Numbers of the Convex Hull (PDF) |
| 18-20 |
Bounds on the Generalization Error of Voting Classifiers (PDF 1) (PDF 2) (PDF 3) |
| 21 |
Bounds in Terms of Sparsity (PDF) |
| 22 |
Martingale-difference Inequalities (PDF) |
| 23 |
Empirical and Rademacher Processes; Comparison Inequality for Rademacher Processes (PDF) |
| 24-25 |
Generalization Bounds for Neural Networks (PDF 1) (PDF 2) |
| 26 |
Talagrand's Convex-hull Distance Inequality (PDF) |
| 27 |
Consequences of Talagrand's Convex-hull Distance Inequality (PDF) |
| 28 |
Talagrand's Concentration Inequality for Empirical Processes (PDF) |
| 29 |
Talagrand's Two-point Inequality (PDF) |
| 30 |
Talagrand's Concentration Inequality for Empirical Processes (cont.) (PDF) |
| 31 |
Applications of Talagrand's Concentration Inequality (PDF) |
| 32 |
Applications of Talagrand's Convex-hull Distance Inequality; Bin Packing (PDF) |
| 33 |
Generalization Bounds for Kernel Methods (PDF) |
| 34 |
Optimistic VC Inequality for Random Classes of Sets (PDF) |
| 35 |
Applications of Random VC Inequality to Voting Algorithms and SVM (PDF) |