This is an archived course. A more recent version may be available at ocw.mit.edu.

Help   |   Contact Us

 

Lecture Notes

This section includes the complete lecture notes from Fall 2008, based on the third edition of the course textbook, both as one file and broken down by session. For reference, it also includes the complete lecture notes from Fall 2003, based on the second edition of the textbook.

Complete Lecture Slides

Complete slides, Fall 2008 (PDF - 2.4 MB)

Complete slides, Fall 2003 (PDF - 1.9 MB)

Lecture Slides by Session

SES # TOPICS
1 Introduction to dynamic programming; examples and formulation (PDF)
2 The dynamic programming algorithm (PDF)
3 Deterministic systems and the shortest path problem (PDF)
4 Shortest path algorithms (PDF)
5 Deterministic continuous-time optimal control (PDF)
6 Stopping and scheduling problems (PDF)
7 Linear systems with quadratic costs and inventory control (PDF)
8 Problems with imperfect state information (PDF)
9 Sufficient statistics (PDF)
10 Suboptimal control (PDF)
11 Rollout algorithms (PDF)
12 More on suboptimal control (PDF)
13 Infinite horizon I: stochastic shortest path problems (PDF)
14 Infinite horizon II: discounted problems (PDF)
15 Infinite horizon III: average cost problems (PDF)
16 Semi-Markov problems (PDF)
17 Infinite horizon: discounted problems I (PDF)
18 Infinite horizon: discounted problems II (PDF)
Midterm
19 Stochastic shortest path problems (PDF)
20

Overview of main approaches in approximate dynamic programming (PDF)

Detailed outline for approximate dynamic programming, lectures 20-25 (PDF)
21 Cost approximation: discounted cost (PDF)
22 Projected equation methods (PDF)
23 More on projected equations: Q-learning (PDF)
24 Extensions to stochastic shortest path and average cost (PDF)
25 Gradient methods for approximation in policy space (PDF)
26 Project presentations I
27 Project presentations II