This is an archived course. A more recent version may be available at ocw.mit.edu.

Help   |   Contact Us

 

Calendar

In the table below, some topics are given along with sections in the course textbook:

Amazon logo Bertsekas, Dimitri. Dynamic Programming and Optimal Control. Vol. I and II. 3rd ed. Nashua, NH: Athena Scientific, 2007. ISBN: 9781886529083.

SES # TOPICS TEXTBOOK SECTIONS KEY DATES
1 Introduction to dynamic programming; examples and formulation
2 The dynamic programming algorithm Problem set 1 out
3 Deterministic systems and the shortest path problem
4 Shortest path algorithms

Problem set 1 due

Problem set 2 out
5 Deterministic continuous-time optimal control
6 Stopping and scheduling problems

Problem set 2 due

Problem set 3 out
7 Linear systems with quadratic costs and inventory control
8 Problems with imperfect state information

Problem set 3 due

Problem set 4 out
9 Sufficient statistics
10 Suboptimal control

Problem set 4 due

Problem set 5 out
11 Rollout algorithms
12 More on suboptimal control
13 Infinite horizon I: stochastic shortest path problems Sections 7.1-7.2, vol. I
14 Infinite horizon II: discounted problems Section 7.3, vol. I

Problem set 5 due

Problem set 6 out
15 Infinite horizon III: average cost problems Section 7.4, vol. I
16 Semi-Markov problems Section 7.5, vol. I

Problem set 6 due

Problem set 7 out
17 Infinite horizon: discounted problems I Sections 1.1-1.3, vol. II
18 Infinite horizon: discounted problems II Sections 1.3-1.4, vol. II Problem set 7 due
Midterm
19 Stochastic shortest path problems Chapter 2, vol. II
20 Overview of main approaches in approximate dynamic programming Problem set 8 out
21 Cost approximation: discounted cost Section 6.2, vol. II
22 Projected equation methods Section 6.3, vol. II and references

Problem set 8 due

Problem set 9 out
23 More on projected equations: Q-learning Sections 6.3-6.4, vol. II and references
24 Extensions to stochastic shortest path and average cost Sections 6.5-6.6, vol. II and references Problem set 9 due
25 Gradient methods for approximation in policy space Section 6.7, vol. II and references
26 Project presentations I
27 Project presentations II