This is an archived course. A more recent version may be available at ocw.mit.edu.
Lectures: 2 sessions / week, 1.5 hours / session
Recitations: 1 session / week, 1 hour / session
The course covers the basic models and solution techniques for problems of sequential decision making under uncertainty (stochastic control). We will consider optimal control of a dynamical system over both a finite and an infinite number of stages. This includes systems with finite or infinite state spaces, as well as perfectly or imperfectly observed systems. We will also discuss approximation methods for problems involving large state spaces. Applications of dynamic programming in a variety of fields will be covered in recitations.
The course will roughly follow this schedule:
Solid knowledge of undergraduate probability, at the level of 6.041, especially conditional distributions and expectations, and Markov chains. Mathematical maturity and the ability to write down precise and rigorous arguments are also important. A class in analysis (e.g., 18.100) will be helpful, although this prerequisite will not be strictly enforced.
Bertsekas, Dimitri P. Dynamic Programming and Optimal Control, Volume I. 3rd ed. Athena Scientific, 2005. ISBN: 9781886529267.
———. Dynamic Programming and Optimal Control, Volume II: Approximate Dynamic Programming. 4th ed. Athena Scientific, 2012. ISBN: 9781886529441.
The two volumes can also be purchased as a set. ISBN: 9781886529083.
Lectures on approximate dynamic programming will be based on the online chapter (
PDF - 2.1MB).
A term paper or project will be required, of one of the following types: (i) read some of the literature and provide a critical report, with suggestions for further work; (ii) formulate a new model, motivated by some application that interests you, and study it, analytically or computationally. More detailed instructions, together with pointers to the literature and possible topics, will be provided in due time.
There will be short project presentations during exam week. A fairly complete version of your paper needs to be handed in before the presentation.
There will be two quizzes, and 8–10 problem sets.
| ACTIVITIES | PERCENTAGES |
|---|---|
| Homework | 20% |
| Quizzes | 50% |
| Project | 30% |
You may interact with fellow students when preparing your homework solutions. However, at the end, you must write up solutions on your own. Duplicating a solution that someone else has written (verbatim or edited), or providing solutions for a fellow student to copy, is not acceptable. If you do collaborate on homework, you must cite in your written solution your collaborators. Also, if you use sources other than the textbook in one of your solutions, e.g., a "friendly expert" or another text, be sure to cite the source. There is no penalty for such collaboration or use of other sources, as long as it is disclosed.
However, while the use of other sources is allowed (with proper citation), you are not allowed to consult problem set solutions from previous semesters.
In general, we expect students to adhere to basic, common sense concepts of academic honesty. Presenting somebody else's work as if it were your own, or cheating in exams, is unacceptable.
Support for MIT OPENCOURSEWARE'S 15th anniversary is provided by 
MIT OpenCourseWare makes the materials used in the teaching of almost all of MIT's subjects available on the Web, free of charge. With more than 2,200 courses available, OCW is delivering on the promise of open sharing of knowledge. Learn more »
© 2001–2015
Massachusetts Institute of Technology
Your use of the MIT OpenCourseWare site and materials is subject to our Creative Commons License and other terms of use.
Subscribe to the OCW Newsletter
