Bounds for Markov Decision Processes
Name
bounds-2011.pdf
Size
491.13 KB
Format
Adobe PDF
Checksum (MD5)
19ec6d8ab8a2f39834346d65ad063024
Author(s)
Desai, Vijay V.
Farias, Vivek F.
Moallemi, Ciamac C.
Date Issued
2013
Journal
Reinforcement Learning and Approximate Dynamic Programming for Feedback Control
Publisher
John Wiley & Sons, Inc.
Citation
Desai, Vijay V., Vivek F. Farias, and Ciamac C. Moallemi. “Bounds for Markov Decision Processes.” Reinforcement Learning and Approximate Dynamic Programming for Feedback Control, edited by Frank L. Lewis and Derong Liu, John Wiley & Sons, Inc., 2013, pp. 452–473.
Version
Original manuscript
Abstract
We consider the problem of producing lower bounds on the optimal cost-to-go function of a Markov decision problem. We present two approaches to this problem: one based on the methodology of approximate linear programming (ALP) and another based on the so-called martingale duality approach. We show that these two approaches are intimately connected. Exploring this connection leads us to the problem of finding "optimal" martingale penalties within the martingale duality approach which we dub the pathwise optimization (PO) problem. We show interesting cases where the PO problem admits a tractable solution and establish that these solutions produce tighter approximations than the ALP approach. © 2013 The Institute of Electrical and Electronics Engineers, Inc.
MIT Department
Sloan School of Management
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1002/9781118453988.ch20
