Loss bounds for uncertain transition probabilities in Markov decision processes
Name
Jaillet_Loss bounds.pdf
Size
420.5 KB
Format
Adobe PDF
Checksum (MD5)
f1a5dbed1605a73ceac5564302da3e06
Author(s)
Jaillet, Patrick
Mastin, Dana Andrew
Date Issued
December 2012
Journal
Proceedings of the 2012 IEEE 51st IEEE Conference on Decision and Control (CDC)
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Citation
Mastin, Andrew, and Patrick Jaillet. “Loss Bounds for Uncertain Transition Probabilities in Markov Decision Processes.” 2012 IEEE 51st IEEE Conference on Decision and Control (CDC) (n.d.).
Version
Author's final manuscript
Abstract
We analyze losses resulting from uncertain transition probabilities in Markov decision processes with bounded nonnegative rewards. We assume that policies are precomputed using exact dynamic programming with the estimated transition probabilities, but the system evolves according to different, true transition probabilities. Given a bound on the total variation error of estimated transition probability distributions, we derive upper bounds on the loss of expected total reward. The approach analyzes the growth of errors incurred by stepping backwards in time while precomputing value functions, which requires bounding a multilinear program. Loss bounds are given for the finite horizon undiscounted, finite horizon discounted, and infinite horizon discounted cases, and a tight example is shown.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Massachusetts Institute of Technology. Laboratory for Information and Decision Systems
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1109/CDC.2012.6426504
