Regret Based Robust Solutions for Uncertain Markov Decision Processes
Name
Jaillet_Regret based.pdf
Size
345.49 KB
Format
Adobe PDF
Checksum (MD5)
f16f64f3f1e73cb433911a8654dd7b42
Author(s)
Ahmed, Asrar
Varakantham, Pradeep
Adulyasak, Yossiri
Jaillet, Patrick
Date Issued
2013
Journal
Advances in Neural Information Processing Systems (NIPS)
Publisher
Neural Information Processing Systems
Citation
Ahmed, Asrar, Pradeep Varakantham, Yossiri Adulyasak, and Patrick Jaillet. "Regret Based Robust Solutions for Uncertain Markov Decision Processes." Advances in Neural Information Processing Systems 26 (NIPS 2013).
Version
Final published version
Abstract
In this paper, we seek robust policies for uncertain Markov Decision Processes (MDPs). Most robust optimization approaches for these problems have focussed on the computation of {\em maximin} policies which maximize the value corresponding to the worst realization of the uncertainty. Recent work has proposed {\em minimax} regret as a suitable alternative to the {\em maximin} objective for robust optimization. However, existing algorithms for handling {\em minimax} regret are restricted to models with uncertainty over rewards only. We provide algorithms that employ sampling to improve across multiple dimensions: (a) Handle uncertainties over both transition and reward models; (b) Dependence of model uncertainties across state, action pairs and decision epochs; (c) Scalability and quality bounds. Finally, to demonstrate the empirical effectiveness of our sampling approaches, we provide comparisons against benchmark algorithms on two domains from literature. We also provide a Sample Average Approximation (SAA) analysis to compute a posteriori error bounds.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
http://papers.nips.cc/paper/4970-regret-based-robust-solutions-for-uncertain-markov-decision-processes
