Robust Adaptive Markov Decision Processes in Multi-vehicle Applications
Author(s)How, Jonathan P.; Bertuccelli, Luca F.; Bethke, Brett M.
DownloadBertuccelli-2009-Robust Adaptive Markov Decision Processes in Multi-vehicle Applications.pdf (470.1Kb)
MetadataShow full item record
This paper presents a new robust and adaptive framework for Markov decision processes that accounts for errors in the transition probabilities. Robust policies are typically found off-line, but can be extremely conservative when implemented in the real system. Adaptive policies, on the other hand, are specifically suited for on-line implementation, but may display undesirable transient performance as the model is updated though learning. A new method that exploits the individual strengths of the two approaches is presented in this paper. This robust and adaptive framework protects the adaptation process from exhibiting a worst-case performance during the model updating, and is shown to converge to the true, optimal value function in the limit of a large number of state transition observations. The proposed framework is investigated in simulation and actual flight experiments, and shown to improve transient behavior in the adaptation process and overall mission performance.
DepartmentMassachusetts Institute of Technology. Aerospace Controls Laboratory; Massachusetts Institute of Technology. Department of Aeronautics and Astronautics
American Control Conference, 2009. ACC '09
Institute of Electrical and Electronics Engineers
Bertuccelli, L.F., B. Bethke, and J.P. How. “Robust adaptive Markov Decision Processes in multi-vehicle applications.” American Control Conference, 2009. ACC '09. 2009. 1304-1309. ©2009 Institute of Electrical and Electronics Engineers.
Final published version
INSPEC Accession Number: 10775888