Robust Adaptive Markov Decision Processes in Multi-vehicle Applications
Name
Bertuccelli-2009-Robust Adaptive Markov Decision Processes in Multi-vehicle Applications.pdf
Size
470.2 KB
Format
Adobe PDF
Checksum (MD5)
ebd5149d5001ce71b8a15857bd5113c7
Author(s)
How, Jonathan P.
Bertuccelli, Luca F.
Bethke, Brett M.
Date Issued
July 2009
Journal
American Control Conference, 2009. ACC '09
Publisher
Institute of Electrical and Electronics Engineers
Citation
Bertuccelli, L.F., B. Bethke, and J.P. How. “Robust adaptive Markov Decision Processes in multi-vehicle applications.” American Control Conference, 2009. ACC '09. 2009. 1304-1309. ©2009 Institute of Electrical and Electronics Engineers.
Version
Final published version
Abstract
This paper presents a new robust and adaptive framework for Markov decision processes that accounts for errors in the transition probabilities. Robust policies are typically found off-line, but can be extremely conservative when implemented in the real system. Adaptive policies, on the other hand, are specifically suited for on-line implementation, but may display undesirable transient performance as the model is updated though learning. A new method that exploits the individual strengths of the two approaches is presented in this paper. This robust and adaptive framework protects the adaptation process from exhibiting a worst-case performance during the model updating, and is shown to converge to the true, optimal value function in the limit of a large number of state transition observations. The proposed framework is investigated in simulation and actual flight experiments, and shown to improve transient behavior in the adaptation process and overall mission performance.
MIT Department
Massachusetts Institute of Technology. Aerospace Controls Laboratory
Massachusetts Institute of Technology. Department of Aeronautics and Astronautics
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1109/ACC.2009.5160511
