Active Learning in Persistent Surveillance UAV Missions
Name
How_Active learning.pdf
Size
342.29 KB
Format
Adobe PDF
Checksum (MD5)
c2070173ab3eb2f753782ede765d3810
Author(s)
Redding, Joshua
Bethke, Brett M.
Bertuccelli, Luca F.
How, Jonathan P.
Date Issued
April 2009
Journal
Proceedings of the AIAA Infotech@Aerospace Conference and AIAA Unmanned...Unlimited Conference
Publisher
American Institute of Aeronautics and Astronautics
Citation
Redding, Joshua, Brett Bethke, Luca Bertuccelli, and Jonathan How. “Active Learning in Persistent Surveillance UAV Missions.” In AIAA Infotech@Aerospace Conference and AIAA Unmanned...Unlimited Conference. American Institute of Aeronautics and Astronautics, 2009.
Version
Author's final manuscript
Abstract
The performance of many complex UAV decision-making problems can be extremely sensitive to small errors in the model parameters. One way of mitigating this sensitivity is by designing algorithms that more effectively learn the model throughout the course of a mission. This paper addresses this important problem by considering model uncertainty in a multi-agent Markov Decision Process (MDP) and using an active learning approach to quickly learn transition model parameters. We build on previous research that allowed UAVs to passively update model parameter estimates by incorporating new state transition observations. In this work, however, the UAVs choose to actively reduce the uncertainty in their model parameters by taking exploratory and informative actions. These actions result in a faster adaptation and, by explicitly accounting for UAV fuel dynamics, also mitigates the risk of the exploration. This paper compares the nominal, passive learning approach against two methods for incorporating active learning into the MDP framework: (1) All state transitions are rewarded equally, and (2) State transition rewards are weighted according to the expected resulting reduction in the variance of the model parameter. In both cases, agent behaviors emerge that enable faster convergence of the uncertain model parameters to their true values.
MIT Department
Massachusetts Institute of Technology. Aerospace Controls Laboratory
Massachusetts Institute of Technology. Department of Aeronautics and Astronautics
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike 3.0
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.2514/6.2009-1981
