Efficient Planning under Uncertainty with Macro-actions
Name
He-2011-Efficient Planning under Uncertainty with Macro-actions.pdf
Size
1.42 MB
Format
Adobe PDF
Checksum (MD5)
90a4a07617949d9adbb618dbca21bf8c
Author(s)
He, Ruijie
Brunskill, Emma
Roy, Nicholas
Date Issued
February 2011
Journal
Journal of Artificial Intelligence Research
Publisher
AI Access Foundation
Citation
He, Ruijie, Emma Brunskill, and Nicholas Roy. "Efficient Planning under Uncertainty with Macro-actions." Journal of Artificial Intelligence Research 40 (2011) 523-570. © 2011 AI Access Foundation.
Version
Final published version
Abstract
Deciding how to act in partially observable environments remains an active area of research.
Identifying good sequences of decisions is particularly challenging when good control performance
requires planning multiple steps into the future in domains with many states. Towards addressing
this challenge, we present an online, forward-search algorithm called the Posterior Belief Distribution
(PBD). PBD leverages a novel method for calculating the posterior distribution over beliefs
that result after a sequence of actions is taken, given the set of observation sequences that could be
received during this process. This method allows us to efficiently evaluate the expected reward of a
sequence of primitive actions, which we refer to as macro-actions. We present a formal analysis of
our approach, and examine its performance on two very large simulation experiments: scientific exploration
and a target monitoring domain. We also demonstrate our algorithm being used to control
a real robotic helicopter in a target monitoring experiment, which suggests that our approach has
practical potential for planning in real-world, large partially observable domains where a multi-step
look ahead is required to achieve good performance.
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Massachusetts Institute of Technology. Department of Aeronautics and Astronautics
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1613/jair.3171
