Monte-Carlo planning in large POMDPs
Name
Silver_Monte-carlo.pdf
Size
366.76 KB
Format
Adobe PDF
Checksum (MD5)
837fb8a4b5e86055dfb8f159e0c8db66
Author(s)
Silver, David
Veness, Joel
Date Issued
2010
Journal
Advances in Neural Information Processing Systems (NIPS)
Publisher
Neural Information Processing Systems
Citation
Silver, David, and Joel Veness. "Monte-Carlo planning in large POMDPs." Advances in Neural Information Processing Systems 23 (NIPS) (2010).
Version
Author's final manuscript
Abstract
This paper introduces a Monte-Carlo algorithm for online planning in large POMDPs. The algorithm combines a Monte-Carlo update of the agent's belief state with a Monte-Carlo tree search from the current belief state. The new algorithm, POMCP, has two important properties. First, Monte-Carlo sampling is used to break the curse of dimensionality both during belief state updates and during planning. Second, only a black box simulator of the POMDP is required, rather than explicit probability distributions. These properties enable POMCP to plan effectively in significantly larger POMDPs than has previously been possible. We demonstrate its effectiveness in three large POMDPs. We scale up a well-known benchmark problem, Rocksample, by several orders of magnitude. We also introduce two challenging new POMDPs: 10x10 Battleship and Partially Observable PacMan, with approximately 10^18 and 10^56 states respectively. Our Monte-Carlo planning algorithm achieved a high level of performance with no prior knowledge, and was also able to exploit simple domain knowledge to achieve better results with less search. POMCP is the first general purpose planner to achieve high performance in such large and unfactored POMDPs.
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
http://papers.nips.cc/book/advances-in-neural-information-processing-systems-23-2010
