A Monte-Carlo AIXI Approximation
Name
Veness-2011-A Monte-Carlo AIXI Approximation.pdf
Size
668.25 KB
Format
Adobe PDF
Checksum (MD5)
8606c367d03c61a0b210fb0fe23ea3b6
Author(s)
Veness, Joel
Ng, Kee Siong
Hutter, Marcus
Uther, William
Silver, David
Date Issued
January 2011
Journal
Journal of Artificial Intelligence Research
Publisher
AI Access Foundation
Citation
Veness, Joel et al. (2011) "A Monte-Carlo AIXI Approximation", Journal of Artificial Intelligence Research (2011) Volume 40, pages 95-142. © 2011 AI Access Foundation
Version
Final published version
Abstract
This paper introduces a principled approach for the design of a scalable general reinforcement learning agent. Our approach is based on a direct approximation of AIXI, a Bayesian optimality notion for general reinforcement learning agents. Previously, it has been unclear whether the theory of AIXI could motivate the design of practical algorithms. We answer this hitherto open question in the affirmative, by providing the first computationally feasible approximation to the AIXI agent. To develop our approximation, we introduce a new Monte-Carlo Tree Search algorithm along with an agent-specific extension to the Context Tree Weighting algorithm. Empirically, we present a set of encouraging results on a variety of stochastic and partially observable domains. We conclude by proposing a number of directions for future research.
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1613/jair.3125
