Bayesian Nonparametric Methods for Partially-Observable Reinforcement Learning
Author(s)Doshi-Velez, Finale P.; Pfau, David; Wood, Frank; Roy, Nicholas
MetadataShow full item record
Making intelligent decisions from incomplete information is critical in many applications: for example, robots must choose actions based on imperfect sensors, and speech-based interfaces must infer a user’s needs from noisy microphone inputs. What makes these tasks hard is that often we do not have a natural representation with which to model the domain and use for choosing actions; we must learn about the domain’s properties while simultaneously performing the task. Learning a representation also involves trade-offs between modeling the data that we have seen previously and being able to make predictions about new data. This article explores learning representations of stochastic systems using Bayesian nonparametric statistics. Bayesian nonparametric methods allow the sophistication of a representation to scale gracefully with the complexity in the data. Our main contribution is a careful empirical evaluation of how representations learned using Bayesian nonparametric methods compare to other standard learning approaches, especially in support of planning and control. We show that the Bayesian aspects of the methods result in achieving state-of-the-art performance in decision making with relatively few samples, while the nonparametric aspects often result in fewer computations. These results hold across a variety of different techniques for choosing actions given a representation.
DepartmentMassachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory; Massachusetts Institute of Technology. Department of Aeronautics and Astronautics
IEEE Transactions on Pattern Analysis and Machine Intelligence
Institute of Electrical and Electronics Engineers (IEEE)
Doshi-Velez, Finale, David Pfau, Frank Wood, and Nicholas Roy. “Bayesian Nonparametric Methods for Partially-Observable Reinforcement Learning.” IEEE Trans. Pattern Anal. Mach. Intell. 37, no. 2 (February 2015): 394–407.