Compositional Policy Priors
Name
MIT-CSAIL-TR-2013-007.pdf
Size
577.91 KB
Format
Adobe PDF
Checksum (MD5)
034eaa017707e7e3bb3aedbc25130a5b
Author(s)
Wingate, David
Diuk, Carlos
O'Donnell, Timothy
Tenenbaum, Joshua
Gershman, Samuel
Advisor(s)
Joshua Tenenbaum
Date Issued
April 12, 2013
Series/Report no.
MIT-CSAIL-TR-2013-007
Abstract
This paper describes a probabilistic framework for incorporating structured inductive biases into reinforcement learning. These inductive biases arise from policy priors, probability distributions over optimal policies. Borrowing recent ideas from computational linguistics and Bayesian nonparametrics, we define several families of policy priors that express compositional, abstract structure in a domain. Compositionality is expressed using probabilistic context-free grammars, enabling a compact representation of hierarchically organized sub-tasks. Useful sequences of sub-tasks can be cached and reused by extending the grammars nonparametrically using Fragment Grammars. We present Monte Carlo methods for performing inference, and show how structured policy priors lead to substantially faster learning in complex domains compared to methods without inductive biases.
Persistent DSpace Link
