DetH*: Approximate Hierarchical Solution of Large Markov Decision Processes
Author(s)Barry, Jennifer; Kaelbling, Leslie P.; Lozano-Perez, Tomas
MetadataShow full item record
This paper presents an algorithm for finding approximately optimal policies in very large Markov decision processes by constructing a hierarchical model and then solving it approximately. It exploits factored representations to achieve compactness and efficiency and to discover connectivity properties of the domain. We provide a bound on the quality of the solutions and give asymptotic analysis of the runtimes; in addition we demonstrate performance on a collection of very large domains. Results show that the quality of resulting policies is very good and the total running times, for both creating and solving the hierarchy, are significantly less than for an optimal factored MDP solver.
DepartmentMassachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory; Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Proceedings of the 22nd 2011 International Joint Conference on Artificial Intelligence
AAAI Press/International Joint Conferences on Artificial Intelligence
Barry, Jennifer, Leslie Pack Kaelbling, and Tomás Lozano-Pérez. "DetH*: Approximate Hierarchical Solution of Large Markov Decision Processes. In 22nd 2011 International Joint Conference on Artificial Intelligence, IJCAI-11, Barcelona, Catalonia, Spain, 16–22 July 2011. AAAI Press, (2011): p.1928-1935.
Author's final manuscript