Adaptive Planning for Markov Decision Processes with Uncertain Transition Models via Incremental Feature Dependency Discovery
Name
How_Adaptive planning.pdf
Size
623.46 KB
Format
Adobe PDF
Checksum (MD5)
c414e692b150284792b05c105ed0a223
Author(s)
Geramifard, Alborz
Chowdhary, Girish
How, Jonathan P.
Ure, Nazim Kemal
Date Issued
September 2012
Journal
Machine Learning and Knowledge Discovery in Databases
Publisher
Springer-Verlag
Citation
Ure, N.Kemal et al. “Adaptive Planning for Markov Decision Processes with Uncertain Transition Models via Incremental Feature Dependency Discovery.” Machine Learning and Knowledge Discovery in Databases. Ed. PeterA. Flach, Tijl Bie, and Nello Cristianini. Vol. 7524. Springer Berlin Heidelberg, 2012. 99–115. Lecture Notes in Computer Science.
Version
Author's final manuscript
Abstract
Solving large scale sequential decision making problems without prior knowledge of the state transition model is a key problem in the planning literature. One approach to tackle this problem is to learn the state transition model online using limited observed measurements. We present an adaptive function approximator (incremental Feature Dependency Discovery (iFDD)) that grows the set of features online to approximately represent the transition model. The approach leverages existing feature-dependencies to build a sparse representation of the state transition model. Theoretical analysis and numerical simulations in domains with state space sizes varying from thousands to millions are used to illustrate the benefit of using iFDD for incrementally building transition models in a planning framework.
MIT Department
Massachusetts Institute of Technology. Department of Aeronautics and Astronautics
Massachusetts Institute of Technology. Laboratory for Information and Decision Systems
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike 3.0
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1007/978-3-642-33486-3_7
