Basis Function Adaptation Methods for Cost Approximation in MDP
Name
Bertsekas_Basis function.pdf
Size
440.48 KB
Format
Adobe PDF
Checksum (MD5)
87e817aa40da814a764c47705a1d8592
Author(s)
Yu, Huizhen
Bertsekas, Dimitri P.
Date Issued
May 2009
Journal
IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, 2009. ADPRL '09
Publisher
Institute of Electrical and Electronics Engineers
Citation
Huizhen Yu, and D.P. Bertsekas. “Basis function adaptation methods for cost approximation in MDP.” Adaptive Dynamic Programming and Reinforcement Learning, 2009. ADPRL '09. IEEE Symposium on. 2009. 74-81. ©2009 Institute of Electrical and Electronics Engineers.
Version
Final published version
Abstract
We generalize a basis adaptation method for cost approximation in Markov decision processes (MDP), extending earlier work of Menache, Mannor, and Shimkin. In our context, basis functions are parametrized and their parameters are tuned by minimizing an objective function involving the cost function approximation obtained when a temporal differences (TD) or other method is used. The adaptation scheme involves only low order calculations and can be implemented in a way analogous to policy gradient methods. In the generalized basis adaptation framework we provide extensions to TD methods for nonlinear optimal stopping problems and to alternative cost approximations beyond those based on TD.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Massachusetts Institute of Technology. Laboratory for Information and Decision Systems
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1109/ADPRL.2009.4927528
