A unified framework for temporal difference methods
Name
Bertsekas-2009-A unified framework for temporal difference methods.pdf
Size
362.04 KB
Format
Adobe PDF
Checksum (MD5)
6dd977ed5aecb084bef3fc4217ba05d3
Author(s)
Bertsekas, Dimitri P.
Date Issued
March 2009
Journal
IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning
Publisher
Institute of Electrical and Electronics Engineers
Citation
Bertsekas, D.P. “A unified framework for temporal difference methods.” Adaptive Dynamic Programming and Reinforcement Learning, 2009. ADPRL '09. IEEE Symposium on. 2009. 1-7. © 2009, IEEE
Version
Final published version
Abstract
We propose a unified framework for a broad class of methods to solve projected equations that approximate the solution of a high-dimensional fixed point problem within a subspace S spanned by a small number of basis functions or features. These methods originated in approximate dynamic programming (DP), where they are collectively known as temporal difference (TD) methods. Our framework is based on a connection with projection methods for monotone variational inequalities, which involve alternative representations of the subspace S (feature scaling). Our methods admit simulation-based implementations, and even when specialized to DP problems, include extensions/new versions of the standard TD algorithms, which offer some special implementation advantages and reduced overhead.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Massachusetts Institute of Technology. Laboratory for Information and Decision Systems
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1109/ADPRL.2009.4927518
