Towards feature selection in actor-critic algorithms
Name
Tedrake_Feature selection.pdf
Size
183.06 KB
Format
Adobe PDF
Checksum (MD5)
f4655b271f75f642cf65ccd508ccafca
Author(s)
Rohanimanesh, Khashayar
Roy, Nicholas
Tedrake, Russell Louis
Date Issued
June 2009
Journal
Proceedings of Workshop on Abstraction in Reinforcement Learning, Joint workshop at ICML, UAI, and COLT 2009
Citation
Rohanimanesh, Khashayar, Nicholas Roy and Russ Tedrake. "Towards feature selection in actor-critic algorithms." in Proceedings of the ICML/UAI/COLT Workshop on Abstraction in Reinforcement Learning, Montreal, Canada, 2009.
Version
Author's final manuscript
Abstract
Choosing features for the critic in actor-critic algorithms
with function approximation is known
to be a challenge. Too few critic features can
lead to degeneracy of the actor gradient, and too
many features may lead to slower convergence
of the learner. In this paper, we show that a wellstudied
class of actor policies satisfy the known
requirements for convergence when the actor features
are selected carefully. We demonstrate that
two popular representations for value methods -
the barycentric interpolators and the graph Laplacian
proto-value functions - can be used to represent
the actor in order to satisfy these conditions.
A consequence of this work is a generalization
of the proto-value function methods to the continuous
action actor-critic domain. Finally, we
analyze the performance of this approach using
a simulation of a torque-limited inverted pendulum.
Description
URL to paper listed on conference page
MIT Department
Massachusetts Institute of Technology. Department of Aeronautics and Astronautics
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike 3.0
Persistent DSpace Link
DOI of Published Version
http://www-all.cs.umass.edu/~gdk/arl/papers.html
