Feedback controller parameterizations for reinforcement learning
Name
Tedrake_Feedback controller.pdf
Size
226.05 KB
Format
Adobe PDF
Checksum (MD5)
b40d4c02eddbc826745fdda3c972a6f2
Author(s)
Roberts, John William
Manchester, Ian R.
Tedrake, Russell Louis
Date Issued
April 2011
Journal
Proceedings of the 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL)
Publisher
Institute of Electrical and Electronics Engineers
Citation
Roberts, John William et al. "Feedback controller parameterizations for reinforcement learning." Forthcoming in Proceedings of the 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL)
Version
Author's final manuscript
Abstract
Reinforcement Learning offers a very general framework for learning controllers, but its effectiveness is closely tied to the controller parameterization used. Especially when learning feedback controllers for weakly stable systems, ineffective parameterizations can result in unstable controllers and poor performance both in terms of learning convergence and in the cost of the resulting policy. In this paper we explore four linear controller parameterizations in the context of REINFORCE, applying them to the control of a reaching task with a linearized flexible manipulator. We find that some natural but naive parameterizations perform very poorly, while the Youla Parameterization (a popular parameterization from the controls literature) offers a number of robustness and performance advantages.
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Massachusetts Institute of Technology. Department of Mechanical Engineering
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike 3.0
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1109/ADPRL.2011.5967370
