Policy search approaches to reinforcement learning for quadruped locomotion
Name
79650848-MIT.pdf
Description
Full printable version
Size
4.54 MB
Format
Adobe PDF
Checksum (MD5)
db550e5cc2497c61612bf77997c404d3
Author(s)
Jimenez, Antonio R
Advisor(s)
Leslie P. Kaelbling and Paul A. DeBitetto.
Date Issued
2006
Publisher
Massachusetts Institute of Technology
Abstract
Legged locomotion is a challenging problem for machine learning to solve. A quadruped has 12 degrees of freedom which results in a large state space for the resulting Markov Decision Problem (DP). It is too difficult for computers to completely learn the state space, while it is too difficult for humans to fully understand the system dynamics and directly program the most efficient controller. This thesis combines these two approaches by integrating a model-based controller approach with reinforcement learning to develop an effective walk for a quadruped robot. We then evaluate different policy search approaches to reinforcement learning. To solve the Partially Observable Markov Decision Problem (POMIDP), a deterministic simulation is developed that generates a model which allows us to conduct a direct. policy search using dynamic programming. This is compared against using a, nondeterministic simulation to generate a model that evaluates policies. We show that using deterministic transitions to allow the use of dynamic programming has little impact on the performance of our system. Two local policy search approaches are implemented.
(cont.) A hill climbing algorithm is compared to a policy gradient algorithm to optimize parameters for the robot's model-based controller. The optimal machine-learned policy achieved a 155'% increase in performance over the hand-tuned policy. The baseline hill climbing algorithm is shown to outperform the policy gradient. algorithm with this particular gait.
Description
Thesis (M. Eng.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2006.
Includes bibliographical references (leaves 63-65).
Subjects
Electrical Engineering and Computer Science.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
M.I.T. theses are protected by copyright. They may be viewed from this source for any purpose, but reproduction or distribution in any format is prohibited without written permission. See provided URL for inquiries about permission.
Persistent DSpace Link
