2026-09-19T17:37:56Zhttps://dspace.mit.edu/server/oai/requestoai:dspace.mit.edu:1721.1/367982022-01-13T07:54:29Zcom_1721.1_7582com_1721.1_7581col_1721.1_131023
Leslie P. Kaelbling and Paul A. DeBitetto.
Jimenez, Antonio R
Massachusetts Institute of Technology. Dept. of Electrical Engineering and Computer Science.
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
2007-03-12T17:54:43Z
2007-03-12T17:54:43Z
2006
2006
http://hdl.handle.net/1721.1/36798
79650848
Thesis (M. Eng.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2006.
Includes bibliographical references (leaves 63-65).
Legged locomotion is a challenging problem for machine learning to solve. A quadruped has 12 degrees of freedom which results in a large state space for the resulting Markov Decision Problem (DP). It is too difficult for computers to completely learn the state space, while it is too difficult for humans to fully understand the system dynamics and directly program the most efficient controller. This thesis combines these two approaches by integrating a model-based controller approach with reinforcement learning to develop an effective walk for a quadruped robot. We then evaluate different policy search approaches to reinforcement learning. To solve the Partially Observable Markov Decision Problem (POMIDP), a deterministic simulation is developed that generates a model which allows us to conduct a direct. policy search using dynamic programming. This is compared against using a, nondeterministic simulation to generate a model that evaluates policies. We show that using deterministic transitions to allow the use of dynamic programming has little impact on the performance of our system. Two local policy search approaches are implemented.
(cont.) A hill climbing algorithm is compared to a policy gradient algorithm to optimize parameters for the robot's model-based controller. The optimal machine-learned policy achieved a 155'% increase in performance over the hand-tuned policy. The baseline hill climbing algorithm is shown to outperform the policy gradient. algorithm with this particular gait.
by Antonio R. Jimenez.
M.Eng.
65 leaves
eng
Massachusetts Institute of Technology
M.I.T. theses are protected by copyright. They may be viewed from this source for any purpose, but reproduction or distribution in any format is prohibited without written permission. See provided URL for inquiries about permission.
http://dspace.mit.edu/handle/1721.1/7582
Electrical Engineering and Computer Science.
Policy search approaches to reinforcement learning for quadruped locomotion
Thesis
application/pdf
false
Publication
unknown
unknown
<Publication xmlns="https://www.openaire.eu/cerif-profile/1.1/" id="a1482f71-47fb-4335-bbf9-1dbe193be379">
<Type xmlns="https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types">http://purl.org/coar/resource_type/c_1843</Type>
<Language>eng</Language>
<Title>Policy search approaches to reinforcement learning for quadruped locomotion</Title>
<PublishedIn>
<Publication>
</Publication>
</PublishedIn>
<PublicationDate>2006</PublicationDate>
<Authors>
<Author>
<DisplayName>Jimenez, Antonio R</DisplayName>
<Affiliation>
<OrgUnit>
</OrgUnit>
</Affiliation>
</Author>
</Authors>
<Editors>
</Editors>
<Publishers>
<Publisher>
<DisplayName>Massachusetts Institute of Technology</DisplayName>
<OrgUnit />
</Publisher>
</Publishers>
<License>http://dspace.mit.edu/handle/1721.1/7582</License>
<Keyword>Electrical Engineering and Computer Science.</Keyword>
<Abstract>Legged locomotion is a challenging problem for machine learning to solve. A quadruped has 12 degrees of freedom which results in a large state space for the resulting Markov Decision Problem (DP). It is too difficult for computers to completely learn the state space, while it is too difficult for humans to fully understand the system dynamics and directly program the most efficient controller. This thesis combines these two approaches by integrating a model-based controller approach with reinforcement learning to develop an effective walk for a quadruped robot. We then evaluate different policy search approaches to reinforcement learning. To solve the Partially Observable Markov Decision Problem (POMIDP), a deterministic simulation is developed that generates a model which allows us to conduct a direct. policy search using dynamic programming. This is compared against using a, nondeterministic simulation to generate a model that evaluates policies. We show that using deterministic transitions to allow the use of dynamic programming has little impact on the performance of our system. Two local policy search approaches are implemented.</Abstract>
<Abstract>(cont.) A hill climbing algorithm is compared to a policy gradient algorithm to optimize parameters for the robot's model-based controller. The optimal machine-learned policy achieved a 155'% increase in performance over the hand-tuned policy. The baseline hill climbing algorithm is shown to outperform the policy gradient. algorithm with this particular gait.</Abstract>
<Access xmlns="http://purl.org/coar/access_right"
>
</Access>
</Publication>