2026-09-18T21:58:19Zhttps://dspace.mit.edu/server/oai/requestoai:dspace.mit.edu:1721.1/1474352023-01-20T03:37:06Zcom_1721.1_7582com_1721.1_7581col_1721.1_131023
Kim, Sangbae
Ackerman, Liam J.
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
2023-01-19T19:50:12Z
2023-01-19T19:50:12Z
2022-09
2022-09-16T20:23:57.251Z
https://hdl.handle.net/1721.1/147435
Deep reinforcement learning has been used to craft robust and performant control policies for legged robotics. However, the engineering processes to create these policies are often plagued by long training times that slow down engineering iteration. This thesis suggests that model-based controllers offer a wealth of successful computation that may be used within reinforcement learning control pipelines to improve learning efficiency. Two ideas incorporate this engineering expertise to increase reinforcement learning efficiency. First, successful model-based computations are pre-processed and incorporated directly into network observations. Introducing these terms into the reinforcement learning architecture is shown to increase learning speeds and policy performance dramatically. Next, inspired by model-based task hierarchies, more structure is added to the reinforcement learning objective function to activate and deactivate reward terms based on an agent’s state. This structure is intended to avoid local minima which impede learning. This reward restructure is shown to avoid local minima during training but degrades final policy performance at edge-cases.
M.Eng.
Massachusetts Institute of Technology
In Copyright - Educational Use Permitted
Copyright MIT
http://rightsstatements.org/page/InC-EDU/1.0/
Leveraging Engineering Expertise in Deep Reinforcement Learning
Thesis
application/pdf
Master
Master of Engineering in Electrical Engineering and Computer Science
Publication
unknown
unknown
<Publication xmlns="https://www.openaire.eu/cerif-profile/1.1/" id="6dc4d3cf-f037-45cd-9eee-b06dc763f6d0">
<Type xmlns="https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types">http://purl.org/coar/resource_type/c_1843</Type>
<Title>Leveraging Engineering Expertise in Deep Reinforcement Learning</Title>
<PublishedIn>
<Publication>
</Publication>
</PublishedIn>
<PublicationDate>2022-09</PublicationDate>
<Authors>
<Author>
<DisplayName>Ackerman, Liam J.</DisplayName>
<Affiliation>
<OrgUnit>
</OrgUnit>
</Affiliation>
</Author>
</Authors>
<Editors>
</Editors>
<Publishers>
<Publisher>
<DisplayName>Massachusetts Institute of Technology</DisplayName>
<OrgUnit />
</Publisher>
</Publishers>
<License>http://rightsstatements.org/page/InC-EDU/1.0/</License>
<Abstract>Deep reinforcement learning has been used to craft robust and performant control policies for legged robotics. However, the engineering processes to create these policies are often plagued by long training times that slow down engineering iteration. This thesis suggests that model-based controllers offer a wealth of successful computation that may be used within reinforcement learning control pipelines to improve learning efficiency. Two ideas incorporate this engineering expertise to increase reinforcement learning efficiency. First, successful model-based computations are pre-processed and incorporated directly into network observations. Introducing these terms into the reinforcement learning architecture is shown to increase learning speeds and policy performance dramatically. Next, inspired by model-based task hierarchies, more structure is added to the reinforcement learning objective function to activate and deactivate reward terms based on an agent’s state. This structure is intended to avoid local minima which impede learning. This reward restructure is shown to avoid local minima during training but degrades final policy performance at edge-cases.</Abstract>
<Access xmlns="http://purl.org/coar/access_right"
>
</Access>
</Publication>