2026-09-19T00:26:52Zhttps://dspace.mit.edu/server/oai/request

oai:dspace.mit.edu:1721.1/1400902022-02-08T03:35:38Zcom_1721.1_7582com_1721.1_7581col_1721.1_131023

How, Jonathan P. Abdulhai, Marwa Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science 2022-02-07T15:23:31Z 2022-02-07T15:23:31Z 2021-09 2021-11-03T19:25:27.883Z https://hdl.handle.net/1721.1/140090 Hierarchical reinforcement learning has focused on discovering temporally extended actions (options) to provide efficient solutions for long-horizon decision-making problems with sparse rewards. One promising approach that learns these options end-toend in this setting is the option-critic (OC) framework. However, there are several practical limitations of this method, including the lack of diversity between the learned sub-policies and sample inefficiency. This thesis shows that the OC framework does not decompose problems into smaller and largely independent components, but instead increases the problem complexity with each option by considering the entire state space during learning. To address this issue, we introduce state abstracted option-critic (SOC), a new framework that considers both temporal and state abstraction to effectively reduce the problem complexity in sparse reward settings. Our contribution includes learning a factored state space to enable each option to map to a sub-section of the state space. We test our method against hierarchical, nonhierarchical, and state abstraction baselines to demonstrate better sample efficiency and higher overall performance in both image and large vector-state representations under sparse reward settings. M.Eng. Massachusetts Institute of Technology In Copyright - Educational Use Permitted Copyright MIT http://rightsstatements.org/page/InC-EDU/1.0/ Factored State Abstraction for Option Learning Thesis application/pdf Master Master of Engineering in Electrical Engineering and Computer Science Publication unknown unknown <Publication xmlns="https://www.openaire.eu/cerif-profile/1.1/" id="80c38890-d5a9-45a8-ad1f-811434452048"> <Type xmlns="https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types">http://purl.org/coar/resource_type/c_1843</Type> <Title>Factored State Abstraction for Option Learning</Title> <PublishedIn> <Publication> </Publication> </PublishedIn> <PublicationDate>2021-09</PublicationDate> <Authors> <Author> <DisplayName>Abdulhai, Marwa</DisplayName> <Affiliation> <OrgUnit> </OrgUnit> </Affiliation> </Author> </Authors> <Editors> </Editors> <Publishers> <Publisher> <DisplayName>Massachusetts Institute of Technology</DisplayName> <OrgUnit /> </Publisher> </Publishers> <License>http://rightsstatements.org/page/InC-EDU/1.0/</License> <Abstract>Hierarchical reinforcement learning has focused on discovering temporally extended actions (options) to provide efficient solutions for long-horizon decision-making problems with sparse rewards. One promising approach that learns these options end-toend in this setting is the option-critic (OC) framework. However, there are several practical limitations of this method, including the lack of diversity between the learned sub-policies and sample inefficiency. This thesis shows that the OC framework does not decompose problems into smaller and largely independent components, but instead increases the problem complexity with each option by considering the entire state space during learning. To address this issue, we introduce state abstracted option-critic (SOC), a new framework that considers both temporal and state abstraction to effectively reduce the problem complexity in sparse reward settings. Our contribution includes learning a factored state space to enable each option to map to a sub-section of the state space. We test our method against hierarchical, nonhierarchical, and state abstraction baselines to demonstrate better sample efficiency and higher overall performance in both image and large vector-state representations under sparse reward settings.</Abstract> <Access xmlns="http://purl.org/coar/access_right" > </Access> </Publication>