2026-09-19T00:38:49Zhttps://dspace.mit.edu/server/oai/request

oai:dspace.mit.edu:1721.1/1446692022-08-30T03:28:55Zcom_1721.1_7582com_1721.1_7581col_1721.1_131022

Katz, Boris Barbu, Andrei Kuo, Yen-Ling Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science 2022-08-29T16:03:30Z 2022-08-29T16:03:30Z 2022-05 2022-06-21T19:15:39.613Z https://hdl.handle.net/1721.1/144669 https://orcid.org/0000-0002-6433-6713 To understand environments effectively and to interact safely with humans, robots must generalize their learned models to scenarios they have never been trained on before, such as new commands and new agents. Humans have shown a remarkable ability to compose concepts they have learned before in order to interpret and to act in a novel environment. In contrast, many deep-learning based methods fail at compositional generalization, i.e., an ability to generalize to novel combinations of concepts that have not been seen before in training. This thesis presents several learning-based approaches that leverage compositionally to enable generalization in various reasoning skills such as language understanding and social interactions. First, we show how we can derive networks directly from compositional linguistic structures to enable robots to follow novel commands and act rationally in new scenarios. These networks are extended to encode temporal relationships in linear temporal logic and augmented with sampling-based planners to plan in continuous space. Then we show how we can formulate social interactions as reward operations and apply recursive reward estimation to enable robots to reason about novel social interactions. Finally, we explore an alternative approach to incorporate compositionality when there is no explicit compositional structure — using language as an intermediate representation for internal reasoning. We show how this linguistic representation can help robots predict the trajectories of other agents. Ph.D. Massachusetts Institute of Technology In Copyright - Educational Use Permitted Copyright MIT http://rightsstatements.org/page/InC-EDU/1.0/ Compositional Robot Learning for Generalizable Interactions Thesis application/pdf Doctoral Doctor of Philosophy Publication unknown unknown <Publication xmlns="https://www.openaire.eu/cerif-profile/1.1/" id="99de5eb7-f93e-4923-9237-9f6c9049cce4"> <Type xmlns="https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types">http://purl.org/coar/resource_type/c_1843</Type> <Title>Compositional Robot Learning for Generalizable Interactions</Title> <PublishedIn> <Publication> </Publication> </PublishedIn> <PublicationDate>2022-05</PublicationDate> <Authors> <Author> <DisplayName>Kuo, Yen-Ling</DisplayName> <Affiliation> <OrgUnit> </OrgUnit> </Affiliation> </Author> </Authors> <Editors> </Editors> <Publishers> <Publisher> <DisplayName>Massachusetts Institute of Technology</DisplayName> <OrgUnit /> </Publisher> </Publishers> <License>http://rightsstatements.org/page/InC-EDU/1.0/</License> <Abstract>To understand environments effectively and to interact safely with humans, robots must generalize their learned models to scenarios they have never been trained on before, such as new commands and new agents. Humans have shown a remarkable ability to compose concepts they have learned before in order to interpret and to act in a novel environment. In contrast, many deep-learning based methods fail at compositional generalization, i.e., an ability to generalize to novel combinations of concepts that have not been seen before in training. This thesis presents several learning-based approaches that leverage compositionally to enable generalization in various reasoning skills such as language understanding and social interactions. First, we show how we can derive networks directly from compositional linguistic structures to enable robots to follow novel commands and act rationally in new scenarios. These networks are extended to encode temporal relationships in linear temporal logic and augmented with sampling-based planners to plan in continuous space. Then we show how we can formulate social interactions as reward operations and apply recursive reward estimation to enable robots to reason about novel social interactions. Finally, we explore an alternative approach to incorporate compositionality when there is no explicit compositional structure — using language as an intermediate representation for internal reasoning. We show how this linguistic representation can help robots predict the trajectories of other agents.</Abstract> <Access xmlns="http://purl.org/coar/access_right" > </Access> </Publication>