Compositional Robot Learning for Generalizable Interactions
Name
Kuo-ylkuo-PhD-EECS-2022-thesis.pdf
Description
Thesis PDF
Size
3.48 MB
Format
Adobe PDF
Checksum (MD5)
bfa8ac459db34508d92318af004059bc
Author(s)
Kuo, Yen-Ling
Advisor(s)
Katz, Boris
Barbu, Andrei
Date Issued
May 2022
Publisher
Massachusetts Institute of Technology
Abstract
To understand environments effectively and to interact safely with humans, robots must generalize their learned models to scenarios they have never been trained on before, such as new commands and new agents. Humans have shown a remarkable ability to compose concepts they have learned before in order to interpret and to act in a novel environment. In contrast, many deep-learning based methods fail at compositional generalization, i.e., an ability to generalize to novel combinations of concepts that have not been seen before in training. This thesis presents several learning-based approaches that leverage compositionally to enable generalization in various reasoning skills such as language understanding and social interactions. First, we show how we can derive networks directly from compositional linguistic structures to enable robots to follow novel commands and act rationally in new scenarios. These networks are extended to encode temporal relationships in linear temporal logic and augmented with sampling-based planners to plan in continuous space. Then we show how we can formulate social interactions as reward operations and apply recursive reward estimation to enable robots to reason about novel social interactions. Finally, we explore an alternative approach to incorporate compositionality when there is no explicit compositional structure — using language as an intermediate representation for internal reasoning. We show how this linguistic representation can help robots predict the trajectories of other agents.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
In Copyright - Educational Use Permitted
Copyright MIT
Persistent DSpace Link
