Learning, Reasoning, and Planning with Relational and Temporal Neural Networks
Name
Mao-jiayuanm-SM-EECS-2021-thesis.pdf
Description
Thesis PDF
Size
3.78 MB
Format
Adobe PDF
Checksum (MD5)
4dc3fc9e9a9ec0532a9336ddc008b650
Author(s)
Mao, Jiayuan
Advisor(s)
Kaelbling, Leslie Pack
Tenenbaum, Joshua B.
Date Issued
September 2021
Publisher
Massachusetts Institute of Technology
Abstract
Every day, people interpret events and actions in terms of concepts, defined over evolving relations among agents and objects and their goals. We learn these concepts from a limited amount of data, generalizing directly over different numbers and arrangements of agents and objects, and detailed timings of trajectories. We also effectively recompose these concepts to describe unseen behaviors from other agents, and leverage the causal relationships among actions to make plans for ourselves.
This thesis gives an overview of a neuro-symbolic framework for learning, reasoning, and planning with relational and temporal neural networks. The key idea is to exploit a structural bias in neural network learning that enables us to describe complex relational-temporal events and actions. These structures form a minimal amount of prior knowledge but are generic and crucial: scenes are composed of objects; events are temporally related; actions have preconditions and goals. Our systems learn from trajectories with rich temporal and relational patterns and labels for events and actions. We demonstrate that they can generalize from small amounts of data to scenarios containing more objects than were present during training and to temporal warpings of input sequences, and exploits the goal-centric representation of actions to make plans for novel goals.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
In Copyright - Educational Use Permitted
Copyright MIT
Persistent DSpace Link
