Modular graph-structured models for prediction and control

Abstract

In contrast to models that assume no structural prior about the system being modelled, structured models requires far less computational resources to train and are more interpretable by nature of their modularity. In this thesis we discuss two works in machine learning, both of which model a system of interest using a graph-structured neural model. The first work details an approach to neural relational inference, the problem of modelling dynamical systems composed of different types of entities and relations and inferring the types of from observational data. We frame this problem using modular meta-learning, learning a set of neural modules which we can compose using a graph neural network to model the entities and relations of the system. Modular meta-learning is a model-based solution that offers several advantages over non model-based approaches including time-invariance of observations, the ability to infer relations in the context of one another rather than independently, and capacity for other inference tasks such as predicting the path of unseen entities based on their observed influence on other entities in the system. The second work addresses the problem of learning policies for continuous control in graph-structured agents. We incorporate the agent's structural information into the policy using graph neural networks, modelling the joints using a node module and the limbs between joints using an edge module. Individual joint modules communicate to one another via message-passing, allowing all joints to collaborate as a single agent, but output actions only for the single joint that they model. Further, we explore goal-setting as a form of intrinsic reward in reward-sparse environments, which allows us to learn complex tasks more efficient learning. Both works show the power of the inductive bias of structure in learning problems.