On the Expressiveness and Generalization of Hypergraph Neural Networks

Abstract

Graph Neural Networks have demonstrated their success on many applications, including analyzing molecules and social networks. Although these graph neural networks can effectively determine pairwise connections between nodes, the data structure in reality sometimes goes beyond pairwise relations and can be complicated, involving multiple nodes. This requires the graph neural networks to be extended to hypergraphs to deal with higher-order relations. It is critical to understand what type of problems these hypergraph neural networks can solve and effectively learn from data.

In this thesis, we describe how we use Neural Logical Machines as a unified framework for analyzing the expressiveness, learning, and (structural) generalization of hypergraph neural networks (HyperGNNs). Specifically, we focus on how HyperGNNs can learn from finite datasets and generalize structurally to graph reasoning problems of arbitrary input sizes. Our first contribution is a fine-grained analysis of the expressiveness of HyperGNNs, that is, the set of functions that they can realize. Our result is a hierarchy of problems they can solve, defined in terms of various hyperparameters such as depths and edge arities. Next, we analyze the learning properties of these neural networks, especially focusing on how they can be trained on a finite set of small graphs and generalize to larger graphs, which we term structural generalization. Our theoretical results are further supported by the empirical results.