Optimal transport in structured domains : algorithms and applications

Abstract

Optimal transport provides a powerful mathematical framework for comparing probability distributions, and has found successful application in various problems in machine learning, including point cloud matching, generative modeling, and document comparison. However, some important limitations curtail its broader applicability. In many applications there is often additional structural information that is not captured by the classic formulation of the problem. This information can range from explicit tree and graph-like structure, to global structural invariances. Failure to fully model this structure can hinder--if not preclude--the use of optimal transport-based approaches. This thesis presents several extensions of the optimal transport problem to incorporate structural information. First, a non-linear generalization of the cost objective based on submodularity is proposed.

The resulting formulation provides a flexible framework to model explicit or latent discrete structure in the data and admits efficient optimization. Next, we investigate the issue of geometric invariances when matching embedded representations, for which a general framework for optimal transport in the presence of latent global transformations is developed. Various approaches to solve the resulting optimization problem are proposed and compared. The last part of the thesis addresses the problem of aligning datasets in which the structure is encoded through non-Euclidean manifolds, such as hyperbolic spaces. In response to an unexpected type of invariance that hyperbolic embeddings learned from data exhibit, a novel framework that interweaves optimal transport and hyperbolic nonlinear registration with deep neural networks is proposed.

While these extensions are formulated in general terms, the experimental results presented in this thesis are focused on motivating applications in natural language processing, including unsupervised word translation, sentence similarity, domain adaptation, and ontology alignment.