Structure learning in high-dimensional graphical models

Abstract

In this thesis, we develop efficient and provably consistent algorithms for learning the structure of undirected and directed (causal) graphical models in the high-dimensional setting. Structure learning in graphical models is a central problem in statistics with numerous applications including learning gene regulatory networks from RNA-seq data and learning the dependence structure among stocks from financial time series. Part I of this thesis investigates the problem of learning causal directed acyclic graph (DAG) models from a combination of observational and interventional data. While previous methods considered greedy search algorithms on the space of graphs, we propose to view a DAG as given by a permutation and an undirected graph and instead consider greedy search on the smaller space of permutations. We present the first consistency guarantees of a permutation-based greedy search algorithm based on observational data.

In addition, we show that this algorithm naturally extends to the interventional setting, thereby resulting in the first provably consistent algorithm for causal structure discovery from a mix of observational and interventional data. In Part II, we consider causal inference based on heterogeneous observational data collected from naturally perturbed systems. Specifically, we investigate two questions, namely 1) learning the difference between two causal DAGs, and 2) jointly estimating multiple related causal DAGs. To answer question 1), we provide the first provably consistent method for directly estimating the differences in a pair of causal DAGs without separately learning two possibly large and dense DAGs. To answer question 2), we provide a joint estimation procedure based on ℓ0-penalized maximum likelihood estimation and prove that such procedure leads to a faster convergence rate than estimating each DAG separately.

Finally, in Part III, we consider the problem of estimating undirected graphical models under distributional constraints. More specifically, we consider a particular form of positive dependence, known as total positivity. Such a constraint is relevant for example for portfolio selection, since assets are often positively dependent. Methods for learning undirected graphical models usually require a particular choice of the tuning parameter for consistent estimation, which is in general unknown a priori and hence a major limitation in applications. We here show that an undirected graphical model under total positivity can be learned consistently without any tuning parameters. The proposed methods are illustrated on various synthetic and real datasets from genomics and finance.