Estimation of GMRFs by recursive cavity modeling

Abstract

This thesis develops the novel method of recursive cavity modeling as a tractable approach to approximate inference in large Gauss-Markov random fields. The main idea is to recursively dissect the field, constructing a cavity model for each subfield at each level of dissection. The cavity model provides a compact yet (nearly) faithful model for the surface of one subfield sufficient for inferring other parts of the field. This basic idea is developed into a two-pass inference/modeling procedure which recursively builds cavity models by an "upward" pass and then builds complementary blanket models by a "downward" pass. Marginal models are then constructed at the finest level of dissection. Information-theoretic principles are employed for model thinning so as to develop compact yet faithful cavity and blanket models thereby providing tractable yet near-optimal inference. In this regard, recursive cavity modeling blends recursive inference and iterative modeling methodologies. While the main focus is on Gaussian processes, general principles are emphasized throughout suggesting the applicability of the basic framework for more general families of Markov random fields. The main objective of the method is to provide efficient, scalable, near-optimal inference for many problems of practical interest. Experiments performed thus far, with simulated Gauss-Markov random fields defined on two-dimensional grids, indicate good reliability and scalability of the method. The recursive cavity modeling method intersects with a variety of inference techniques arising in the graphical modeling literature including multiscale modeling, junction trees, projection filtering, Markov-blanket filtering, expectation propagation and other methods relying on reduction of embedded models. These connections are explored and important distinctions and extensions are noted. The author believes this thesis represents a significant generalization of existing methods, extending the class of Markov random fields for which reliable, scalable inference is available. But much work remains to better characterize and investigate this claim. Recommendations for furthering this agenda are outlined.