Truncated Bayesian nonparametrics
Name
971020152-MIT.pdf
Description
Full printable version
Size
14.21 MB
Format
Adobe PDF
Checksum (MD5)
6e49d6c9a8304e586a1d75c134464ef3
Author(s)
Campbell, Trevor D. J. (Trevor David Jan)
Advisor(s)
Jonathan P. How.
Date Issued
2016
Publisher
Massachusetts Institute of Technology
Abstract
Many datasets can be thought of as expressing a collection of underlying traits with unknown cardinality. Moreover, these datasets are often persistently growing, and we expect the number of expressed traits to likewise increase over time. Priors from Bayesian nonparametrics are well-suited to this modeling challenge: they generate a countably infinite number of underlying traits, which allows the number of expressed traits to both be random and to grow with the dataset size. We also require corresponding streaming, distributed inference algorithms that handle persistently growing datasets without slowing down over time. However, a key ingredient in streaming, distributed inference-an explicit representation of the latent variables used to statistically decouple the data-is not available for nonparametric priors, as we cannot simulate or store infinitely many random variables in practice. One approach is to approximate the nonparametric prior by developing a sequential representation-such that the traits are generated by a sequence of finite-dimensional distributions-and subsequently truncating it at some finite level, thus allowing explicit representation. However, truncated sequential representations have been developed only for a small number of priors in Bayesian nonparametrics, and the order they impose on the traits creates identifiability issues in the streaming, distributed setting. This thesis provides a comprehensive theoretical treatment of sequential representations and truncation in Bayesian nonparametrics. It details three sequential representations of a large class of nonparametric priors, and analyzes their truncation error and computational complexity. The results generalize and improve upon those existing in the literature. Next, the truncated explicit representations are used to develop the first streaming, distributed, asynchronous inference procedures for models from Bayesian nonparametrics. The combinatorial issues associated with trait identifiability in such models are resolved via a novel matching optimization. The resulting algorithms are fast, learning rate-free, and truncation-free. Taken together, these contributions provide the practitioner with the means to (1) develop multiple finite approximations for a given nonparametric prior; (2) determine which is the best for their application; and (3) use that approximation in the development of efficient streaming, distributed, asynchronous inference algorithms.
Description
Thesis: Ph. D., Massachusetts Institute of Technology, Department of Aeronautics and Astronautics, 2016.
Cataloged from PDF version of thesis.
Includes bibliographical references (pages 167-175).
Subjects
Aeronautics and Astronautics.
MIT Department
Massachusetts Institute of Technology. Department of Aeronautics and Astronautics
Terms of Use
MIT theses are protected by copyright. They may be viewed, downloaded, or printed from this source but further reproduction or distribution in any format is prohibited without written permission.
Persistent DSpace Link
