Provable algorithms for inference in topic models
Name
1605.08491.pdf
Size
530.26 KB
Format
Adobe PDF
Checksum (MD5)
ad927ea3ab707e4dc93b9d2a37f3ce48
Author(s)
Arora, Sanjeev
Ge, Rong
Ma, Tengyu
Koehler, Frederic
Moitra, Ankur
Date Issued
June 2016
Journal
Proceedings of The 33rd International Conference on Machine Learning
Publisher
PMLR
Citation
Arora, Sanjeev et al. "Provable Algorithms for Inference in Topic Models." Proceedings of The 33rd International Conference on Machine Learning, 19-24 June, 2016, New York City, New York, PMLR, 2016. © 2016 the Authors
Version
Author's final manuscript
Abstract
Recently, there has been considerable progress on designing algorithms with provable guarantees - typically using linear algebraic methods - for parameter learning in latent variable models. But designing provable algorithms for inference has proven to be more challenging. Here we take a first step towards provable inference in topic models. We leverage a property of topic models that enables us to construct simple linear estimators for the unknown topic proportions that have small variance, and consequently can work with short documents. Our estimators also correspond to finding an estimate around which the posterior is well-concentrated. We show lower bounds that for shorter documents it can be information theoretically impossible to find the hidden topics. Finally, we give empirical results that demonstrate that our algorithm works on realistic topic models. It yields good solutions on synthetic data and runs in time comparable to a single iteration of Gibbs sampling.
MIT Department
Massachusetts Institute of Technology. Department of Mathematics
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
http://proceedings.mlr.press/v48/arorab16.html
