Linear response methods for accurate covariance estimates from mean field variational bayes
Name
5755-linear-response-methods-for-accurate-covariance-estimates-from-mean-field-variational-bayes.pdf
Size
867.05 KB
Format
Adobe PDF
Checksum (MD5)
676786150d210945bea92d9f9a732b3e
Author(s)
Giordano, Ryan
Jordan, Michael
Broderick, Tamara A
Date Issued
December 2015
Journal
Advances in Neural Information Processing Systems 28 (NIPS 2015)
Publisher
Neural Information Processing Systems Foundation
Citation
Giordano, Ryan, Tamara Broderick, Tamara and Michael Jordan. "Linear Response Methods for Accurate Covariance Estimates from Mean Field Variational Bayes." Advances in Neural Information Processing Systems 28 (NIPS 2015),
Version
Final published version
Abstract
Mean field variational Bayes (MFVB) is a popular posterior approximation method due to its fast runtime on large-scale data sets. However, a well known failing of MFVB is that it underestimates the uncertainty of model variables (sometimes severely) and provides no information about model variable covariance. We generalize linear response methods from statistical physics to deliver accurate uncertainty estimates for model variables---both for individual variables and coherently across variables. We call our method linear response variational Bayes (LRVB). When the MFVB posterior approximation is in the exponential family, LRVB has a simple, analytic form, even for non-conjugate models. Indeed, we make no assumptions about the form of the true posterior. We demonstrate the accuracy and scalability of our method on a range of models for both simulated and real data.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://papers.nips.cc/book/advances-in-neural-information-processing-systems-28-2015
