Stochastic Variational Inference for Bayesian Sparse Gaussian Process Regression
Name
1711.00221.pdf
Description
Submitted version
Size
1.03 MB
Format
Unknown
Checksum (MD5)
3452ac28e29df924876d20034f9fb2ee
Author(s)
Hoang, Trong Nghia
Jaillet, Patrick
Date Issued
September 2019
Journal
Proceedings of the International Joint Conference on Neural Networks (IJCNN'19)
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Citation
Yu, Haibin et al. “Stochastic Variational Inference for Bayesian Sparse Gaussian Process Regression.” Proceedings of the International Joint Conference on Neural Networks, July 2019, Budapest, Hungary, Institute of Electrical and Electronics Engineers, September 2019. © 2019 The Author(s)
Version
Original manuscript
Abstract
This paper presents a novel variational inference framework for deriving a family of Bayesian sparse Gaussian process regression (SGPR) models whose approximations are variationally optimal with respect to the full-rank GPR model enriched with various corresponding correlation structures of the observation noises. Our variational Bayesian SGPR (VBSGPR) models jointly treat both the distributions of the inducing variables and hyperparameters as variational parameters, which enables the decomposability of the variational lower bound that in turn can be exploited for stochastic optimization. Such a stochastic optimization involves iteratively following the stochastic gradient of the variational lower bound to improve its estimates of the optimal variational distributions of the inducing variables and hyperparameters (and hence the predictive distribution) of our VBSGPR models and is guaranteed to achieve asymptotic convergence to them. We show that the stochastic gradient is an unbiased estimator of the exact gradient and can be computed in constant time per iteration, hence achieving scalability to big data. We empirically evaluate the performance of our proposed framework on two real-world, massive datasets.
MIT Department
Massachusetts Institute of Technology. Laboratory for Information and Decision Systems
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1109/IJCNN.2019.8852481
