Parallel Gaussian process regression for big data: Low-rank representation meets markov approximation
Name
Jaillet_Parallel gausian process.pdf
Size
453.3 KB
Format
Adobe PDF
Checksum (MD5)
af5652c35642708700a119050abaf43e
Author(s)
Low, Kian Hsiang
Yu, Jiangbo
Chen, Jie
Jaillet, Patrick
Date Issued
January 2015
Journal
AAAI'15 Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence
Publisher
Association for Computing Machinery
Citation
Low, Kian Hsiang. "Parallel gaussian process regression for big data: low-rank representation meets markov approximation." AAAI'15 Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, 25-30 January, 2015, Austin, Texas, ACM, 2015, pp. 2821-2827.
Version
Author's final manuscript
Abstract
The expressive power of a Gaussian process (GP) model comes at a cost of poor scalability in the data size. To improve its scalability, this paper presents a low-rank-cum-Markov approximation (LMA) of the GP model that is novel in leveraging the dual computational advantages stemming from complementing a low-rank approximate representation of the full-rank GP based on a support set of inputs with a Markov approximation of the resulting residual process; the latter approximation is guaranteed to be closest in the Kullback-Leibler distance criterion subject to some constraint and is considerably more refined than that of existing sparse GP models utilizing low-rank representations due to its more relaxed conditional independence assumption (especially with larger data). As a result, our LMA method can trade off between the size of the support set and the order of the Markov property to (a) incur lower computational cost than such sparse GP models while achieving predictive performance comparable to them and (b) accurately represent features/patterns of any scale. Interestingly, varying the Markov order produces a spectrum of LMAs with PIC approximation and full-rank GP at the two extremes. An advantage of our LMA method is that it is amenable to parallelization on multiple machines/cores, thereby gaining greater scalability. Empirical evaluation on three real-world datasets in clusters of up to 32 computing nodes shows that our centralized and parallel LMA methods are significantly more time-efficient and scalable than state-of-the-art sparse and full-rank GP regression methods while achieving comparable predictive performances.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
http://dl.acm.org/citation.cfm?id=2886714
