Sparse methods for automatic relevance determination
Name
2005.08741.pdf
Description
Submitted version
Size
976.44 KB
Format
Unknown
Checksum (MD5)
d3c6643fb6cb456fcc85522bb156cf88
Author(s)
Rudy, Samuel H
Sapsis, Themistoklis Panagiotis
Date Issued
April 2021
Journal
Physica D: Nonlinear Phenomena
Publisher
Elsevier BV
Citation
Rudy, Samuel H and Sapsis, Themistoklis P. 2021. "Sparse methods for automatic relevance determination." Physica D: Nonlinear Phenomena, 418.
Version
Original manuscript
Abstract
© 2021 Elsevier B.V. This work considers methods for imposing sparsity in Bayesian regression with applications in nonlinear system identification. We first review automatic relevance determination (ARD) and analytically demonstrate the need to additional regularization or thresholding to achieve sparse models. We then discuss two classes of methods, regularization based and thresholding based, which build on ARD to learn parsimonious solutions to linear problems. In the case of orthogonal features, we analytically demonstrate favorable performance with regard to learning a small set of active terms in a linear system with a sparse solution. Several example problems are presented to compare the set of proposed methods in terms of advantages and limitations to ARD in bases with hundreds of elements. The aim of this paper is to analyze and understand the assumptions that lead to several algorithms and to provide theoretical and empirical results so that the reader may gain insight and make more informed choices regarding sparse Bayesian regression.
MIT Department
Massachusetts Institute of Technology. Department of Mechanical Engineering
Terms of Use
Creative Commons Attribution-NonCommercial-NoDerivs License
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1016/J.PHYSD.2021.132843
