Subset Selection with Shrinkage: Sparse Linear Modeling When the SNR Is Low
Name
1708.03288.pdf
Description
Accepted version
Size
1.28 MB
Format
Adobe PDF
Checksum (MD5)
1d8fd0f6b8c87ab8181829a26d3c336d
Author(s)
Mazumder, Rahul
Radchenko, Peter
Dedieu, Antoine
Date Issued
May 24, 2022
Journal
Operations Research
Publisher
Institute for Operations Research and the Management Sciences (INFORMS)
Citation
Mazumder, Rahul, Radchenko, Peter and Dedieu, Antoine. 2022. "Subset Selection with Shrinkage: Sparse Linear Modeling When the SNR Is Low." Operations Research.
Version
Author's final manuscript
Abstract
Learning Compact High-Dimensional Models in Noisy Environments Building compact, interpretable statistical models where the output depends upon a small number of input features is a well-known problem in modern analytics applications. A fundamental tool used in this context is the prominent best subset selection (BSS) procedure, which seeks to obtain the best linear fit to data subject to a constraint on the number of nonzero features. Whereas the BSS procedure works exceptionally well in some regimes, it performs pretty poorly in out-of-sample predictive performance when the underlying data are noisy, which is quite common in practice. In this paper, we explore this relatively less-understood overfitting behavior of BSS in low-signal noisy environments and propose alternatives that appear to mitigate such shortcomings. We study the theoretical statistical properties of our proposed regularized BSS procedure and show promising computational results on various data sets, using tools from integer programming and first-order methods.
MIT Department
Sloan School of Management
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1287/opre.2022.2276
