High dimensional linear regression using lattice basis reduction
Name
NeurIPS-2018-high-dimensional-linear-regression-using-lattice-basis-reduction-Paper.pdf
Description
Published version
Size
698.05 KB
Format
Unknown
Checksum (MD5)
b158c2b1fbb03ec5030a586207b5129d
Author(s)
Gamarnik, David
Zadik, Ilias
Date Issued
December 2018
Journal
Advances in Neural Information Processing Systems
Citation
2018. "High dimensional linear regression using lattice basis reduction." Advances in Neural Information Processing Systems, 2018-December.
Version
Final published version
Abstract
© 2018 Curran Associates Inc.All rights reserved. We consider a high dimensional linear regression problem where the goal is to efficiently recover an unknown vector β∗ from n noisy linear observations Y = Xβ∗ + W ∈ Rn, for known X ∈ Rn×p and unknown W ∈ Rn. Unlike most of the literature on this model we make no sparsity assumption on β∗. Instead we adopt a regularization based on assuming that the underlying vectors β∗ have rational entries with the same denominator Q ∈ Z>0. We call this Q-rationality assumption. We propose a new polynomial-time algorithm for this task which is based on the seminal Lenstra-Lenstra-Lovasz (LLL) lattice basis reduction algorithm. We establish that under the Q-rationality assumption, our algorithm recovers exactly the vector β∗ for a large class of distributions for the iid entries of X and non-zero noise W. We prove that it is successful under small noise, even when the learner has access to only one observation (n = 1). Furthermore, we prove that in the case of the Gaussian white noise for W, n = o(p/log p) and Q sufficiently large, our algorithm tolerates a nearly optimal information-theoretic level of the noise.
MIT Department
Sloan School of Management
Massachusetts Institute of Technology. Operations Research Center
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://papers.nips.cc/paper/2018/hash/ccc0aa1b81bf81e16c676ddb977c5881-Abstract.html
