Approximate sparse linear regression

Abstract

© 2018 Schloss Dagstuhl- Leibniz-Zentrum fur Informatik GmbH, Dagstuhl Publishing. All rights reserved. In the Sparse Linear Regression (SLR) problem, given a d × n matrix M and a d-dimensional query q, the goal is to compute a k-sparse n-dimensional vector τ such that the error Mτ − q is minimized. This problem is equivalent to the following geometric problem: given a set P of n points and a query point q in d dimensions, find the closest k-dimensional subspace to q, that is spanned by a subset of k points in P. In this paper, we present data-structures/algorithms and conditional lower bounds for several variants of this problem (such as finding the closest induced k dimensional flat/simplex instead of a subspace). In particular, we present approximation algorithms for the online variants of the above problems with query timeO(nk−1), which are of interest in the "low sparsity regime" where k is small, e.g., 2 or 3. For k = d, this matches, up to polylogarithmic factors, the lower bound that relies on the a nely degenerate conjecture (i.e., deciding if n points in Rd contains d+ 1 points contained in a hyperplane takes (nd) time). Moreover, our algorithms involve formulating and solving several geometric subproblems, which we believe to be of independent interest.