MIT Libraries logoDSpace@MIT

MIT
View Item 
  • DSpace@MIT Home
  • MIT Open Access Articles
  • MIT Open Access Articles
  • View Item
  • DSpace@MIT Home
  • MIT Open Access Articles
  • MIT Open Access Articles
  • View Item
JavaScript is disabled for your browser. Some features of this site may not work without it.

Learning-based low-rank approximations

Author(s)
Indyk, Piotr
Thumbnail
DownloadPublished version (1.282Mb)
Publisher Policy

Publisher Policy

Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.

Terms of use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Metadata
Show full item record
Abstract
We introduce a “learning-based” algorithm for the low-rank decomposition problem: given an n × d matrix A, and a parameter k, compute a rank-k matrix A0 that minimizes the approximation loss ||A - A0||F. The algorithm uses a training set of input matrices in order to optimize its performance. Specifically, some of the most efficient approximate algorithms for computing low-rank approximations proceed by computing a projection SA, where S is a sparse random m × n “sketching matrix”, and then performing the singular value decomposition of SA. We show how to replace the random matrix S with a “learned” matrix of the same sparsity to reduce the error. Our experiments show that, for multiple types of data sets, a learned sketch matrix can substantially reduce the approximation loss compared to a random matrix S, sometimes by one order of magnitude. We also study mixed matrices where only some of the rows are trained and the remaining ones are random, and show that matrices still offer improved performance while retaining worst-case guarantees. Finally, to understand the theoretical aspects of our approach, we study the special case of m = 1. In particular, we give an approximation algorithm for minimizing the empirical loss, with approximation factor depending on the stable rank of matrices in the training set. We also show generalization bounds for the sketch matrix learning problem.
Date issued
2019-12
URI
https://hdl.handle.net/1721.1/129423
Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Journal
Advances in Neural Information Processing Systems
Publisher
Morgan Kaufmann Publishers
Citation
Indyk, Piotr et al. “Learning-based low-rank approximations.” Advances in Neural Information Processing Systems, 32 (December 2019) © 2019 The Author(s)
Version: Final published version
ISSN
1049-5258

Collections
  • MIT Open Access Articles

Browse

All of DSpaceCommunities & CollectionsBy Issue DateAuthorsTitlesSubjectsThis CollectionBy Issue DateAuthorsTitlesSubjects

My Account

Login

Statistics

OA StatisticsStatistics by CountryStatistics by Department
MIT Libraries
PrivacyPermissionsAccessibilityContact us
MIT
Content created by the MIT Libraries, CC BY-NC unless otherwise noted. Notify us about copyright concerns.