SoS Certificates for Sparse Singular Values and Their Applications: Robust Statistics, Subspace Distortion, and More
Name
3717823.3718293.pdf
Size
711.63 KB
Format
Adobe PDF
Checksum (MD5)
727c7e590a940f2c7ebbd45b7d6bf863
Author(s)
Diakonikolas, Ilias
Hopkins, Samuel B.
Pensia, Ankit
Tiegel, Stefan
Date Issued
June 15, 2025
Publisher
ACM|Proceedings of the 57th Annual ACM Symposium on Theory of Computing
Citation
Ilias Diakonikolas, Samuel B. Hopkins, Ankit Pensia, and Stefan Tiegel. 2025. SoS Certificates for Sparse Singular Values and Their Applications: Robust Statistics, Subspace Distortion, and More. In Proceedings of the 57th Annual ACM Symposium on Theory of Computing (STOC '25). Association for Computing Machinery, New York, NY, USA, 1701–1709.
Version
Final published version
Abstract
We study sparse singular value certificates for random rectangular matrices. If M is a d × n matrix with independent Gaussian entries, we give a new family of polynomial-time algorithms which can certify upper bounds on the maximum of ||M u||, where u is a unit vector with at most η n nonzero entries for a given η ∈ (0,1). This basic algorithmic primitive lies at the heart of a wide range of problems across algorithmic statistics and theoretical computer science, including robust mean and covariance estimation, certification of distortion of random subspaces of n, certification of the 2 → p norm of a random matrix, and sparse principal component analysis.
Our algorithms certify a bound which is asymptotically smaller than the naive one, given by the maximum singular value of M, for nearly the widest-possible range of n,d, and η. Efficiently certifying such a bound for a range of n,d and η which is larger by any polynomial factor than what is achieved by our algorithm would violate lower bounds in the statistical query and low-degree polynomials models. Our certification algorithm makes essential use of the Sum-of-Squares hierarchy. To prove the correctness of our algorithm, we develop a new combinatorial connection between the graph matrix approach to analyze random matrices with dependent entries, and the Efron-Stein decomposition of functions of independent random variables.
As applications of our certification algorithm, we obtain new efficient algorithms for a wide range of well-studied algorithmic tasks. In algorithmic robust statistics, we obtain new algorithms for robust mean and covariance estimation with tradeoffs between breakdown point and sample complexity, which are nearly matched by statistical query and low-degree polynomial lower bounds (that we establish). We also obtain new polynomial-time guarantees for certification of ℓ1/ℓ2 distortion of random subspaces of n (also with nearly matching lower bounds), sparse principal component analysis, and certification of the 2→ p norm of a random matrix.
Description
STOC ’25, Prague, Czechia
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/3717823.3718293
