Sample-Optimal Private Regression in Polynomial Time
Name
3717823.3718218.pdf
Size
642.41 KB
Format
Adobe PDF
Checksum (MD5)
ebdd2f901f3a3053c6b14ed54988f968
Author(s)
Anderson, Prashanti
Bakshi, Ainesh
Majid, Mahbod
Tiegel, Stefan
Date Issued
June 15, 2025
Publisher
ACM|Proceedings of the 57th Annual ACM Symposium on Theory of Computing
Citation
Prashanti Anderson, Ainesh Bakshi, Mahbod Majid, and Stefan Tiegel. 2025. Sample-Optimal Private Regression in Polynomial Time. In Proceedings of the 57th Annual ACM Symposium on Theory of Computing (STOC '25). Association for Computing Machinery, New York, NY, USA, 2341–2349.
Version
Final published version
Abstract
We consider the task of privately obtaining prediction error guarantees in ordinary least-squares regression problems with Gaussian covariates (with unknown covariance structure). We provide the first sample-optimal polynomial time algorithm for this task under both pure and approximate differential privacy. We show that any improvement to the sample complexity of our algorithm would violate either statistical-query or information-theoretic lower bounds. Additionally, our algorithm is robust to a small fraction of arbitrary outliers and achieves optimal error rates as a function of the fraction of outliers. In contrast, all prior efficient algorithms either incurred sample complexities with sub-optimal dimension dependence, scaling with the condition number of the covariates, or obtained a polynomially worse dependence on the privacy parameters.
Our technical contributions are two-fold: first, we leverage resilience guarantees of Gaussians within the sum-of-squares framework. As a consequence, we obtain efficient sum-of-squares algorithms for regression with optimal robustness rates and sample complexity. Second, we generalize the recent robustness-to-privacy framework of Hopkins, Kamath, Majid, and Narayanan to account for the geometry induced by the covariance of the input samples. This framework crucially relies on the robust estimators to be sum-of-squares algorithms, and combining the two steps yields a sample-optimal private regression algorithm. We believe our techniques are of independent interest, and we demonstrate this by obtaining an efficient algorithm for covariance-aware mean estimation, with an optimal dependence on the privacy parameters.
Description
STOC ’25, Prague, Czechia
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Massachusetts Institute of Technology. Department of Mathematics
Terms of Use
Creative Commons Attribution
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/3717823.3718218
