Interaction Is Necessary for Distributed Learning with Privacy or Communication Constraints
Name
3357713.3384315.pdf
Size
985.33 KB
Format
Adobe PDF
Checksum (MD5)
5c935d5b2bcf2402196d5c6613c6eb73
Author(s)
Dagan, Yuval
Feldman, Vitaly
Date Issued
June 22, 2020
Publisher
ACM|Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing
Citation
Yuval Dagan and Vitaly Feldman. 2020. Interaction is necessary for distributed learning with privacy or communication constraints. In Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing (STOC 2020). Association for Computing Machinery, New York, NY, USA, 450–462.
Version
Final published version
Abstract
Local differential privacy (LDP) is a model where users send privatized data to an untrusted central server whose goal it to solve some data analysis task. In the non-interactive version of this model the protocol consists of a single round in which a server sends requests to all users then receives their responses. This version is deployed in industry due to its practical advantages and has attracted significant research interest.
Our main result is an exponential lower bound on the number of samples necessary to solve the standard task of learning a large-margin linear separator in the non-interactive LDP model. Via a standard reduction this lower bound implies an exponential lower bound for stochastic convex optimization and specifically, for learning linear models with a convex, Lipschitz and smooth loss. These results answer the questions posed by Smith, Thakurta, and Upadhyay (IEEE Symposium on Security and Privacy 2017) and Daniely and Feldman (NeurIPS 2019). Our lower bound relies on a new technique for constructing pairs of distributions with nearly matching moments but whose supports can be nearly separated by a large margin hyperplane. These lower bounds also hold in the model where communication from each user is limited and follow from a lower bound on learning using non-adaptive statistical queries.
Description
STOC ’20, June 22–26, 2020, Chicago, IL, USA
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1145/3357713.3384315
