Adversarially Robust Generalization Requires More Data
Name
7749-adversarially-robust-generalization-requires-more-data.pdf
Description
Published version
Size
424.04 KB
Format
Adobe PDF
Checksum (MD5)
39c4c1c9a6efe4ad9b4339fcb4298740
Author(s)
Schmidt, Ludwig
Santurkar, Shibani
Tsipras, Dimitris
Talwar, Kunal
Madry, Aleksander
Date Issued
2018
Citation
Schmidt, Ludwig, Santurkar, Shibani, Tsipras, Dimitris, Talwar, Kunal and Madry, Aleksander. 2018. "Adversarially Robust Generalization Requires More Data."
Version
Final published version
Abstract
© 2018 Curran Associates Inc..All rights reserved. Machine learning models are often susceptible to adversarial perturbations of their inputs. Even small perturbations can cause state-of-the-art classifiers with high “standard” accuracy to produce an incorrect prediction with high confidence. To better understand this phenomenon, we study adversarially robust learning from the viewpoint of generalization. We show that already in a simple natural data model, the sample complexity of robust learning can be significantly larger than that of “standard” learning. This gap is information theoretic and holds irrespective of the training algorithm or the model family. We complement our theoretical results with experiments on popular image classification datasets and show that a similar gap exists here as well. We postulate that the difficulty of training robust classifiers stems, at least partially, from this inherently larger sample complexity.
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://papers.nips.cc/paper/7749-adversarially-robust-generalization-requires-more-data
