The lottery ticket hypothesis in an adversarial setting
Author(s)Gilles, James(James H.)
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science.
MetadataShow full item record
Deep neural networks are vulnerable to adversarial examples, inputs which appear natural to humans but are misclassified by deep models with a high degree of confidence. The best known defenses against adversarial examples are network capacity and adversarial training. These defenses are very expensive, greatly increasing storage, computation, and energy costs. The Lottery Ticket Hypothesis ("LTH") may help ameliorate this problem. LTH proposes that deep neural networks contain "matching subnetworks", sparse subnetworks to which the network can be pruned early in training, without losing accuracy. In this thesis, we study whether LTH applies in the setting of adversarial training for image classification networks. We find that sparse matching subnetworks indeed exist, and can reduce model sizes as much as 96% early in training. We also find that the size of an architecture's smallest matching subnetworks is always roughly the same, whether or not adversarial training is used.
Thesis: M. Eng., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, September, 2020Cataloged from student-submitted PDF of thesis.Includes bibliographical references (pages 61-65).
DepartmentMassachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Massachusetts Institute of Technology
Electrical Engineering and Computer Science.