The lottery ticket hypothesis in an adversarial setting
Name
1227275418-MIT.pdf
Size
1.66 MB
Format
Adobe PDF
Checksum (MD5)
bed6e4aa8eb5d13f15aa6fd9e8d2321e
Author(s)
Gilles, James(James H.)
Advisor(s)
Michael Carbin.
Date Issued
2020
Publisher
Massachusetts Institute of Technology
Abstract
Deep neural networks are vulnerable to adversarial examples, inputs which appear natural to humans but are misclassified by deep models with a high degree of confidence. The best known defenses against adversarial examples are network capacity and adversarial training. These defenses are very expensive, greatly increasing storage, computation, and energy costs. The Lottery Ticket Hypothesis ("LTH") may help ameliorate this problem. LTH proposes that deep neural networks contain "matching subnetworks", sparse subnetworks to which the network can be pruned early in training, without losing accuracy. In this thesis, we study whether LTH applies in the setting of adversarial training for image classification networks. We find that sparse matching subnetworks indeed exist, and can reduce model sizes as much as 96% early in training. We also find that the size of an architecture's smallest matching subnetworks is always roughly the same, whether or not adversarial training is used.
Description
Thesis: M. Eng., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, September, 2020
Cataloged from student-submitted PDF of thesis.
Includes bibliographical references (pages 61-65).
Subjects
Electrical Engineering and Computer Science.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
MIT theses may be protected by copyright. Please reuse MIT thesis content according to the MIT Libraries Permissions Policy, which is available through the URL provided.
Persistent DSpace Link
