2026-09-20T06:52:09Zhttps://dspace.mit.edu/server/oai/requestoai:dspace.mit.edu:1721.1/1231242026-06-06T01:03:15Zcom_1721.1_7582com_1721.1_7581col_1721.1_131023
Aleksander Ma̧dry.
Venigalla, Abhinav S.
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science.
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
2019-12-05T18:04:47Z
2019-12-05T18:04:47Z
2019
2019
https://hdl.handle.net/1721.1/123124
1128270896
This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.
Thesis: M. Eng. in Computer Science and Engineering, Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2019
Cataloged from student-submitted PDF version of thesis.
Includes bibliographical references (pages 59-60).
Training deep neural networks requires large quantities of labeled training data, on the order of thousands of examples per class. These requirements make model training both time-consuming and expensive, which provides an incentive for adversaries to steal, or copy, other users' models. In this work, we examine a recent defense method called neural network watermarking via memorized examples, where an owner intentionally trains his model to mislabel particular inputs. We try to isolate the mechanism by which memorized examples are learned by a model in order to better evaluate their robustness. We find that memorized examples are indeed strongly embedded in trained models and actually transfer to stolen models under one form of model stealing. When access to local input-logit gradient information is used by an attacker, the stolen model also learns to mislabel the memorized examples. We show that this transfer is robust to architecture mismatch and perturbations of the query set used for stealing. We present different possible mechanisms for memorized example transfer and find that local input geometry is insufficient to explain the phenomenon. Finally, we describe a simple method for a model owner to boost the transfer rate of memorized examples, increasing their effectiveness as a defense against model stealing.
by Abhinav S. Venigalla.
M.Eng. in Computer Science and Engineering
M.Eng.inComputerScienceandEngineering Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science
60 pages
eng
Massachusetts Institute of Technology
MIT theses are protected by copyright. They may be viewed, downloaded, or printed from this source but further reproduction or distribution in any format is prohibited without written permission.
http://dspace.mit.edu/handle/1721.1/7582
Electrical Engineering and Computer Science.
Strongly-transferring memorized examples in deep neural networks
Thesis
application/pdf
2019-12-05T18:04:46Z
Publication
Master
EECS
unknown
unknown
<Publication xmlns="https://www.openaire.eu/cerif-profile/1.1/" id="eaedb2c3-2f01-4bdc-9691-8d6f60b334e8">
<Type xmlns="https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types">http://purl.org/coar/resource_type/c_1843</Type>
<Language>eng</Language>
<Title>Strongly-transferring memorized examples in deep neural networks</Title>
<PublishedIn>
<Publication>
</Publication>
</PublishedIn>
<PublicationDate>2019</PublicationDate>
<Authors>
<Author>
<DisplayName>Venigalla, Abhinav S.</DisplayName>
<Affiliation>
<OrgUnit>
</OrgUnit>
</Affiliation>
</Author>
</Authors>
<Editors>
</Editors>
<Publishers>
<Publisher>
<DisplayName>Massachusetts Institute of Technology</DisplayName>
<OrgUnit />
</Publisher>
</Publishers>
<License>http://dspace.mit.edu/handle/1721.1/7582</License>
<Keyword>Electrical Engineering and Computer Science.</Keyword>
<Abstract>Training deep neural networks requires large quantities of labeled training data, on the order of thousands of examples per class. These requirements make model training both time-consuming and expensive, which provides an incentive for adversaries to steal, or copy, other users' models. In this work, we examine a recent defense method called neural network watermarking via memorized examples, where an owner intentionally trains his model to mislabel particular inputs. We try to isolate the mechanism by which memorized examples are learned by a model in order to better evaluate their robustness. We find that memorized examples are indeed strongly embedded in trained models and actually transfer to stolen models under one form of model stealing. When access to local input-logit gradient information is used by an attacker, the stolen model also learns to mislabel the memorized examples. We show that this transfer is robust to architecture mismatch and perturbations of the query set used for stealing. We present different possible mechanisms for memorized example transfer and find that local input geometry is insufficient to explain the phenomenon. Finally, we describe a simple method for a model owner to boost the transfer rate of memorized examples, increasing their effectiveness as a defense against model stealing.</Abstract>
<Access xmlns="http://purl.org/coar/access_right"
>
</Access>
</Publication>