2026-09-20T11:02:16Zhttps://dspace.mit.edu/server/oai/requestoai:dspace.mit.edu:1721.1/1472782023-01-20T03:27:28Zcom_1721.1_7582com_1721.1_7581col_1721.1_131023
Madry, Aleksander
Khaddaj, Alaa
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
2023-01-19T18:42:23Z
2023-01-19T18:42:23Z
2022-09
2022-10-19T18:57:35.581Z
https://hdl.handle.net/1721.1/147278
It is commonly believed that in transfer learning including more pre-training data translates into better performance. However, recent evidence suggests that removing data from the source dataset can actually help too. In this work, we take a closer look at the role of the source dataset's composition in transfer learning and present a framework for probing its impact on downstream performance. Our framework gives rise to new capabilities such as pinpointing transfer learning brittleness as well as detecting pathologies such as data-leakage and the presence of misleading examples in the source dataset. In particular, we demonstrate that removing detrimental datapoints identified by our framework improves transfer learning performance from ImageNet on a variety of target tasks.
S.M.
Massachusetts Institute of Technology
In Copyright - Educational Use Permitted
Copyright MIT
http://rightsstatements.org/page/InC-EDU/1.0/
On the Role of the Source Dataset in Transfer Learning
Thesis
application/pdf
Master
Master of Science in Electrical Engineering and Computer Science
Publication
unknown
unknown
<Publication xmlns="https://www.openaire.eu/cerif-profile/1.1/" id="9efd2c66-1442-4a22-b152-e487c16d8e1d">
<Type xmlns="https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types">http://purl.org/coar/resource_type/c_1843</Type>
<Title>On the Role of the Source Dataset in Transfer Learning</Title>
<PublishedIn>
<Publication>
</Publication>
</PublishedIn>
<PublicationDate>2022-09</PublicationDate>
<Authors>
<Author>
<DisplayName>Khaddaj, Alaa</DisplayName>
<Affiliation>
<OrgUnit>
</OrgUnit>
</Affiliation>
</Author>
</Authors>
<Editors>
</Editors>
<Publishers>
<Publisher>
<DisplayName>Massachusetts Institute of Technology</DisplayName>
<OrgUnit />
</Publisher>
</Publishers>
<License>http://rightsstatements.org/page/InC-EDU/1.0/</License>
<Abstract>It is commonly believed that in transfer learning including more pre-training data translates into better performance. However, recent evidence suggests that removing data from the source dataset can actually help too. In this work, we take a closer look at the role of the source dataset's composition in transfer learning and present a framework for probing its impact on downstream performance. Our framework gives rise to new capabilities such as pinpointing transfer learning brittleness as well as detecting pathologies such as data-leakage and the presence of misleading examples in the source dataset. In particular, we demonstrate that removing detrimental datapoints identified by our framework improves transfer learning performance from ImageNet on a variety of target tasks.</Abstract>
<Access xmlns="http://purl.org/coar/access_right"
>
</Access>
</Publication>