On the Role of the Source Dataset in Transfer Learning
Name
Khaddaj-alaakh-SM-EECS-2022-thesis.pdf
Description
Thesis PDF
Size
4.8 MB
Format
Adobe PDF
Checksum (MD5)
8f78d3d75be8cdd36861646cea3f8615
Author(s)
Khaddaj, Alaa
Advisor(s)
Madry, Aleksander
Date Issued
September 2022
Publisher
Massachusetts Institute of Technology
Abstract
It is commonly believed that in transfer learning including more pre-training data translates into better performance. However, recent evidence suggests that removing data from the source dataset can actually help too. In this work, we take a closer look at the role of the source dataset's composition in transfer learning and present a framework for probing its impact on downstream performance. Our framework gives rise to new capabilities such as pinpointing transfer learning brittleness as well as detecting pathologies such as data-leakage and the presence of misleading examples in the source dataset. In particular, we demonstrate that removing detrimental datapoints identified by our framework improves transfer learning performance from ImageNet on a variety of target tasks.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
In Copyright - Educational Use Permitted
Copyright MIT
Persistent DSpace Link
