Revisiting Contrastive Learning through the Lens of Neighborhood Component Analysis
Name
Ko-cyko-SM-EECS-2022-thesis.pdf
Description
Thesis PDF
Size
1.45 MB
Format
Adobe PDF
Checksum (MD5)
1f2f4087f931378324adea28db5b068b
Author(s)
Ko, Ching-Yun
Advisor(s)
Daniel, Luca
Date Issued
May 2022
Publisher
Massachusetts Institute of Technology
Abstract
As a seminal tool in self-supervised representation learning, contrastive learning has gained unprecedented attention in recent years. In essence, contrastive learning aims to leverage pairs of positive and negative samples for representation learning, which relates to exploiting neighborhood information in a feature space. However, as a self-supervised learning method, the current contrastive learning method have encoded priors on the downstream classification tasks implicitly. In this thesis, by investigating the connection between contrastive learning and neighborhood component analysis (NCA), we provide a novel stochastic nearest neighbor viewpoint of contrastive learning and subsequently propose a series of contrastive losses that outperform the existing ones. Under our proposed framework, we show a new methodology to design integrated contrastive losses that could simultaneously achieve good accuracy and robustness on downstream tasks.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
In Copyright - Educational Use Permitted
Copyright MIT
Persistent DSpace Link
