Enhancing Self-Supervised Learning through Transformations in Higher Activation Space
Name
gabrielsson-brg-sm-eecs-2023-thesis.pdf
Description
Thesis PDF
Size
982.99 KB
Format
Adobe PDF
Checksum (MD5)
190f521c5bcae331807bd3d484acf051
Author(s)
Gabrielsson, Rickard Brüel
Advisor(s)
Solomon, Justin
Date Issued
September 2023
Publisher
Massachusetts Institute of Technology
Abstract
We introduce Deep Augmentation, an approach to data augmentation using dropout to
dynamically transform a targeted layer within a neural network, with the option to use
the stop-gradient operation, offering significant improvements in model performance and
generalization. We demonstrate the efficacy of Deep Augmentation through extensive
experiments on contrastive learning tasks in computer vision and NLP domains, where we
observe substantial performance gains with ResNets and Transformers as the underlying
models. Our experimentation reveals that targeting deeper layers with Deep Augmentation
outperforms augmenting the input data, and the simple network- and data-agnostic nature of
this approach enables its seamless integration into computer vision and NLP pipelines.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
In Copyright - Educational Use Permitted
Copyright retained by author(s)
Persistent DSpace Link
