2026-09-18T17:34:50Zhttps://dspace.mit.edu/server/oai/requestoai:dspace.mit.edu:1721.1/1527282023-11-03T03:50:49Zcom_1721.1_7582com_1721.1_7581col_1721.1_131023
Solomon, Justin
Gabrielsson, Rickard Brüel
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
2023-11-02T20:11:33Z
2023-11-02T20:11:33Z
2023-09
2023-09-21T14:26:34.499Z
https://hdl.handle.net/1721.1/152728
We introduce Deep Augmentation, an approach to data augmentation using dropout to
dynamically transform a targeted layer within a neural network, with the option to use
the stop-gradient operation, offering significant improvements in model performance and
generalization. We demonstrate the efficacy of Deep Augmentation through extensive
experiments on contrastive learning tasks in computer vision and NLP domains, where we
observe substantial performance gains with ResNets and Transformers as the underlying
models. Our experimentation reveals that targeting deeper layers with Deep Augmentation
outperforms augmenting the input data, and the simple network- and data-agnostic nature of
this approach enables its seamless integration into computer vision and NLP pipelines.
S.M.
Massachusetts Institute of Technology
In Copyright - Educational Use Permitted
Copyright retained by author(s)
https://rightsstatements.org/page/InC-EDU/1.0/
Enhancing Self-Supervised Learning through Transformations in Higher Activation Space
Thesis
application/pdf
Master
Master of Science in Electrical Engineering and Computer Science
Publication
unknown
unknown
<Publication xmlns="https://www.openaire.eu/cerif-profile/1.1/" id="3ffba35a-86d7-46c5-8c9d-edb7366292bc">
<Type xmlns="https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types">http://purl.org/coar/resource_type/c_1843</Type>
<Title>Enhancing Self-Supervised Learning through Transformations in Higher Activation Space</Title>
<PublishedIn>
<Publication>
</Publication>
</PublishedIn>
<PublicationDate>2023-09</PublicationDate>
<Authors>
<Author>
<DisplayName>Gabrielsson, Rickard Brüel</DisplayName>
<Affiliation>
<OrgUnit>
</OrgUnit>
</Affiliation>
</Author>
</Authors>
<Editors>
</Editors>
<Publishers>
<Publisher>
<DisplayName>Massachusetts Institute of Technology</DisplayName>
<OrgUnit />
</Publisher>
</Publishers>
<License>https://rightsstatements.org/page/InC-EDU/1.0/</License>
<Abstract>We introduce Deep Augmentation, an approach to data augmentation using dropout to
dynamically transform a targeted layer within a neural network, with the option to use
the stop-gradient operation, offering significant improvements in model performance and
generalization. We demonstrate the efficacy of Deep Augmentation through extensive
experiments on contrastive learning tasks in computer vision and NLP domains, where we
observe substantial performance gains with ResNets and Transformers as the underlying
models. Our experimentation reveals that targeting deeper layers with Deep Augmentation
outperforms augmenting the input data, and the simple network- and data-agnostic nature of
this approach enables its seamless integration into computer vision and NLP pipelines.</Abstract>
<Access xmlns="http://purl.org/coar/access_right"
>
</Access>
</Publication>