Representation Learning Through the Lens of Science: Symmetry, Language and Symbolic Inductive Biases
Name
dangovski-rumenrd-phd-eecs-2023-thesis.pdf
Description
Thesis PDF
Size
18.87 MB
Format
Adobe PDF
Checksum (MD5)
cb5351d97a81c67be644e928ba614a9b
Author(s)
Dangovski, Rumen Rumenov
Advisor(s)
Soljačić, Marin
Date Issued
September 2023
Publisher
Massachusetts Institute of Technology
Abstract
In this thesis, we explore representation learning, a key technique in machine learning and artificial intelligence that has led to remarkable advancements in fields such as speech, vision, language perception and generation, as well as solving complex scientific problems like protein folding. Despite its success, the prevailing method of end-to-end supervised learning faces challenges, including the need for large datasets, non-interpretable classifications, and difficulties in transferring representations.
To address these limitations, we adopt a scientific perspective, focusing on machine learning tasks that are particularly affected by these issues, and developing benchmarks inspired by scientific principles. Our approach centers on the identification and development of novel inductive biases (assumptions made by the learning algorithm to improve generalization) based on symmetry, language, and symbolic properties. These inductive biases prove beneficial for both solving scientific problems using machine learning and enhancing representation learning methods.
We term this methodology “Representation Learning through the Lens of Science” and demonstrate its effectiveness in various applications. Finally, we discuss the limitations of our approach and propose directions for future research to further refine and expand upon the concepts introduced in this thesis.
MIT Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
In Copyright - Educational Use Permitted
Copyright retained by author(s)
Persistent DSpace Link
