2026-09-19T11:53:45Zhttps://dspace.mit.edu/server/oai/requestoai:dspace.mit.edu:1721.1/1565482024-09-04T03:39:10Zcom_1721.1_7582com_1721.1_7581col_1721.1_131022
Rakhlin, Alexander (Sasha)
Zhai, Xiyu
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
2024-09-03T21:06:28Z
2024-09-03T21:06:28Z
2024-05
2024-07-10T13:02:27.164Z
https://hdl.handle.net/1721.1/156548
Deep learning has been quite successful in the past decade, with fantastic progress made across multiple domains such as computer vision, natural language processing, and Reinforcement learning. However, the theoretical understanding of its success is limited, and its behavior constantly defies our traditional theoretical understanding of machine learning. We will present our work on deep learning theory and show powerful techniques we developed for studying wide neural networks’ optimization and generalization behavior that help narrow the gap between theory and practice. Inspired by the recent success of transformer architectures like ChatGPT and SORA, we would like to present our work on the expressive power of vision transformers. Besides, we have been working on new AI theories and algorithms that go beyond deep learning and a new AI programming system that aims to make the implementation of these new ideas possible. It goes beyond the scope of a PhD to finish a demonstration of the new AI, but we aim to show strong evidence of its feasibility.
Ph.D.
Massachusetts Institute of Technology
Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0)
Copyright retained by author(s)
https://creativecommons.org/licenses/by-nc-nd/4.0/
Towards Effective Theories for Deep Learning and Beyond
Thesis
application/pdf
Doctoral
Doctor of Philosophy
Publication
unknown
unknown
<Publication xmlns="https://www.openaire.eu/cerif-profile/1.1/" id="62bc2a5d-50c7-4553-9db3-86f974890f57">
<Type xmlns="https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types">http://purl.org/coar/resource_type/c_1843</Type>
<Title>Towards Effective Theories for Deep Learning and Beyond</Title>
<PublishedIn>
<Publication>
</Publication>
</PublishedIn>
<PublicationDate>2024-05</PublicationDate>
<Authors>
<Author>
<DisplayName>Zhai, Xiyu</DisplayName>
<Affiliation>
<OrgUnit>
</OrgUnit>
</Affiliation>
</Author>
</Authors>
<Editors>
</Editors>
<Publishers>
<Publisher>
<DisplayName>Massachusetts Institute of Technology</DisplayName>
<OrgUnit />
</Publisher>
</Publishers>
<License>https://creativecommons.org/licenses/by-nc-nd/4.0/</License>
<Abstract>Deep learning has been quite successful in the past decade, with fantastic progress made across multiple domains such as computer vision, natural language processing, and Reinforcement learning. However, the theoretical understanding of its success is limited, and its behavior constantly defies our traditional theoretical understanding of machine learning. We will present our work on deep learning theory and show powerful techniques we developed for studying wide neural networks’ optimization and generalization behavior that help narrow the gap between theory and practice. Inspired by the recent success of transformer architectures like ChatGPT and SORA, we would like to present our work on the expressive power of vision transformers. Besides, we have been working on new AI theories and algorithms that go beyond deep learning and a new AI programming system that aims to make the implementation of these new ideas possible. It goes beyond the scope of a PhD to finish a demonstration of the new AI, but we aim to show strong evidence of its feasibility.</Abstract>
<Access xmlns="http://purl.org/coar/access_right"
>
</Access>
</Publication>