2026-09-21T18:40:09Zhttps://dspace.mit.edu/server/oai/requestoai:dspace.mit.edu:1721.1/1446802022-08-30T03:39:29Zcom_1721.1_7582com_1721.1_7581col_1721.1_131023
Andreas, Jacob
Bau, Anthony
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
2022-08-29T16:04:17Z
2022-08-29T16:04:17Z
2022-05
2022-06-02T20:24:16.955Z
https://hdl.handle.net/1721.1/144680
How do syntax and semantics interact in neural language models? Is the intuitive dichotomy between semantics and syntax useful as a mental model of their behavior? In this thesis, I systematically investigate how models handle the interactions between sentence corruptions of each kind. I develop a setup for generating “syntactic” and “semantic” corruptions and combining them. Using this setup, I develop a general framework for understanding how classes of corruptions interact with each other, arguing that this can be understood in terms of a single “entanglement” parameter. This allows me to quantify entanglement between syntax and semantics in a variety of situations. The result is that semantics is more entangled with syntax than it is with itself, and that syntax is highly self-entangled.
M.Eng.
Massachusetts Institute of Technology
In Copyright - Educational Use Permitted
Copyright MIT
http://rightsstatements.org/page/InC-EDU/1.0/
Interactions Between Syntax and Semantics in Language Models
Thesis
application/pdf
Master
Master of Engineering in Electrical Engineering and Computer Science
Publication
unknown
unknown
<Publication xmlns="https://www.openaire.eu/cerif-profile/1.1/" id="6b0e294b-61d2-4216-81e3-3360d83a8de6">
<Type xmlns="https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types">http://purl.org/coar/resource_type/c_1843</Type>
<Title>Interactions Between Syntax and Semantics in Language Models</Title>
<PublishedIn>
<Publication>
</Publication>
</PublishedIn>
<PublicationDate>2022-05</PublicationDate>
<Authors>
<Author>
<DisplayName>Bau, Anthony</DisplayName>
<Affiliation>
<OrgUnit>
</OrgUnit>
</Affiliation>
</Author>
</Authors>
<Editors>
</Editors>
<Publishers>
<Publisher>
<DisplayName>Massachusetts Institute of Technology</DisplayName>
<OrgUnit />
</Publisher>
</Publishers>
<License>http://rightsstatements.org/page/InC-EDU/1.0/</License>
<Abstract>How do syntax and semantics interact in neural language models? Is the intuitive dichotomy between semantics and syntax useful as a mental model of their behavior? In this thesis, I systematically investigate how models handle the interactions between sentence corruptions of each kind. I develop a setup for generating “syntactic” and “semantic” corruptions and combining them. Using this setup, I develop a general framework for understanding how classes of corruptions interact with each other, arguing that this can be understood in terms of a single “entanglement” parameter. This allows me to quantify entanglement between syntax and semantics in a variety of situations. The result is that semantics is more entangled with syntax than it is with itself, and that syntax is highly self-entangled.</Abstract>
<Access xmlns="http://purl.org/coar/access_right"
>
</Access>
</Publication>