MIT Libraries logoDSpace@MIT

MIT
View Item 
  • DSpace@MIT Home
  • MIT Libraries
  • MIT Theses
  • Graduate Theses
  • View Item
  • DSpace@MIT Home
  • MIT Libraries
  • MIT Theses
  • Graduate Theses
  • View Item
JavaScript is disabled for your browser. Some features of this site may not work without it.

Synthesizing controversial sentences for testing the brain-predictivity of language models

Author(s)
Rakocevic, Lara I.
Thumbnail
Download1251801747-MIT.pdf (1.238Mb)
Other Contributors
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science.
Advisor
Evelina Fedorenko and Noga Zaslavsky.
Terms of use
MIT theses may be protected by copyright. Please reuse MIT thesis content according to the MIT Libraries Permissions Policy, which is available through the URL provided. http://dspace.mit.edu/handle/1721.1/7582
Metadata
Show full item record
Abstract
Recent research has seen the rise of powerful neural-network language models that are sufficiently computationally precise and neurally plausible as to serve as a jumping-off base for our understanding of language processing in the brain. Because these models have been developed for optimizing a similar objective (word prediction), their brain predictions are often correlated, even though the models differ along several architectural and conceptual features, yielding a major challenge for testing which model features are most relevant for predicting language processing in the brain. Here, we address this challenge by synthesizing new sentence stimuli that maximally expose the disagreement between the predictions of a set of language models ('controversial stimuli'), which would not naturally occur in large language corpora . To do so, we develop a platform for systematizing this sentence synthesis process, providing a way to test different model-based hypotheses easily and efficiently. An initial exploration with this platform has begun to give us some intuition for how choosing from different pools of candidate words affect the kinds of sentences produced, and what kinds of changes tend to produce controversial sentences. For example, we show that the disagreement score, or the maximum amount of disagreement between models for a sentence, converges. This approach will eventually allow us to determine which models perform in the most human-like way and are most successful in predicting language processing in the brain, thus hopefully leading to insights on the mechanisms of human language understanding.
Description
Thesis: M. Eng., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, February, 2021
 
Cataloged from the official PDF of thesis.
 
Includes bibliographical references (pages 55-58).
 
Date issued
2021
URI
https://hdl.handle.net/1721.1/130713
Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Publisher
Massachusetts Institute of Technology
Keywords
Electrical Engineering and Computer Science.

Collections
  • Graduate Theses

Browse

All of DSpaceCommunities & CollectionsBy Issue DateAuthorsTitlesSubjectsThis CollectionBy Issue DateAuthorsTitlesSubjects

My Account

Login

Statistics

OA StatisticsStatistics by CountryStatistics by Department
MIT Libraries
PrivacyPermissionsAccessibilityContact us
MIT
Content created by the MIT Libraries, CC BY-NC unless otherwise noted. Notify us about copyright concerns.