MIT Libraries logoDSpace@MIT

MIT
View Item 
  • DSpace@MIT Home
  • MIT Libraries
  • MIT Theses
  • Graduate Theses
  • View Item
  • DSpace@MIT Home
  • MIT Libraries
  • MIT Theses
  • Graduate Theses
  • View Item
JavaScript is disabled for your browser. Some features of this site may not work without it.

On factuality in neural language models

Author(s)
Nadeem, Moin.
Thumbnail
Download1251800584-MIT.pdf (2.522Mb)
Other Contributors
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science.
Advisor
James Glass.
Terms of use
MIT theses may be protected by copyright. Please reuse MIT thesis content according to the MIT Libraries Permissions Policy, which is available through the URL provided. http://dspace.mit.edu/handle/1721.1/7582
Metadata
Show full item record
Abstract
In the past several years, language modeling has made significant advances on artificial benchmarks. However, despite these advancements, language models still face significant issues when deployed in real-world settings. In particular, these models tend to hallucinate facts and demonstrate significant harmful societal biases that render them impractical in the real-world. This thesis introduces datasets, models, and methodologies for studying how language models incorporate world factuality into their decision making processes. First, I study how neural language models can be used to prove or disprove facts. Motivated by the results, I subsequently study how the choice of training tasks affects the stance detection model. In order to study the acquisition of harmful knowledge, I build a dataset to probe models for their societal stereotypes. Finally, I extend this evaluation to a generative setting, and study how the choice of sampling algorithm affects model factuality. Taken together, this thesis provides a comprehensive analysis of how language models capture world factuality via the pre-training process.
Description
Thesis: M. Eng., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, February, 2021
 
Cataloged from the official PDF of thesis.
 
Includes bibliographical references (pages 107-108).
 
Date issued
2021
URI
https://hdl.handle.net/1721.1/130705
Department
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Publisher
Massachusetts Institute of Technology
Keywords
Electrical Engineering and Computer Science.

Collections
  • Graduate Theses

Browse

All of DSpaceCommunities & CollectionsBy Issue DateAuthorsTitlesSubjectsThis CollectionBy Issue DateAuthorsTitlesSubjects

My Account

Login

Statistics

OA StatisticsStatistics by CountryStatistics by Department
MIT Libraries
PrivacyPermissionsAccessibilityContact us
MIT
Content created by the MIT Libraries, CC BY-NC unless otherwise noted. Notify us about copyright concerns.