Neural language models and human linguistic knowledge
Name
hu-jennhu-phd-bcs-2023-thesis.pdf
Description
Thesis PDF
Size
1.31 MB
Format
Adobe PDF
Checksum (MD5)
f809b33e298c2bcf5a3948ce51aee489
Author(s)
Hu, Jennifer
Advisor(s)
Levy, Roger P.
Date Issued
June 2023
Publisher
Massachusetts Institute of Technology
Abstract
Language is one of the hallmarks of intelligence, demanding explanation in a theory of human cognition. However, language presents unique practical challenges for quantitative empirical research, making many linguistic theories difficult to test at naturalistic scales. Artificial neural network language models (LMs) provide a new tool for studying language with mathematical precision and control, as they exhibit remarkably sophisticated linguistic behaviors while being fully intervenable. While LMs differ from humans in many ways, the learning outcomes of these models can reveal the behaviors that may emerge through expressive statistical learning algorithms applied to linguistic input.
In this thesis, I demonstrate this approach through three case studies using LMs to investigate open questions in language acquisition and comprehension. First, I use LMs to perform controlled manipulations of language learning, and find that syntactic generalizations depend more on a learner's inductive bias than on training data size. Second, I use LMs to explain systematic variation in scalar inferences by approximating human listeners' expectations over unspoken alternative sentences (e.g., "The bill was supported overwhelmingly" implies that the bill was not supported unanimously). Finally, I show that LMs and humans exhibit similar behaviors on a set of non-literal comprehension tasks which are hypothesized to require social reasoning (e.g., inferring a speaker's intended meaning from ironic statements). These findings suggest that certain aspects of linguistic knowledge could emerge through domain-general prediction mechanisms, while other aspects may require specific inductive biases and conceptual structures.
MIT Department
Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences
Terms of Use
Attribution 4.0 International (CC BY 4.0)
Copyright retained by author(s)
Persistent DSpace Link
