Neural language models as psycholinguistic subjects: Representations of syntactic state
Name
N19-1004.pdf
Description
Published version
Size
194.11 KB
Format
Adobe PDF
Checksum (MD5)
24e5910ac075226e1e8f1e1fa8c89634
Author(s)
Qian, Peng
Levy, Roger P
Date Issued
June 2019
Journal
NAACL HLT 2019 - 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies - Proceedings of the Conference
Publisher
Association for Computational Linguistics
Citation
Futrell, Richard et al. “Neural language models as psycholinguistic subjects: Representations of syntactic state.” Paper presented at the Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, Minnesota, June 2-June 7, 2019., Association for Computational Linguistics: 32–42 © 2019 The Author(s)
Version
Final published version
Abstract
We investigate the extent to which the behavior of neural network language models reflects incremental representations of syntactic state. To do so, we employ experimental methodologies which were originally developed in the field of psycholinguistics to study syntactic representation in the human mind. We examine neural network model behavior on sets of artificial sentences containing a variety of syntactically complex structures. These sentences not only test whether the networks have a representation of syntactic state, they also reveal the specific lexical cues that networks use to update these states. We test four models: Two publicly available LSTM sequence models of English (Jozefowicz et al., 2016; Gulordava et al., 2018) trained on large datasets; an RNN Grammar (Dyer et al., 2016) trained on a small, parsed dataset; and an LSTM trained on the same small corpus as the RNNG. We find evidence for basic syntactic state representations in all models, but only the models trained on large datasets are sensitive to subtle lexical cues signalling changes in syntactic state.
MIT Department
Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.18653/V1/N19-1004
