Investigating Novel Verb Learning in BERT: Selectional Preference Classes and Alternation-Based Syntactic Generalization
Name
2020.blackboxnlp-1.25.pdf
Description
Published version
Size
313.79 KB
Format
Adobe PDF
Checksum (MD5)
7905f8a7bd4a56e4317f5d67e1bf7f07
Author(s)
Thrush, Tristan
Wilcox, Ethan
Levy, Roger
Date Issued
2020
Journal
Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP
Publisher
Association for Computational Linguistics
Citation
Thrush, Tristan, Wilcox, Ethan and Levy, Roger. 2020. "Investigating Novel Verb Learning in BERT: Selectional Preference Classes and Alternation-Based Syntactic Generalization." Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP.
Version
Final published version
Abstract
Previous studies investigating the syntactic
abilities of deep learning models have not
targeted the relationship between the strength
of the grammatical generalization and the
amount of evidence to which the model is
exposed during training. We address this issue
by deploying a novel word-learning paradigm
to test BERT’s (Devlin et al., 2018) few-shot
learning capabilities for two aspects of English
verbs: alternations and classes of selectional
preferences. For the former, we fine-tune
BERT on a single frame in a verbal-alternation
pair and ask whether the model expects the
novel verb to occur in its sister frame. For the
latter, we fine-tune BERT on an incomplete
selectional network of verbal objects and ask
whether it expects unattested but plausible
verb/object pairs. We find that BERT makes
robust grammatical generalizations after just
one or two instances of a novel word in
fine-tuning. For the verbal alternation tests,
we find that the model displays behavior that
is consistent with a transitivity bias: verbs
seen few times are expected to take direct
objects, but verbs seen with direct objects
are not expected to occur intransitively. The
code for our experiments is available at
https://github.com/TristanThrush/
few-shot-lm-learning.
MIT Department
Massachusetts Institute of Technology. Department of Brain and Cognitive Sciences
Terms of Use
Creative Commons Attribution 4.0 International license
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.18653/V1/2020.BLACKBOXNLP-1.25
