Deriving Machine Attention from Human Rationales
Name
D18-1216.pdf
Description
Published version
Size
1.18 MB
Format
Adobe PDF
Checksum (MD5)
66d355e2de7b25ee6815006a169beb00
Author(s)
Bao, Yujia
Chang, Shiyu
Yu, Mo
Barzilay, Regina
Date Issued
October 2018
Journal
Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
Publisher
Association for Computational Linguistics (ACL)
Citation
Bao, Yujia et al. "Deriving Machine Attention from Human Rationales." Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, October-November 2018, Brussels, Belgium, Association for Computational Linguistics, October 2018. © 2018 Association for Computational Linguistics
Version
Final published version
Abstract
Attention-based models are successful when trained on large amounts of data. In this paper, we demonstrate that even in the low-resource scenario, attention can be learned effectively. To this end, we start with discrete human-annotated rationales and map them into continuous attention. Our central hypothesis is that this mapping is general across domains, and thus can be transferred from resource-rich domains to low-resource ones. Our model jointly learns a domain-invariant representation and induces the desired mapping between rationales and attention. Our empirical results validate this hypothesis and show that our approach delivers significant gains over state-of-the-art baselines, yielding over 15% average error reduction on benchmark datasets.
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Terms of Use
Creative Commons Attribution 4.0 International license
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.18653/v1/d18-1216
