Deriving Machine Attention from Human Rationales
Author(s)
Bao, Yujia; Chang, Shiyu; Yu, Mo; Barzilay, Regina
DownloadPublished version (1.184Mb)
Publisher with Creative Commons License
Publisher with Creative Commons License
Creative Commons Attribution
Terms of use
Metadata
Show full item recordAbstract
Attention-based models are successful when trained on large amounts of data. In this paper, we demonstrate that even in the low-resource scenario, attention can be learned effectively. To this end, we start with discrete human-annotated rationales and map them into continuous attention. Our central hypothesis is that this mapping is general across domains, and thus can be transferred from resource-rich domains to low-resource ones. Our model jointly learns a domain-invariant representation and induces the desired mapping between rationales and attention. Our empirical results validate this hypothesis and show that our approach delivers significant gains over state-of-the-art baselines, yielding over 15% average error reduction on benchmark datasets.
Date issued
2018-10Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence LaboratoryJournal
Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
Publisher
Association for Computational Linguistics (ACL)
Citation
Bao, Yujia et al. "Deriving Machine Attention from Human Rationales." Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, October-November 2018, Brussels, Belgium, Association for Computational Linguistics, October 2018. © 2018 Association for Computational Linguistics
Version: Final published version