This is not the latest version of this item. The latest version can be found here.
Towards Debiasing Fact Verification Models
Name
D19-1341.pdf
Description
Published version
Size
464.43 KB
Format
Adobe PDF
Checksum (MD5)
aeb3d8d9d65261272322273ae3e42d92
Author(s)
Schuster, Tal
Shah, Darsh
Yeo, Yun Jie Serene
Roberto Filizzola Ortiz, Daniel
Santus, Enrico
Barzilay, Regina
Date Issued
2019
Journal
EMNLP-IJCNLP 2019 - 2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing, Proceedings of the Conference
Publisher
Association for Computational Linguistics
Citation
Schuster, Tal, Shah, Darsh, Yeo, Yun Jie Serene, Roberto Filizzola Ortiz, Daniel, Santus, Enrico et al. 2019. "Towards Debiasing Fact Verification Models." EMNLP-IJCNLP 2019 - 2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing, Proceedings of the Conference.
Version
Final published version
Abstract
© 2019 Association for Computational Linguistics Fact verification requires validating a claim in the context of evidence. We show, however, that in the popular FEVER dataset this might not necessarily be the case. Claim-only classifiers perform competitively with top evidence-aware models. In this paper, we investigate the cause of this phenomenon, identifying strong cues for predicting labels solely based on the claim, without considering any evidence. We create an evaluation set that avoids those idiosyncrasies. The performance of FEVER-trained models significantly drops when evaluated on this test set. Therefore, we introduce a regularization method which alleviates the effect of bias in the training data, obtaining improvements on the newly created test set. This work is a step towards a more sound evaluation of reasoning capabilities in fact verification models.
Terms of Use
Creative Commons Attribution 4.0 International license
Persistent DSpace Link
DOI of Published Version
10.18653/V1/D19-1341
