Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment
Name
6311-Article Text-9536-1-10-20200516.pdf
Description
Published version
Size
503.74 KB
Format
Adobe PDF
Checksum (MD5)
38c6ccda29482b57867332cf4d2f05b2
Author(s)
Jin, Di
Jin, Zhijing
Zhou, Joey Tianyi
Szolovits, Peter
Date Issued
2020
Journal
Proceedings of the AAAI Conference on Artificial Intelligence
Publisher
Association for the Advancement of Artificial Intelligence (AAAI)
Citation
Jin, Di, Jin, Zhijing, Zhou, Joey Tianyi and Szolovits, Peter. 2020. "Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment." Proceedings of the AAAI Conference on Artificial Intelligence, 34 (05).
Version
Final published version
Abstract
Machine learning algorithms are often vulnerable to adversarial examples that have imperceptible alterations from the original counterparts but can fool the state-of-the-art models. It is helpful to evaluate or even improve the robustness of these models by exposing the maliciously crafted adversarial examples. In this paper, we present TextFooler, a simple but strong baseline to generate adversarial text. By applying it to two fundamental natural language tasks, text classification and textual entailment, we successfully attacked three target models, including the powerful pre-trained BERT, and the widely used convolutional and recurrent neural networks. We demonstrate three advantages of this framework: (1) effective—it outperforms previous attacks by success rate and perturbation rate, (2) utility-preserving—it preserves semantic content, grammaticality, and correct types classified by humans, and (3) efficient—it generates adversarial text with computational complexity linear to the text length.1
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Terms of Use
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use.
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1609/AAAI.V34I05.6311
