Single Word Change Is All You Need: Using LLMs to Create Synthetic Training Examples for Text Classifiers
Name
Expert Systems - 2025 - Xu - Single Word Change Is All You Need Using LLMs to Create Synthetic Training Examples for Text.pdf
Description
Published version
Size
1.01 MB
Format
Adobe PDF
Checksum (MD5)
5e57bc79a340579e991ce7c3a10c2428
Author(s)
Xu, Lei
Alnegheimish, Sarah
Berti‐Equille, Laure
Cuesta‐Infante, Alfredo
Veeramachaneni, Kalyan
Date Issued
July 7, 2025
Journal
Expert Systems
Publisher
Wiley
Citation
Xu, L., S. Alnegheimish, L. Berti-Equille, A. Cuesta-Infante, and K. Veeramachaneni. 2025. “ Single Word Change Is All You Need: Using LLMs to Create Synthetic Training Examples for Text Classifiers.” Expert Systems 42, no. 8: e70079.
Version
Final published version
Abstract
In text classification, creating an adversarial example means subtly perturbing a few words in a sentence without changing itsmeaning, causing it to be misclassified by a classifier. A concerning observation is that a significant portion of adversarial exam-ples generated by existing methods change only one word. This single-word perturbation vulnerability represents a significantweakness in classifiers, which malicious users can exploit to efficiently create a multitude of adversarial examples. This paperstudies this problem and makes the following key contributions: (1) We introduce a novel metric 𝜌 to quantitatively assess a clas-sifier's robustness against single-word perturbation. (2) We present the SP-Attack, designed to exploit the single-word perturbationvulnerability, achieving a higher attack success rate, better preserving sentence meaning, while reducing computation costscompared to state-of-the-art adversarial methods. (3) We propose SP-Defence, which aims to improve 𝜌 by applying data augmen-tation in learning. Experimental results on 4 datasets and 2 masked language models show that SP-Defence improves 𝜌 by 14.6%and 13.9% and decreases the attack success rate of SP-Attack by 30.4% and 21.2% on two classifiers respectively, and decreasesthe attack success rate of existing attack methods that involve multiple-word perturbation.
MIT Department
Massachusetts Institute of Technology. Laboratory for Information and Decision Systems
Terms of Use
Creative Commons Attribution
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1111/exsy.70079
