Red Alarm for Pre-trained Models: Universal Vulnerability to Neuron-level Backdoor Attacks
Name
11633_2022_Article_1377.pdf
Size
944.48 KB
Format
Adobe PDF
Checksum (MD5)
22ff0d7f359b2b8a20aca6bc2c5f0d32
Author(s)
Zhang, Zhengyan
Xiao, Guangxuan
Li, Yongwei
Lv, Tian
Qi, Fanchao
Liu, Zhiyuan
Wang, Yasheng
Jiang, Xin
Sun, Maosong
Date Issued
March 2, 2023
Journal
Machine Intelligence Research
Publisher
Springer Science and Business Media LLC
Citation
Zhang, Z., Xiao, G., Li, Y. et al. Red Alarm for Pre-trained Models: Universal Vulnerability to Neuron-level Backdoor Attacks. Mach. Intell. Res. 20, 180–193 (2023).
Version
Final published version
Abstract
The pre-training-then-fine-tuning paradigm has been widely used in deep learning. Due to the huge computation cost for pre-training, practitioners usually download pre-trained models from the Internet and fine-tune them on downstream datasets, while the downloaded models may suffer backdoor attacks. Different from previous attacks aiming at a target task, we show that a backdoored pre-trained model can behave maliciously in various downstream tasks without foreknowing task information. Attackers can restrict the output representations (the values of output neurons) of trigger-embedded samples to arbitrary predefined values through additional training, namely neuron-level backdoor attack (NeuBA). Since fine-tuning has little effect on model parameters, the fine-tuned model will retain the backdoor functionality and predict a specific label for the samples embedded with the same trigger. To provoke multiple labels in a specific task, attackers can introduce several triggers with predefined contrastive values. In the experiments of both natural language processing (NLP) and computer vision (CV), we show that NeuBA can well control the predictions for trigger-embedded instances with different trigger designs. Our findings sound a red alarm for the wide use of pre-trained models. Finally, we apply several defense methods to NeuBA and find that model pruning is a promising technique to resist NeuBA by omitting backdoored neurons.
Terms of Use
Creative Commons Attribution
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1007/s11633-022-1377-5
