Cited By
View all- Saremi MKhalooei MRastgoo RSabokrou M(2024)ProjanKnowledge-Based Systems10.1016/j.knosys.2024.112565304:COnline publication date: 25-Nov-2024
ACQ: Few-shot Backdoor Defense via Activation Clipping and Quantizing
Pages 5410 - 5418
Abstract
In recent years, deep neural networks(DNNs) have relied on an increasing amount of training samples as the premise of the deployment for real-world scenarios. This gives rise to backdoor attacks, where a small fraction of poisoned data is inserted into the training dataset to manipulate the predictions of DNNs when presented with backdoor inputs. Backdoor attacks pose serious security threats during the prediction stage of DNNs. As a result, there is growing research attention to defend against backdoor attacks. This paper proposes Activation Clipping and Quantizing (ACQ), a novel backdoor elimination module via transforming the intermediate-layer output of DNNs during forward propagation by embedding Clipper and Quantizer into the backdoored DNNs. ACQ is motivated by the observation that the backdoored DNNs always output abnormally large or small intermediate-layer activations when presented with backdoored samples, eventually leading to the malicious prediction of backdoored DNNs. ACQ modifies backdoored DNNs to keep the intermediate-layer activations in a proper domain and align the forward propagation of backdoored samples with that of clean samples. Besides, we highlight that ACQ has the ability to eliminate the backdoor of DNNs in few-shot even zero-shot scenarios, which requires much fewer or even no clean samples for the backdoor elimination stage than existing approaches. Experiments demonstrate the effectiveness and robustness of ACQ against various attacks and tasks compared to existing methods. Our code and Appendix can be found in https://github.com/Backdoor-defense/ACQ
Supplemental Material
MP4 File
Video Presentation
- Download
- 12.87 MB
References
[1]
Scott Alfeld, Xiaojin Zhu, and Paul Barford. 2016. Data poisoning attacks against autoregressive models. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 30.
[2]
Mauro Barni, Kassem Kallas, and Benedetta Tondi. 2019. A newbackdoor attack in cnns by training set corruption without label poisoning. In 2019 IEEE International Conference on Image Processing (ICIP). IEEE, 101--105.
[3]
Mariusz Bojarski, Davide Del Testa, Daniel Dworakowski, Bernhard Firner, Beat Flepp, Prasoon Goyal, Lawrence D Jackel, Mathew Monfort, Urs Muller, Jiakai Zhang, et al. 2016. End to end learning for self-driving cars. arXiv preprint arXiv:1604.07316 (2016).
[4]
Lin Chen, Yifei Min, Mingrui Zhang, and Amin Karbasi. 2020. More data can expand the generalization gap between adversarially robust and standard models. In International Conference on Machine Learning. PMLR, 1670--1680.
[5]
Weixin Chen, BaoyuanWu, and Haoqian Wang. 2022. Effective backdoor defense by exploiting sensitivity of poisoned samples. Advances in Neural Information Processing Systems 35 (2022), 9727--9737.
[6]
Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, and Dawn Song. 2017. Targeted backdoor attacks on deep learning systems using data poisoning. arXiv preprint arXiv:1712.05526 (2017).
[7]
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).
[8]
Andre Esteva, Brett Kuprel, Roberto A Novoa, Justin Ko, Susan M Swetter, Helen M Blau, and Sebastian Thrun. 2017. Dermatologist-level classification of skin cancer with deep neural networks. nature 542, 7639 (2017), 115--118.
[9]
Mingtao Feng, Haoran Hou, Liang Zhang, Yulan Guo, Hongshan Yu, Yaonan Wang, and Ajmal Mian. 2023. Exploring Hierarchical Spatial Layout Cues for 3D Point Cloud based Scene Graph Prediction. IEEE Transactions on Multimedia (2023).
[10]
Tianyu Gu, Brendan Dolan-Gavitt, and Siddharth Garg. 2017. Badnets: Identifying vulnerabilities in the machine learning model supply chain. arXiv preprint arXiv:1708.06733 (2017).
[11]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.
[12]
Kunzhe Huang, Yiming Li, Baoyuan Wu, Zhan Qin, and Kui Ren. 2022. Backdoor defense via decoupling the training process. arXiv preprint arXiv:2202.03423 (2022).
[13]
Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning. pmlr, 448--456.
[14]
Yong Rae Jo, Youngki Moon, Minji Jung, Jungyoon Choi, Jihyung Moon, and Won Ik Cho. 2021. VUS at IWSLT 2021: A Finetuned Pipeline for Offline Speech Translation. In Proceedings of the 18th International Conference on Spoken Language Translation (IWSLT 2021). 120--124.
[15]
Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al. 2021. Advances and open problems in federated learning. Foundations and Trends® in Machine Learning 14, 1--2 (2021), 1--210.
[16]
Kshitiz Kumar, Chaojun Liu, Kaisheng Yao, and Yifan Gong. 2015. Intermediate layer DNN adaptation for offline and session-based iterative speaker adaptation. In Sixteenth Annual Conference of the International Speech Communication Association.
[17]
Yige Li, Xixiang Lyu, Nodens Koren, Lingjuan Lyu, Bo Li, and Xingjun Ma. 2021. Anti-backdoor learning: Training clean models on poisoned data. Advances in Neural Information Processing Systems 34 (2021), 14900--14912.
[18]
Yige Li, Xixiang Lyu, Nodens Koren, Lingjuan Lyu, Bo Li, and Xingjun Ma. 2021. Neural attention distillation: Erasing backdoor triggers from deep neural networks. arXiv preprint arXiv:2101.05930 (2021).
[19]
Haowen Lin, Jian Lou, Li Xiong, and Cyrus Shahabi. 2021. Integer-arithmeticonly certified robustness for quantized neural networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 7828--7837.
[20]
Shen Lin, Xiaoyu Zhang, Chenyang Chen, Xiaofeng Chen, and Willy Susilo. 2023. ERM-KTP: Knowledge-Level Machine Unlearning via Knowledge Transfer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 20147--20155.
[21]
Kang Liu, Brendan Dolan-Gavitt, and Siddharth Garg. 2018. Fine-pruning: Defending against backdooring attacks on deep neural networks. In Research in Attacks, Intrusions, and Defenses: 21st International Symposium, RAID 2018, Heraklion, Crete, Greece, September 10-12, 2018, Proceedings 21. Springer, 273--294.
[22]
Yingqi Liu, Shiqing Ma, Yousra Aafer, Wen-Chuan Lee, Juan Zhai, Weihang Wang, and Xiangyu Zhang. 2018. Trojaning attack on neural networks. In 25th Annual Network And Distributed System Security Symposium (NDSS 2018). Internet Soc.
[23]
Yunfei Liu, Xingjun Ma, James Bailey, and Feng Lu. 2020. Reflection backdoor: A natural backdoor attack on deep neural networks. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23--28, 2020, Proceedings, Part X 16. Springer, 182--199.
[24]
Yongxin Liu, JianWang, Shuteng Niu, and Houbing Song. 2020. Deep learning enabled reliable identity verification and spoofing detection. In Wireless Algorithms, Systems, and Applications: 15th International Conference, WASA 2020, Qingdao, China, September 13--15, 2020, Proceedings, Part I 15. Springer, 333--345.
[25]
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. 2021. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision. 10012--10022.
[26]
Xu Ma, Xiaofeng Chen, and Xiaoyu Zhang. 2019. Non-interactive privacypreserving neural network prediction. Information Sciences 481 (2019), 507--519.
[27]
Anh Nguyen and Anh Tran. 2021. Wanet-imperceptible warping-based backdoor attack. arXiv preprint arXiv:2102.10369 (2021).
[28]
Sikha Pentyala, Rafael Dowsley, and Martine De Cock. 2021. Privacy-preserving video classification with convolutional neural networks. In International conference on machine learning. PMLR, 8487--8499.
[29]
Pengfei Tang, Wenjie Wang, Jian Lou, and Li Xiong. 2021. Generating adversarial examples with distance constrained adversarial imitation networks. IEEE Transactions on Dependable and Secure Computing 19, 6 (2021), 4145--4155.
[30]
Han Tian, Chaoliang Zeng, Zhenghang Ren, Di Chai, Junxue Zhang, Kai Chen, and Qiang Yang. 2022. Sphinx: Enabling privacy-preserving online learning over the cloud. In 2022 IEEE Symposium on Security and Privacy (SP). IEEE, 2487--2501.
[31]
Alexander Turner, Dimitris Tsipras, and Aleksander Madry. 2018. Clean-label backdoor attacks. (2018).
[32]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ?ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017).
[33]
Dongxian Wu and Yisen Wang. 2021. Adversarial neuron pruning purifies backdoored deep models. Advances in Neural Information Processing Systems 34 (2021), 16913--16925.
[34]
Guangxuan Xiao, Ji Lin, and Song Han. 2023. Offsite-Tuning: Transfer Learning without Full Model. arXiv preprint arXiv:2302.04870 (2023).
[35]
Huang Xiao, Battista Biggio, Gavin Brown, Giorgio Fumera, Claudia Eckert, and Fabio Roli. 2015. Is feature selection secure against training data poisoning?. In international conference on machine learning. PMLR, 1689--1698.
[36]
Jing Xu and Stjepan Picek. 2022. Poster: Clean-label Backdoor Attack on Graph Neural Networks. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security. 3491--3493.
[37]
Yuanshun Yao, Huiying Li, Haitao Zheng, and Ben Y Zhao. 2019. Latent backdoor attacks on deep neural networks. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security. 2041--2055.
[38]
Sergey Zagoruyko and Nikos Komodakis. 2016. Wide residual networks. arXiv preprint arXiv:1605.07146 (2016).
[39]
Xiaoyu Zhang, Chao Chen, Yi Xie, Xiaofeng Chen, Jun Zhang, and Yang Xiang. 2023. A survey on privacy inference attacks and defenses in cloud-based Deep Neural Network. Computer Standards & Interfaces 83 (2023), 103672.
[40]
Xiaoyu Zhang, Yulin Jin, Tao Wang, Jian Lou, and Xiaofeng Chen. 2022. Purifier: Plug-and-play Backdoor Mitigation for Pre-trained Models Via Anomaly Activation Suppression. In Proceedings of the 30th ACM International Conference on Multimedia. 4291--4299.
[41]
Pu Zhao, Pin-Yu Chen, Payel Das, Karthikeyan Natesan Ramamurthy, and Xue Lin. 2020. Bridging mode connectivity in loss landscapes and adversarial robustness. arXiv preprint arXiv:2005.00060 (2020).
Index Terms
- ACQ: Few-shot Backdoor Defense via Activation Clipping and Quantizing
Recommendations
Purifier: Plug-and-play Backdoor Mitigation for Pre-trained Models Via Anomaly Activation Suppression
MM '22: Proceedings of the 30th ACM International Conference on MultimediaPre-trained models have been widely adopted in deep learning development, benefiting the fine-tuning of downstream user-specific tasks with enormous computation saving. However, backdoor attacks pose severe security threat to the subsequent models built ...
Imperceptible and multi-channel backdoor attack
AbstractRecent researches demonstrate that Deep Neural Networks (DNN) models are vulnerable to backdoor attacks. The backdoored DNN model will behave maliciously when images containing backdoor triggers arrive. To date, almost all the existing backdoor ...
TargetNet Backdoor: Attack on Deep Neural Network with Use of Different Triggers
ICIIT '20: Proceedings of the 2020 5th International Conference on Intelligent Information TechnologyDeep neural networks (DNNs) provide good performance in image recognition, speech recognition, and pattern analysis. However, DNNs are vulnerable to backdoor attacks. Backdoor attacks allow attackers to proactively access DNN training data to train it on ...
Comments
Information & Contributors
Information
Published In
October 2023
9913 pages
ISBN:9798400701085
DOI:10.1145/3581783
- General Chairs:
- Abdulmotaleb El Saddik,
- Tao Mei,
- Rita Cucchiara,
- Program Chairs:
- Marco Bertini,
- Diana Patricia Tobon Vallejo,
- Pradeep K. Atrey,
- M. Shamim Hossain
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 27 October 2023
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- NFSC
Conference
MM '23
Sponsor:
MM '23: The 31st ACM International Conference on Multimedia
October 29 - November 3, 2023
Ottawa ON, Canada
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 273Total Downloads
- Downloads (Last 12 months)158
- Downloads (Last 6 weeks)18
Reflects downloads up to 05 Mar 2025
Other Metrics
Citations
Cited By
View all- Saremi MKhalooei MRastgoo RSabokrou M(2024)ProjanKnowledge-Based Systems10.1016/j.knosys.2024.112565304:COnline publication date: 25-Nov-2024
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in