Abstract
Adversarial patches can disrupt computer vision systems, seriously threatening people’s lives and property security. Existing defense methods seldom consider the generalization for defending against different patches and the compatibility with various models. Furthermore, the severe security situation necessitates the combination of data defense and model defense to build a comprehensive defense system. To address these issues, we propose a defense method named PatchBreaker, which consists of three components. In data defense, PatchBreaker uses the Semantic-Cutter trained by annotated patch images to cut patches and output incomplete images. Next, the Image-Inpainter trained by clean-incomplete image pairs is used to inpaint these incomplete images and output inpainted images. In model defense, the Adversaril-Classifier will be trained by joint adversarial training with clean images and patch images. Finally, PatchBreaker inputs inpainted images into Adversarial-Classifier to output correct results. Comparative experiments show that PatchBreaker outperforms other comparative defense methods in most cases, which indicates the excellent patch generalization and model compatibility of PatchBreaker. Meanwhile, ablation studies show the effectiveness of combining data defense and model defense. Additionally, PatchBreaker has minimal impact on the clean accuracy (about 1\(\%\)).
Graphical abstract
The background, mechanisms and defense effectiveness of PatchBreaker. Intelligent systems are easily disrupted by adversarial patches. Therefore, we propose the PatchBreaker, which consists of data defense and model defense. In data defense, the Semantic-Cutter cuts patches by BiseNetV2 model and output incomplete images, then the Image-Inpainter inpaints incomplete images by a bilateral image inpainting model. In model defense, we utilize clean and patch images to train the Adversarial-Classifier for classifying inpainted images to output correct results. After defense, the green high-light regions of integrated gradients attribution are more regular, which indicates the effectiveness of PatchBreaker











Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Availability of data and materials
Data will be made available on reasonable request.
References
Ren J, Shi M, Chen J, Wang R, Wang X (2022) Hyperspectral image classification using multi-level features fusion capsule network with a dense structure. Appl Intell pp 1–20
Tong K, Wu Y (2022) Deep learning-based detection from the perspective of small or tiny objects: A survey. Image Vis Comput 104471
Jhaldiyal A, Chaudhary N (2022) Semantic segmentation of 3d lidar data using deep learning: a review of projection-based methods. Appl Intell pp 1–12
Wang J, Wang C, Lin Q, Luo C, Wu C, Li J (2022) Adversarial attacks and defenses in deep learning for image recognition: A survey. Neurocomputing
Zhang B, Tondi B, Barni M (2020) Adversarial examples for replay attacks against cnn-based face recognition with anti-spoofing capability. Comput Vis Image Underst 197 102988
Wang Z, Guo Y, Zuo W (2022) Deepfake forensics via an adversarial game. IEEE Trans Image Process 31:3541–3552
Zhang Q, Hu S, Sun J, Chen QA, Mao ZM (2022) On adversarial robustness of trajectory prediction for autonomous vehicles. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, IEEE, pp 15159–15168
Naseer M, Khan S, Porikli F (2019) Local gradients smoothing: Defense against localized adversarial attacks. In: 2019 IEEE Winter conference on applications of computer vision (WACV), IEEE, pp 1300–1307
Hayes J (2018) On visible adversarial perturbations & digital watermarking. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, IEEE, pp 1597–1604
Xu Z, Yu F, Chen X (2020) Lance: A comprehensive and lightweight cnn defense methodology against physical adversarial attacks on embedded multimedia applications. In: 2020 25th Asia and South Pacific design automation conference (ASP-DAC), IEEE, pp 470–475
Chou E, Tramer F, Pellegrino G (2020) Sentinet: Detecting localized universal attacks against deep learning systems. In: 2020 IEEE Security and privacy workshops (SPW), IEEE, pp 48–54
Chen Z, Dash P, Pattabiraman K (2023) Jujutsu: A two-stage defense against adversarial patch attacks on deep neural networks. In: Proceedings of the 2023 ACM Asia conference on computer and communications security, ACM, pp 689–703
Yin L, Wang S, Wang Z, Wang C, Zhan D (2024) Attribution guided purification against adversarial patch. Displays 83:102720
Liu J, Levine A, Lau CP, Chellappa R, Feizi S (2022) Segment and complete: Defending object detectors against adversarial patch attacks with robust patch detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, IEEE, pp 14973–14982
Xu K, Xiao Y, Zheng Z, Cai K, Nevatia R (2023) Patchzero: Defending against adversarial patch attacks by detecting and zeroing the patch. In: Proceedings of the IEEE/CVF winter conference on applications of computer vision, IEEE, pp 4632–4641
Tarchoun B, Ben Khalifa A, Mahjoub MA, Abu-Ghazaleh N, Alouani I (2023) Jedi: entropy-based localization and removal of adversarial patches. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, IEEE, pp 4087–4095
Rao S, Stutz D, Schiele B (2020) Adversarial training against location-optimized adversarial patches. In: European conference on computer vision, Springer, pp 429–448
Gittings T, Schneider S, Collomosse J (2020) Vax-a-net: Training-time defence against adversarial patch attacks. In: Proceedings of the Asian conference on computer vision. AFCV
Metzen JH, Finnie N, Hutmacher R (2021) Meta adversarial training against universal patches. In: ICML 2021 Workshop on adversarial machine learning. IMLS
Zhang Z, Yuan B, McCoyd M, Wagner D (2020) Clipped bagnet: Defending against sticker attacks with clipped bag-of-features. In: 2020 IEEE Security and privacy workshops (SPW), IEEE, pp 55–61
Xiang C, Bhagoji AN, Sehwag V, Mittal P (2021) \(\{\)PatchGuard\(\}\): A provably robust defense against adversarial patches via small receptive fields and masking. In: 30th USENIX security symposium (USENIX security 21), pp 2237–2254. USENIX
Yu C, Chen J, Xue Y, Liu Y, Wan W, Bao J, Ma H (2021) Defending against universal adversarial patches by clipping feature norms. In: Proceedings of the IEEE/CVF international conference on computer vision, IEEE, pp 16434–16442
Xiang C, Mahloujifar S, Mittal P (2022) \(\{\)PatchCleanser\(\}\): Certifiably robust defense against adversarial patches for any image classifier. In: 31st USENIX security symposium (USENIX Security 22), pp 2065–2082. USENIX
Salman H, Jain S, Wong E, Madry A (2022) Certified patch robustness via smoothed vision transformers. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, IEEE, pp 15137–15147
Brown TB, Mané D, Roy A, Abadi M, Gilmer J (2017) Adversarial patch. In: Conference and workshop on neural information processing systems. NIPS
Karmon D, Zoran D, Goldberg Y (2018) Lavan: Localized and visible adversarial noise. In: International conference on machine learning, pp 2507–2515. PMLR
Eykholt K, Evtimov I, Fernandes E, Li B, Rahmati A, Xiao C, Prakash A, Kohno T, Song D (2018) Robust physical-world attacks on deep learning visual classification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, IEEE, pp 1625–1634
Subramanya A, Pillai V, Pirsiavash H (2019) Fooling network interpretation in image classification. In: Proceedings of the IEEE/CVF international conference on computer vision, IEEE, pp 2020–2029
Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D (2017) Grad-cam: Visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE international conference on computer vision, IEEE, pp 618–626
Chindaudom A, Siritanawan P, Sumongkayothin K, Kotani K (2020) Adversarialqr: An adversarial patch in qr code format. In: 2020 Joint 9th international conference on informatics, electronics & vision (ICIEV) and 2020 4th international conference on imaging, vision & pattern recognition (icIVPR), IEEE, pp 1–6
Zhou X, Pan Z, Duan Y, Zhang J, Wang S (2021) A data independent approach to generate adversarial patches. Mach Vis Appl 32(3):1–9
Yang C, Kortylewski A, Xie C, Cao Y, Yuille A (2020) Patchattack: A black-box texture-based attack with reinforcement learning. In: European conference on computer vision, Springer, pp 681–698
Liu A, Liu X, Fan J, Ma Y, Zhang A, Xie H, Tao D (2019) Perceptual-sensitive gan for generating adversarial patches. In: Proceedings of the AAAI conference on artificial intelligence, AAAI, vol 33, pp 1028–1035
Liu X, Yang H, Liu Z, Song L, Li H, Chen Y (2019) Dpatch: An adversarial patch attack on object detectors. In: AAAI Workshop on artificial intelligence safety (SafeAI 2019) AAAI
Lee M, Kolter Z (2019) On physical adversarial patches for object detection. In: ICML 2019 workshop on security and privacy of machine learning. IMLS
Madry A, Makelov A, Schmidt L, Tsipras D, Vladu A (2018) Towards deep learning models resistant to adversarial attacks. In: International conference on learning representations. ICLR
Huang H, Wang Y, Chen Z, Tang Z, Zhang W, Ma K-K (2021) Rpattack: Refined patch attack on general object detectors. In: 2021 IEEE International Conference on Multimedia and Expo (ICME), IEEE, pp 1–6
Lei X, Cai X, Lu C, Jiang Z, Gong Z, Lu L (2022) Using frequency attention to make adversarial patch powerful against person detector. IEEE Access 11:27217–27225
Yang X, Wei F, Zhang H, Zhu J (2020) Design and interpretation of universal adversarial patches in face detection. In: European conference on computer vision, Springer, pp 174–191
Hu Y-C-T, Kung B-H, Tan DS, Chen J-C, Hua K-L, Cheng W-H (2021) Naturalistic physical adversarial patch for object detectors. In: Proceedings of the IEEE/CVF international conference on computer vision, IEEE, pp 7848–7857
Yu C, Gao C, Wang J, Yu G, Shen C, Sang N (2021) Bisenet v2: Bilateral network with guided aggregation for real-time semantic segmentation. Int J Comput Vision 129(11):3051–3068
Guo X, Yang H, Huang D (2021) Image inpainting via conditional texture and structure dual generation. In: Proceedings of the IEEE/CVF international conference on computer vision, IEEE, pp 14134–14143
Isola P, Zhu J-Y, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, IEEE, pp 1125–1134
Sundararajan M, Taly A, Yan Q (2017) Axiomatic attribution for deep networks. In: International conference on machine learning, pp 3319–3328. PMLR
Goodfellow IJ, Shlens J, Szegedy C (2015) Explaining and harnessing adversarial examples. In: International conference on learning representations, ICLR
Acknowledgements
This paper was supported by the National Natural Science Foundation of China (no. 62072106), General Project of Natural Science Foundation in Fujian Province (no. 2020J01168), and Open Project of Fujian Key Laboratory of Severe Weather (no. 2020KFKT04). Meanwhile, thanks to Xiao Zhenjie for the help provided in this paper, including implementing the comparison experiments of Jedi, SAC, and Pix2Pix.
Author information
Authors and Affiliations
Contributions
Shiyu Huang: Conceptualization, Methodology, Investigation, Software - Attack & Adversarial-Classifier, Writing - Original draft. Feng Ye: Conceptualization, Methodology, Supervision, Writing - Review & Editing. Zuchao Huang: Software - Semantic-Cutter & Integrated-Gradients, Writing - Review & Editing. Wei Li: Software - Image-Inpainter, Writing - Review & Editing. Tianqiang Huang: Supervision, Funding acquisition, Resources - Computing resources. Liqing Huang: Methodology, Funding acquisition, Writing - Review & Editing.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Huang, S., Ye, F., Huang, Z. et al. PatchBreaker: defending against adversarial attacks by cutting-inpainting patches and joint adversarial training. Appl Intell 54, 10819–10832 (2024). https://doi.org/10.1007/s10489-024-05735-0
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-024-05735-0