Skip to main content

Advertisement

Log in

PatchBreaker: defending against adversarial attacks by cutting-inpainting patches and joint adversarial training

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Adversarial patches can disrupt computer vision systems, seriously threatening people’s lives and property security. Existing defense methods seldom consider the generalization for defending against different patches and the compatibility with various models. Furthermore, the severe security situation necessitates the combination of data defense and model defense to build a comprehensive defense system. To address these issues, we propose a defense method named PatchBreaker, which consists of three components. In data defense, PatchBreaker uses the Semantic-Cutter trained by annotated patch images to cut patches and output incomplete images. Next, the Image-Inpainter trained by clean-incomplete image pairs is used to inpaint these incomplete images and output inpainted images. In model defense, the Adversaril-Classifier will be trained by joint adversarial training with clean images and patch images. Finally, PatchBreaker inputs inpainted images into Adversarial-Classifier to output correct results. Comparative experiments show that PatchBreaker outperforms other comparative defense methods in most cases, which indicates the excellent patch generalization and model compatibility of PatchBreaker. Meanwhile, ablation studies show the effectiveness of combining data defense and model defense. Additionally, PatchBreaker has minimal impact on the clean accuracy (about 1\(\%\)).

Graphical abstract

The background, mechanisms and defense effectiveness of PatchBreaker. Intelligent systems are easily disrupted by adversarial patches. Therefore, we propose the PatchBreaker, which consists of data defense and model defense. In data defense, the Semantic-Cutter cuts patches by BiseNetV2 model and output incomplete images, then the Image-Inpainter inpaints incomplete images by a bilateral image inpainting model. In model defense, we utilize clean and patch images to train the Adversarial-Classifier for classifying inpainted images to output correct results. After defense, the green high-light regions of integrated gradients attribution are more regular, which indicates the effectiveness of PatchBreaker

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Availability of data and materials

Data will be made available on reasonable request.

References

  1. Ren J, Shi M, Chen J, Wang R, Wang X (2022) Hyperspectral image classification using multi-level features fusion capsule network with a dense structure. Appl Intell pp 1–20

  2. Tong K, Wu Y (2022) Deep learning-based detection from the perspective of small or tiny objects: A survey. Image Vis Comput 104471

  3. Jhaldiyal A, Chaudhary N (2022) Semantic segmentation of 3d lidar data using deep learning: a review of projection-based methods. Appl Intell pp 1–12

  4. Wang J, Wang C, Lin Q, Luo C, Wu C, Li J (2022) Adversarial attacks and defenses in deep learning for image recognition: A survey. Neurocomputing

  5. Zhang B, Tondi B, Barni M (2020) Adversarial examples for replay attacks against cnn-based face recognition with anti-spoofing capability. Comput Vis Image Underst 197 102988

  6. Wang Z, Guo Y, Zuo W (2022) Deepfake forensics via an adversarial game. IEEE Trans Image Process 31:3541–3552

    Article  Google Scholar 

  7. Zhang Q, Hu S, Sun J, Chen QA, Mao ZM (2022) On adversarial robustness of trajectory prediction for autonomous vehicles. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, IEEE, pp 15159–15168

  8. Naseer M, Khan S, Porikli F (2019) Local gradients smoothing: Defense against localized adversarial attacks. In: 2019 IEEE Winter conference on applications of computer vision (WACV), IEEE, pp 1300–1307

  9. Hayes J (2018) On visible adversarial perturbations & digital watermarking. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, IEEE, pp 1597–1604

  10. Xu Z, Yu F, Chen X (2020) Lance: A comprehensive and lightweight cnn defense methodology against physical adversarial attacks on embedded multimedia applications. In: 2020 25th Asia and South Pacific design automation conference (ASP-DAC), IEEE, pp 470–475

  11. Chou E, Tramer F, Pellegrino G (2020) Sentinet: Detecting localized universal attacks against deep learning systems. In: 2020 IEEE Security and privacy workshops (SPW), IEEE, pp 48–54

  12. Chen Z, Dash P, Pattabiraman K (2023) Jujutsu: A two-stage defense against adversarial patch attacks on deep neural networks. In: Proceedings of the 2023 ACM Asia conference on computer and communications security, ACM, pp 689–703

  13. Yin L, Wang S, Wang Z, Wang C, Zhan D (2024) Attribution guided purification against adversarial patch. Displays 83:102720

  14. Liu J, Levine A, Lau CP, Chellappa R, Feizi S (2022) Segment and complete: Defending object detectors against adversarial patch attacks with robust patch detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, IEEE, pp 14973–14982

  15. Xu K, Xiao Y, Zheng Z, Cai K, Nevatia R (2023) Patchzero: Defending against adversarial patch attacks by detecting and zeroing the patch. In: Proceedings of the IEEE/CVF winter conference on applications of computer vision, IEEE, pp 4632–4641

  16. Tarchoun B, Ben Khalifa A, Mahjoub MA, Abu-Ghazaleh N, Alouani I (2023) Jedi: entropy-based localization and removal of adversarial patches. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, IEEE, pp 4087–4095

  17. Rao S, Stutz D, Schiele B (2020) Adversarial training against location-optimized adversarial patches. In: European conference on computer vision, Springer, pp 429–448

  18. Gittings T, Schneider S, Collomosse J (2020) Vax-a-net: Training-time defence against adversarial patch attacks. In: Proceedings of the Asian conference on computer vision. AFCV

  19. Metzen JH, Finnie N, Hutmacher R (2021) Meta adversarial training against universal patches. In: ICML 2021 Workshop on adversarial machine learning. IMLS

  20. Zhang Z, Yuan B, McCoyd M, Wagner D (2020) Clipped bagnet: Defending against sticker attacks with clipped bag-of-features. In: 2020 IEEE Security and privacy workshops (SPW), IEEE, pp 55–61

  21. Xiang C, Bhagoji AN, Sehwag V, Mittal P (2021) \(\{\)PatchGuard\(\}\): A provably robust defense against adversarial patches via small receptive fields and masking. In: 30th USENIX security symposium (USENIX security 21), pp 2237–2254. USENIX

  22. Yu C, Chen J, Xue Y, Liu Y, Wan W, Bao J, Ma H (2021) Defending against universal adversarial patches by clipping feature norms. In: Proceedings of the IEEE/CVF international conference on computer vision, IEEE, pp 16434–16442

  23. Xiang C, Mahloujifar S, Mittal P (2022) \(\{\)PatchCleanser\(\}\): Certifiably robust defense against adversarial patches for any image classifier. In: 31st USENIX security symposium (USENIX Security 22), pp 2065–2082. USENIX

  24. Salman H, Jain S, Wong E, Madry A (2022) Certified patch robustness via smoothed vision transformers. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, IEEE, pp 15137–15147

  25. Brown TB, Mané D, Roy A, Abadi M, Gilmer J (2017) Adversarial patch. In: Conference and workshop on neural information processing systems. NIPS

  26. Karmon D, Zoran D, Goldberg Y (2018) Lavan: Localized and visible adversarial noise. In: International conference on machine learning, pp 2507–2515. PMLR

  27. Eykholt K, Evtimov I, Fernandes E, Li B, Rahmati A, Xiao C, Prakash A, Kohno T, Song D (2018) Robust physical-world attacks on deep learning visual classification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, IEEE, pp 1625–1634

  28. Subramanya A, Pillai V, Pirsiavash H (2019) Fooling network interpretation in image classification. In: Proceedings of the IEEE/CVF international conference on computer vision, IEEE, pp 2020–2029

  29. Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D (2017) Grad-cam: Visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE international conference on computer vision, IEEE, pp 618–626

  30. Chindaudom A, Siritanawan P, Sumongkayothin K, Kotani K (2020) Adversarialqr: An adversarial patch in qr code format. In: 2020 Joint 9th international conference on informatics, electronics & vision (ICIEV) and 2020 4th international conference on imaging, vision & pattern recognition (icIVPR), IEEE, pp 1–6

  31. Zhou X, Pan Z, Duan Y, Zhang J, Wang S (2021) A data independent approach to generate adversarial patches. Mach Vis Appl 32(3):1–9

    Article  Google Scholar 

  32. Yang C, Kortylewski A, Xie C, Cao Y, Yuille A (2020) Patchattack: A black-box texture-based attack with reinforcement learning. In: European conference on computer vision, Springer, pp 681–698

  33. Liu A, Liu X, Fan J, Ma Y, Zhang A, Xie H, Tao D (2019) Perceptual-sensitive gan for generating adversarial patches. In: Proceedings of the AAAI conference on artificial intelligence, AAAI, vol 33, pp 1028–1035

  34. Liu X, Yang H, Liu Z, Song L, Li H, Chen Y (2019) Dpatch: An adversarial patch attack on object detectors. In: AAAI Workshop on artificial intelligence safety (SafeAI 2019) AAAI

  35. Lee M, Kolter Z (2019) On physical adversarial patches for object detection. In: ICML 2019 workshop on security and privacy of machine learning. IMLS

  36. Madry A, Makelov A, Schmidt L, Tsipras D, Vladu A (2018) Towards deep learning models resistant to adversarial attacks. In: International conference on learning representations. ICLR

  37. Huang H, Wang Y, Chen Z, Tang Z, Zhang W, Ma K-K (2021) Rpattack: Refined patch attack on general object detectors. In: 2021 IEEE International Conference on Multimedia and Expo (ICME), IEEE, pp 1–6

  38. Lei X, Cai X, Lu C, Jiang Z, Gong Z, Lu L (2022) Using frequency attention to make adversarial patch powerful against person detector. IEEE Access 11:27217–27225

    Article  Google Scholar 

  39. Yang X, Wei F, Zhang H, Zhu J (2020) Design and interpretation of universal adversarial patches in face detection. In: European conference on computer vision, Springer, pp 174–191

  40. Hu Y-C-T, Kung B-H, Tan DS, Chen J-C, Hua K-L, Cheng W-H (2021) Naturalistic physical adversarial patch for object detectors. In: Proceedings of the IEEE/CVF international conference on computer vision, IEEE, pp 7848–7857

  41. Yu C, Gao C, Wang J, Yu G, Shen C, Sang N (2021) Bisenet v2: Bilateral network with guided aggregation for real-time semantic segmentation. Int J Comput Vision 129(11):3051–3068

    Article  Google Scholar 

  42. Guo X, Yang H, Huang D (2021) Image inpainting via conditional texture and structure dual generation. In: Proceedings of the IEEE/CVF international conference on computer vision, IEEE, pp 14134–14143

  43. Isola P, Zhu J-Y, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, IEEE, pp 1125–1134

  44. Sundararajan M, Taly A, Yan Q (2017) Axiomatic attribution for deep networks. In: International conference on machine learning, pp 3319–3328. PMLR

  45. Goodfellow IJ, Shlens J, Szegedy C (2015) Explaining and harnessing adversarial examples. In: International conference on learning representations, ICLR

Download references

Acknowledgements

This paper was supported by the National Natural Science Foundation of China (no. 62072106), General Project of Natural Science Foundation in Fujian Province (no. 2020J01168), and Open Project of Fujian Key Laboratory of Severe Weather (no. 2020KFKT04). Meanwhile, thanks to Xiao Zhenjie for the help provided in this paper, including implementing the comparison experiments of Jedi, SAC, and Pix2Pix.

Author information

Authors and Affiliations

Authors

Contributions

Shiyu Huang: Conceptualization, Methodology, Investigation, Software - Attack & Adversarial-Classifier, Writing - Original draft. Feng Ye: Conceptualization, Methodology, Supervision, Writing - Review & Editing. Zuchao Huang: Software - Semantic-Cutter & Integrated-Gradients, Writing - Review & Editing. Wei Li: Software - Image-Inpainter, Writing - Review & Editing. Tianqiang Huang: Supervision, Funding acquisition, Resources - Computing resources. Liqing Huang: Methodology, Funding acquisition, Writing - Review & Editing.

Corresponding author

Correspondence to Feng Ye.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huang, S., Ye, F., Huang, Z. et al. PatchBreaker: defending against adversarial attacks by cutting-inpainting patches and joint adversarial training. Appl Intell 54, 10819–10832 (2024). https://doi.org/10.1007/s10489-024-05735-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-024-05735-0

Keywords

Navigation