Skip to main content
Springer Nature Link
Log in

Adversarial Training Against Location-Optimized Adversarial Patches

  • Conference paper
  • First Online:
Computer Vision – ECCV 2020 Workshops (ECCV 2020)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12539))

Included in the following conference series:

Abstract

Deep neural networks have been shown to be susceptible to adversarial examples – small, imperceptible changes constructed to cause mis-classification in otherwise highly accurate image classifiers. As a practical alternative, recent work proposed so-called adversarial patches: clearly visible, but adversarially crafted rectangular patches in images. These patches can easily be printed and applied in the physical world. While defenses against imperceptible adversarial examples have been studied extensively, robustness against adversarial patches is poorly understood. In this work, we first devise a practical approach to obtain adversarial patches while actively optimizing their location within the image. Then, we apply adversarial training on these location-optimized adversarial patches and demonstrate significantly improved robustness on CIFAR10 and GTSRB. Additionally, in contrast to adversarial training on imperceptible adversarial examples, our adversarial patch training does not reduce accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Akhtar, N., Mian, A.: Threat of adversarial attacks on deep learning in computer vision: a survey. IEEE Access 6, 14410–14430 (2018)

    Article  Google Scholar 

  2. Alaifari, R., Alberti, G.S., Gauksson, T.: ADef: an iterative algorithm to construct adversarial deformations. In: International Conference on Learning Representations (2019). https://openreview.net/forum?id=Hk4dFjR5K7

  3. Alayrac, J.B., Uesato, J., Huang, P.S., Fawzi, A., Stanforth, R., Kohli, P.: Are labels required for improving adversarial robustness? In: Wallach, H., Larochelle, H., Beygelzimer, A., d’ Alché-Buc, F., Fox, E., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 32, pp. 12214–12223. Curran Associates, Inc. (2019). http://papers.nips.cc/paper/9388-are-labels-required-for-improving-adversarial-robustness.pdf

  4. Andriushchenko, M., Croce, F., Flammarion, N., Hein, M.: Square attack: a query-efficient black-box adversarial attack via random search. arXiv: 1912.00049 (2019)

  5. Athalye, A., Carlini, N.: On the robustness of the CVPR 2018 white-box adversarial example defenses. arXiv: 1804.03286 (2018)

  6. Athalye, A., Carlini, N., Wagner, D.: Obfuscated gradients give a false sense of security: circumventing defenses to adversarial examples. In: Proceedings of Machine Learning Research, vol. 80, pp. 274–283. PMLR, Stockholmsmässan, Stockholm Sweden, 10–15 July 2018. http://proceedings.mlr.press/v80/athalye18a.html

  7. Bafna, M., Murtagh, J., Vyas, N.: Thwarting adversarial examples: an L\_0-robust sparse fourier transform. In: Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 31, pp. 10075–10085. Curran Associates, Inc. (2018). http://papers.nips.cc/paper/8211-thwarting-adversarial-examples-an-l_0-robust-sparse-fourier-transform.pdf

  8. Balaji, Y., Goldstein, T., Hoffman, J.: Instance adaptive adversarial training: improved accuracy tradeoffs in neural nets. arXiv:1910.08051 (2019)

  9. Bhagoji, A.N., He, W., Li, B., Song, D.: Exploring the space of black-box attacks on deep neural networks. arXiv: 1712.09491 (2017)

  10. Biggio, B., Roli, F.: Wild patterns: ten years after the rise of adversarial machine learning. Pattern Recogn. 84, 317–331 (2018). https://doi.org/10.1016/j.patcog.2018.07.023, http://www.sciencedirect.com/science/article/pii/S0031320318302565

  11. Brendel, W., Bethge, M.: Comment on “biologically inspired protection of deep networks from adversarial attacks”. arXiv: 1704.01547 (2017)

  12. Brown, T.B., Carlini, N., Zhang, C., Olsson, C., Christiano, P., Goodfellow, I.: Unrestricted adversarial examples. arXiv: 1809.08352 (2017)

  13. Brown, T.B., Mané, D., Roy, A., Abadi, M., Gilmer, J.: Adversarial patch. arXiv: 1712.09665 (2017)

  14. Brunner, T., Diehl, F., Knoll, A.: Copy and paste: a simple but effective initialization method for black-box adversarial attacks. arXiv: 1906.06086 (2019)

  15. Carlini, N., Wagner, D.: Towards evaluating the robustness of neural networks. In: 2017 IEEE Symposium on Security and Privacy (SP), pp. 39–57 (2017)

    Google Scholar 

  16. Carlini, N.: Is ami (attacks meet interpretability) robust to adversarial examples? arXiv: 1902.02322 (2019)

  17. Carlini, N., Wagner, D.: Adversarial examples are not easily detected: Bypassing ten detection methods. In: Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, AISec 2017, pp. 3–14. Association for Computing Machinery, New York (2017). https://doi.org/10.1145/3128572.3140444

  18. Carlini, N., Wagner, D.A.: Defensive distillation is not robust to adversarial examples. arXiv: 1607.04311 (2016)

  19. Carlini, N., Wagner, D.A.: Magnet and “efficient defenses against adversarial attacks” are not robust to adversarial examples. arXiv: 1711.08478 (2017)

  20. Carmon, Y., Raghunathan, A., Schmidt, L., Duchi, J.C., Liang, P.S.: Unlabeled data improves adversarial robustness. In: Wallach, H., Larochelle, H., Beygelzimer, A., d’ Alché-Buc, F., Fox, E., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 32, pp. 11192–11203. Curran Associates, Inc. (2019). http://papers.nips.cc/paper/9298-unlabeled-data-improves-adversarial-robustness.pdf

  21. Chen, J., Jordan, M.I.: Boundary Attack++: Query-efficient decision-based adversarial attack. arXiv: 1904.02144 (2019)

  22. Chen, P.Y., Zhang, H., Sharma, Y., Yi, J., Hsieh, C.J.: Zoo: zeroth order optimization based black-box attacks to deep neural networks without training substitute models. In: Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, AISec 2017 pp. 15–26. Association for Computing Machinery, New York (2017). https://doi.org/10.1145/3128572.3140448

  23. Chiang, P., Geiping, J., Goldblum, M., Goldstein, T., Ni, R., Reich, S., Shafahi, A.: Witchcraft: efficient PGD attacks with random step size. In: ICASSP 2020–2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 3747–3751 (2020)

    Google Scholar 

  24. Chiang, P., Ni, R., Abdelkader, A., Zhu, C., Studor, C., Goldstein, T.: Certified defenses for adversarial patches. In: International Conference on Learning Representations (2020). https://openreview.net/forum?id=HyeaSkrYPH

  25. Croce, F., Hein, M.: Sparse and imperceivable adversarial attacks. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2019

    Google Scholar 

  26. Croce, F., Hein, M.: Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In: Proceedings of the International Conference on Machine Learning, vol. 1, pp. 11571–11582 (2020). http://proceedings.mlr.press/v119/croce20b.html

  27. Dhaliwal, J., Hambrook, K.: Recovery guarantees for compressible signals with adversarial noise. arXiv: 1907.06565 (2019)

  28. Dong, Y., Liao, F., Pang, T., Su, H., Zhu, J., Hu, X., Li, J.: Boosting adversarial attacks with momentum. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018

    Google Scholar 

  29. Dumont, B., Maggio, S., Montalvo, P.: Robustness of rotation-equivariant networks to adversarial perturbations. arXiv: 1802.06627 (2018)

  30. Engstrom, L., Ilyas, A., Athalye, A.: Evaluating and understanding the robustness of adversarial logit pairing. arXiv: 1807.10272 (2018)

  31. Engstrom, L., Tsipras, D., Schmidt, L., Madry, A.: A rotation and a translation suffice: Fooling CNNs with simple transformations. arXiv: 1712.02779 (2017)

  32. Eykholt, K., et al.: Robust physical-world attacks on deep learning visual classification. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1625–1634 (2018)

    Google Scholar 

  33. Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. arXiv: 1412.6572 (2014)

  34. Gowal, S., et al.: On the effectiveness of interval bound propagation for training verifiably robust models. arXiv: 1810.12715 (2018)

  35. Guo, C., Gardner, J., You, Y., Wilson, A.G., Weinberger, K.: Simple black-box adversarial attacks. In: International Conference on Machine Learning, pp. 2484–2493 (2019)

    Google Scholar 

  36. Hayes, J.: On visible adversarial perturbations & digital watermarking. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1597–1604 (2018)

    Google Scholar 

  37. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016)

    Google Scholar 

  38. Hosseini, H., Poovendran, R.: Semantic adversarial examples. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1614–1619 (2018)

    Google Scholar 

  39. Huang, R., Xu, B., Schuurmans, D., Szepesvári, C.: Learning with a strong adversary. arXiv: 1511.03034 (2015)

  40. Ilyas, A., Engstrom, L., Athalye, A., Lin, J.: Black-box adversarial attacks with limited queries and information. In: Proceedings of the 35th International Conference on Machine Learning, ICML 2018, July 2018

    Google Scholar 

  41. Kanbak, C., Moosavi-Dezfooli, S.M., Frossard, P.: Geometric robustness of deep networks: Analysis and improvement. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018

    Google Scholar 

  42. Karmon, D., Zoran, D., Goldberg, Y.: LaVAN: localized and visible adversarial noise. In: Proceeding of the International Conference on Machine Learning (ICML), pp. 2512–2520 (2018)

    Google Scholar 

  43. Krizhevsky, A.: Learning multiple layers of features from tiny images. Technical Report (2009)

    Google Scholar 

  44. Lamb, A., Verma, V., Kannala, J., Bengio, Y.: Interpolated adversarial training: achieving robust neural networks without sacrificing too much accuracy. In: Proceedings of the ACM Workshop on Artificial Intelligence and Security, pp. 95–103 (2019)

    Google Scholar 

  45. Lee, H., Han, S., Lee, J.: Generative adversarial trainer: defense to adversarial perturbations with GAN. arXiv: 1705.03387 (2017)

  46. Lee, M., Kolter, Z.: On physical adversarial patches for object detection. arXiv: 1906.11897 (2019)

  47. Liu, X., Yang, H., Song, L., Li, H., Chen, Y.: DPatch: Attacking object detectors with adversarial patches. arXiv: 1806.02299 (2018)

  48. Liu, Y., Zhang, W., Li, S., Yu, N.: Enhanced attacks on defensively distilled deep neural networks. arXiv: 1711.05934 (2017)

  49. Luo, B., Liu, Y., Wei, L., Xu, Q.: Towards imperceptible and robust adversarial example attacks against neural networks. In: McIlraith, S.A., Weinberger, K.Q. (eds.) Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, 2–7 February 2018, pp. 1652–1659. AAAI Press (2018). https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/16217

  50. Madry, A., Makelov, A., Schmidt, L., Tsipras, D., Vladu, A.: Towards deep learning models resistant to adversarial attacks. In: International Conference on Learning Representations (2018). https://openreview.net/forum?id=rJzIBfZAb

  51. Maini, P., Wong, E., Kolter, J.Z.: Adversarial robustness against the union of multiple perturbation models. In: Proceedings of the International Conference on Machine Learning (ICML) (2020)

    Google Scholar 

  52. Mirman, M., Gehr, T., Vechev, M.T.: Differentiable abstract interpretation for provably robust neural networks. In: Proceedings of the International Conference on Machine Learning (ICML), pp. 3575–3583 (2018)

    Google Scholar 

  53. Miyato, T., Maeda, S.i., Koyama, M., Nakae, K., Ishii, S.: Distributional smoothing with virtual adversarial training. arXiv: 1507.00677 (2015)

  54. Mosbach, M., Andriushchenko, M., Trost, T.A., Hein, M., Klakow, D.: Logit pairing methods can fool gradient-based attacks. arXiv: 1810.12042 (2018)

  55. Naseer, M., Khan, S., Porikli, F.: Local gradients smoothing: defense against localized adversarial attacks. In: Proceedings of the IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1300–1307 (2019)

    Google Scholar 

  56. Raghunathan, A., Xie, S.M., Yang, F., Duchi, J.C., Liang, P.: Adversarial training can hurt generalization. arXiv: 1906.06032 (2019)

  57. Ranjan, A., Janai, J., Geiger, A., Black, M.J.: Attacking optical flow. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2019

    Google Scholar 

  58. Schott, L., Rauber, J., Brendel, W., Bethge, M.: Robust perception through analysis by synthesis. arXiv: 1805.09190 (2018)

  59. Shafahi, A., et al.: Adversarial training for free! In: Wallach, H.M., Larochelle, H., Beygelzimer, A., d’Alché-Buc, F., Fox, E.B., Garnett, R. (eds.) Advances in Neural Information Processing Systems (NIPS), pp. 3353–3364 (2019)

    Google Scholar 

  60. Shafahi, A., Najibi, M., Xu, Z., Dickerson, J.P., Davis, L.S., Goldstein, T.: Universal adversarial training. In: The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, 7–12 February 2020, pp. 5636–5643. AAAI Press (2020). https://aaai.org/ojs/index.php/AAAI/article/view/6017

  61. Shaham, U., Yamada, Y., Negahban, S.: Understanding adversarial training: increasing local stability of neural nets through robust optimization. arXiv: 1511.05432 (2015)

  62. Sharma, Y., Chen, P.Y.: Attacking the madry defense model with l1-based adversarial examples. arXiv: 1710.10733 (2017)

  63. Sinha, A., Namkoong, H., Duchi, J.: Certifiable distributional robustness with principled adversarial training. In: International Conference on Learning Representations (2018). https://openreview.net/forum?id=Hk6kPgZA-

  64. Song, Y., Shu, R., Kushman, N., Ermon, S.: Generative adversarial examples. arXiv: 1805.07894 (2018)

  65. Stallkamp, J., Schlipsing, M., Salmen, J., Igel, C.: Man vs. computer: benchmarking machine learning algorithms for traffic sign recognition. Neural Netw. 32, 323–332 (2012). https://doi.org/10.1016/j.neunet.2012.02.016, http://www.sciencedirect.com/science/article/pii/S0893608012000457

  66. Stutz, D., Hein, M., Schiele, B.: Disentangling adversarial robustness and generalization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019

    Google Scholar 

  67. Stutz, D., Hein, M., Schiele, B.: Confidence-calibrated adversarial training: generalizing to unseen attacks. In: Proceedings of the International Conference on Machine Learning ICML (2020)

    Google Scholar 

  68. Szegedy, C., et al.: Intriguing properties of neural networks. In: Proceedings of the International Conference on Learning Representations (ICLR) (2014)

    Google Scholar 

  69. Tramér, F., Boneh, D.: Adversarial training and robustness for multiple perturbations. In: Wallach, H., Larochelle, H., Beygelzimer, A., d’ Alché-Buc, F., Fox, E., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 32, pp. 5866–5876. Curran Associates, Inc. (2019). http://papers.nips.cc/paper/8821-adversarial-training-and-robustness-for-multiple-perturbations.pdf

  70. Tramèr, F., Carlini, N., Brendel, W., Madry, A.: On adaptive attacks to adversarial example defenses. arXiv: 2002.08347 (2020)

  71. Tsipras, D., Santurkar, S., Engstrom, L., Turner, A., Madry, A.: Robustness may be at odds with accuracy. In: International Conference on Learning Representations (2019). https://openreview.net/forum?id=SyxAb30cY7

  72. Wang, J., Zhang, H.: Bilateral adversarial training: towards fast training of more robust models against adversarial attacks. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2019

    Google Scholar 

  73. Wiyatno, R., Xu, A.: Physical adversarial textures that fool visual object tracking. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 4821–4830 (2019)

    Google Scholar 

  74. Wong, E., Rice, L., Kolter, J.Z.: Fast is better than free: revisiting adversarial training. In: International Conference on Learning Representations (2020). https://openreview.net/forum?id=BJx040EFvH

  75. Wu, T., Tong, L., Vorobeychik, Y.: Defending against physically realizable attacks on image classification. In: International Conference on Learning Representations (2020). https://openreview.net/forum?id=H1xscnEKDr

  76. Xiao, C., Zhu, J.Y., Li, B., He, W., Liu, M., Song, D.: Spatially transformed adversarial examples. In: International Conference on Learning Representations (2018). https://openreview.net/forum?id=HyydRMZC-

  77. Xu, H., et al.: Adversarial attacks and defenses in images, graphs and text: a review. Int. J. Autom. Comput. 17, 151–178 (2020)

    Article  Google Scholar 

  78. Xu, K., et al.: Structured adversarial attack: towards general implementation and better interpretability. In: International Conference on Learning Representations (2019). https://openreview.net/forum?id=BkgzniCqY7

  79. Yuan, X., He, P., Zhu, Q., Li, X.: Adversarial examples: attacks and defenses for deep learning. IEEE Trans. Neural Netw. Learn. Syst. 30(9), 2805–2824 (2019)

    Article  MathSciNet  Google Scholar 

  80. Zajac, M., Zołna, K., Rostamzadeh, N., Pinheiro, P.O.: Adversarial framing for image and video classification. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 10077–10078 (2019)

    Google Scholar 

  81. Zhang, H., Yu, Y., Jiao, J., Xing, E.P., Ghaoui, L.E., Jordan, M.I.: Theoretically principled trade-off between robustness and accuracy. In: Proceedings of the International Conference on Machine Learning (ICML), pp. 7472–7482 (2019)

    Google Scholar 

  82. Zhang, H., Chen, H., Song, Z., Boning, D.S., Dhillon, I.S., Hsieh, C.: The limitations of adversarial training and the blind-spot attack. In: 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, 6–9 May 2019. OpenReview.net (2019). https://openreview.net/forum?id=HylTBhA5tQ

  83. Zhang, S., Huang, K., Zhu, J., Liu, Y.: Manifold adversarial learning. arXiv: 1807.05832v1 (2018)

  84. Zhao, Z., Dua, D., Singh, S.: Generating natural adversarial examples. In: International Conference on Learning Representations (2018). https://openreview.net/forum?id=H1BLjgZCb

  85. Zhao, Z., Liu, Z., Larson, M.A.: A differentiable color filter for generating unrestricted adversarial images. arXiv: 2002.01008 (2020)

Download references

Author information

Authors and Affiliations

  1. Max Planck Institute for Informatics, Saarland Informatics Campus, Saarbrücken, Germany

    Sukrut Rao, David Stutz & Bernt Schiele

Authors
  1. Sukrut Rao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David Stutz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bernt Schiele
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sukrut Rao .

Editor information

Editors and Affiliations

  1. University of Clermont Auvergne, Clermont Ferrand, France

    Adrien Bartoli

  2. Università degli Studi di Udine, Udine, Italy

    Andrea Fusiello

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Rao, S., Stutz, D., Schiele, B. (2020). Adversarial Training Against Location-Optimized Adversarial Patches. In: Bartoli, A., Fusiello, A. (eds) Computer Vision – ECCV 2020 Workshops. ECCV 2020. Lecture Notes in Computer Science(), vol 12539. Springer, Cham. https://doi.org/10.1007/978-3-030-68238-5_32

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-68238-5_32

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-68237-8

  • Online ISBN: 978-3-030-68238-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics