Skip to main content
SpringerLink
Log in

Self-Adaptive Logit Balancing for Deep Learning Robustness in Computer Vision

  • Conference paper
  • First Online:
Image Analysis and Processing – ICIAP 2022 (ICIAP 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13231))

Included in the following conference series:

  • 1668 Accesses

Abstract

With wide applications of machine learning algorithms, machine learning security has become a significant issue. The vulnerability to adversarial perturbations exists in most machine learning algorithms, including cutting-edge deep neural networks. The standard adversarial perturbation defence techniques with adversarial training need to generate adversarial examples during the training process, which require high computational costs. This paper proposed a novel defence method using self-adaptive logit balancing and Gaussian noise boost training. This method can improve the robustness of deep neural networks without high computational cost and achieve competitive results compared with the adversarial training methods. Meanwhile, this defence method enables deep learning systems to have proactive and reactive defence during the operation. A sub-classifier is trained to determine whether the system is under attack and detect attack algorithms via the patterns of the Log-Softmax values. It can achieve high accuracy for detecting clean inputs and adversarial examples created by seven attack methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Aigrain, J., Detyniecki, M.: Detecting adversarial examples and other misclassifications in neural networks by introspection. arXiv preprint arXiv:1905.09186 (2019). 10.48550/arXiv. 1905.09186

  2. Andriushchenko, M., Croce, F., Flammarion, N., Hein, M.: Square attack: a query-efficient black-box adversarial attack via random search. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12368, pp. 484–501. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58592-1_29

    Chapter  Google Scholar 

  3. Athalye, A., Engstrom, L., Ilyas, A., Kwok, K.: Synthesizing robust adversarial examples. In: International conference on machine learning, pp. 284–293. PMLR (2018). https://doi.org/10.48550/arXiv.1707.07397

  4. Carlini, N., et al.: On evaluating adversarial robustness. arXiv preprint arXiv:1902.06705 (2019). 10.48550/arXiv. 1902.06705

  5. Carmon, Y., Raghunathan, A., Schmidt, L., Liang, P., Duchi, J.C.: Unlabeled data improves adversarial robustness. arXiv preprint arXiv:1905.13736 (2019). 10.48550/arXiv. 1905.13736

  6. le Cessie, S., van Houwelingen, J.: Ridge estimators in logistic regression. Appl. Stat. 41(1), 191–201 (1992)

    Article  Google Scholar 

  7. Croce, F., Hein, M.: Minimally distorted adversarial examples with a fast adaptive boundary attack. In: International Conference on Machine Learning, pp. 2196–2205. PMLR (2020). 10.48550/arXiv. 1907.02044

    Google Scholar 

  8. Croce, F., Hein, M.: Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In: International Conference on Machine Learning, pp. 2206–2216. PMLR (2020). 10.48550/arXiv. 2003.01690

    Google Scholar 

  9. Engstrom, L., Ilyas, A., Athalye, A.: Evaluating and understanding the robustness of adversarial logit pairing. arXiv preprint arXiv:1807.10272 (2018). 10.48550/arXiv. 1807.10272

  10. Fu, C., Chen, H., Ruan, N., Jia, W.: Label smoothing and adversarial robustness. arXiv preprint arXiv:2009.08233 (2020). 10.48550/arXiv. 2009.08233

  11. Goodfellow, I.J., et al.: Generative adversarial networks. arXiv preprint arXiv:1406.2661 (2014). 10.48550/arXiv. 1406.2661

  12. Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 (2014). 10.48550/arXiv. 1412.6572

  13. Hosseini, H., Kannan, S., Poovendran, R.: Are odds really odd? bypassing statistical detection of adversarial examples. arXiv preprint arXiv:1907.12138 (2019). 10.48550/arXiv. 1907.12138

  14. Huang, X., Kwiatkowska, M., Wang, S., Wu, M.: Safety verification of deep neural networks. In: Majumdar, R., Kunčak, V. (eds.) CAV 2017. LNCS, vol. 10426, pp. 3–29. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-63387-9_1

    Chapter  Google Scholar 

  15. Jin, G., Shen, S., Zhang, D., Dai, F., Zhang, Y.: APE-GAN: Adversarial perturbation elimination with gan. In: ICASSP 2019–2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 3842–3846. IEEE (2019). https://doi.org/10.1109/ICASSP.2019.8683044

  16. Kannan, H., Kurakin, A., Goodfellow, I.: Adversarial logit pairing. arXiv preprint arXiv:1803.06373 (2018). 10.48550/arXiv. 1803.06373

  17. Kim, H.: Torchattacks: A pytorch repository for adversarial attacks. arXiv preprint arXiv:2010.01950 (2020). 10.48550/arXiv. 2010.01950

  18. Liu, X., Li, Y., Wu, C., Hsieh, C.J.: Adv-BNN: Improved adversarial defense through robust bayesian neural network. arXiv preprint arXiv:1810.01279 (2018). 10.48550/arXiv. 1810.01279

  19. Madry, A., Makelov, A., Schmidt, L., Tsipras, D., Vladu, A.: Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083 (2017). 10.48550/arXiv. 1706.06083

  20. Roth, K., Kilcher, Y., Hofmann, T.: The odds are odd: a statistical test for detecting adversarial examples. In: International Conference on Machine Learning, pp. 5498–5507. PMLR (2019)

    Google Scholar 

  21. Sammut, C., Webb, G.I.: Encyclopedia of Machine Learning. Springer Science & Business Media (2011)

    Google Scholar 

  22. Schmidt, L., Santurkar, S., Tsipras, D., Talwar, K., Madry, A.: Adversarially robust generalization requires more data. arXiv preprint arXiv:1804.11285 (2018). 10.48550/arXiv. 1804.11285

  23. Shafahi, A., Ghiasi, A., Najibi, M., Huang, F., Dickerson, J.P., Goldstein, T.: Batch-wise logit-similarity: generalizing logit-squeezing and label-smoothing. In: European Conference on Computer Vision. British Machine Vision Conference (2019)

    Google Scholar 

  24. Shafahi, A., Najibi, M., Ghiasi, A., Xu, Z., Dickerson, J., Studer, C., Davis, L.S., Taylor, G., Goldstein, T.: Adversarial training for free! arXiv preprint arXiv:1904.12843 (2019). 10.48550/arXiv. 1904.12843

  25. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2818–2826 (2016). https://doi.org/10.1109/CVPR.2016.308

  26. Szegedy, C., et al.: Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199 (2013). 10.48550/arXiv. 1312.6199

  27. Wu, D., Xia, S.T., Wang, Y.: Adversarial weight perturbation helps robust generalization. Adv. Neural Inf. Process. Syst. 33, 2958–2969 (2020)

    Google Scholar 

  28. Zagoruyko, S., Komodakis, N.: Wide residual networks. arXiv preprint arXiv:1605.07146 (2016). 10.48550/arXiv. 1605.07146

  29. Zhang, H., Yu, Y., Jiao, J., Xing, E., El Ghaoui, L., Jordan, M.: Theoretically principled trade-off between robustness and accuracy. In: International Conference on Machine Learning, pp. 7472–7482. PMLR (2019)

    Google Scholar 

  30. Zheng, A., Casari, A.: Feature Engineering for Machine Learning: Principles and Techniques for Data Scientists. O’Reilly Media, Inc. (2018)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Loughborough University, Epinal Way, Loughborough, LE11 3TU, UK

    Jiefei Wei & Qinggang Meng

  2. University of Nottingham, University Park, Nottingham, NG7 2RD, UK

    Luyan Yao

Authors
  1. Jiefei Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qinggang Meng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Luyan Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jiefei Wei .

Editor information

Editors and Affiliations

  1. Boston University, Boston, MA, USA

    Stan Sclaroff

  2. National Research Council, Lecce, Italy

    Cosimo Distante

  3. National Research Council, Lecce, Italy

    Marco Leo

  4. University of Catania, Catania, Italy

    Giovanni M. Farinella

  5. Technische Universität München, Garching, Germany

    Federico Tombari

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wei, J., Meng, Q., Yao, L. (2022). Self-Adaptive Logit Balancing for Deep Learning Robustness in Computer Vision. In: Sclaroff, S., Distante, C., Leo, M., Farinella, G.M., Tombari, F. (eds) Image Analysis and Processing – ICIAP 2022. ICIAP 2022. Lecture Notes in Computer Science, vol 13231. Springer, Cham. https://doi.org/10.1007/978-3-031-06427-2_46

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-06427-2_46

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-06426-5

  • Online ISBN: 978-3-031-06427-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation