Skip to main content
SpringerLink
Log in

Improving Adversarial Robustness of Deep Neural Networks via Linear Programming

  • Conference paper
  • First Online:
Theoretical Aspects of Software Engineering (TASE 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13299))

Included in the following conference series:

  • 781 Accesses

Abstract

Adversarial training provides an effective means to improve the robustness of neural networks against adversarial attacks. The nonlinear feature of neural networks makes it difficult to find good adversarial examples where project gradient descent (PGD) based training is reported to perform best. In this paper, we build an iterative training framework to implement effective robust training. It introduces the Least-Squares based linearization to build a set of affine functions to approximate the nonlinear functions calculating the difference of discriminant values between a specific class and the correct class and solves it using LP solvers by simplex methods. The solutions found by LP solvers turn out to be very close to the real optimum so that our method outperforms PGD based adversarial training, as is shown by extensive experiments on the MNIST and CIFAR-10 datasets. Especially, our methods can provide considerable robust networks on CIFAR-10 against the strong strength attacks, where the other methods get stuck and do not converge.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Amato, F., Lopez, A., Pena-Mendez, E.M., Vanhara, P., Hampl, A., Havel, J.: Artificial neural networks in medical diagnosis. J. Appl. Biomed. 11(2), 47–58 (2013)

    Article  Google Scholar 

  2. Andor, D., et al.: Globally normalized transition-based neural networks. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, Berlin, Germany, 7–12 August 2016, Volume 1: Long Papers (2016)

    Google Scholar 

  3. Athalye, A., Carlini, N., Wagner, D.A.: Obfuscated gradients give a false sense of security: circumventing defenses to adversarial examples. In: Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, 10–15 July 2018, pp. 274–283 (2018)

    Google Scholar 

  4. Biggio, B., et al.: Evasion attacks against machine learning at test time. In: Blockeel, H., Kersting, K., Nijssen, S., Železný, F. (eds.) ECML PKDD 2013. LNCS (LNAI), vol. 8190, pp. 387–402. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40994-3_25

    Chapter  Google Scholar 

  5. Bojarski, M., et al: Efficient visualization of CNNs for autonomous driving. In: 2018 IEEE International Conference on Robotics and Automation, ICRA 2018, pp. 1–8 (2018)

    Google Scholar 

  6. Bojchevski, A., Günnemann, S.: Adversarial attacks on node embeddings (2018)

    Google Scholar 

  7. Buckman, J., Roy, A., Raffel, C., Goodfellow, I.J.: Thermometer encoding: one hot way to resist adversarial examples. In: 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, 30 April–3 May 2018, Conference Track Proceedings (2018)

    Google Scholar 

  8. Carlini, N., Katz, G., Barrett, C., Dill, D.L.: Provably minimally-distorted adversarial examples. arXiv preprint arXiv:1709.10207 (2017)

  9. Carlini, N., Wagner, D.A.: Adversarial examples are not easily detected: bypassing ten detection methods. In: Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, pp. 3–14 (2017)

    Google Scholar 

  10. Chen, J., Wu, Y., Xu, X., Chen, Y., Zheng, H., Xuan, Q.: Fast gradient attack on network embedding. arXiv preprint arXiv:1809.02797 (2018)

  11. Dhillon, G.S., et al.: Stochastic activation pruning for robust adversarial defense. In: 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, 30 April–3 May 2018, Conference Track Proceedings (2018)

    Google Scholar 

  12. Ding, G.W., Sharma, Y., Lui, K.Y.C., Huang, R.: Max-margin adversarial (MMA) training: direct input space margin maximization through adversarial training. arXiv preprint arXiv:1812.02637 (2018)

  13. Dong, Y., et al.: Boosting adversarial attacks with momentum. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, 18–22 June 2018, pp. 9185–9193 (2018)

    Google Scholar 

  14. Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge (2016)

    MATH  Google Scholar 

  15. Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. In: 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, 7–9 May 2015, Conference Track Proceedings (2015)

    Google Scholar 

  16. Guo, Y., Li, Q., Chen, H.: Backpropagating linearly improves transferability of adversarial examples. In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M.F., Lin, H. (eds.) Advances in Neural Information Processing Systems, vol. 33, pp. 85–95. Curran Associates Inc., New York (2020)

    Google Scholar 

  17. Hinton, G., et al.: Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. IEEE Signal Process. Mag. 29(6), 82–97 (2012)

    Article  Google Scholar 

  18. Hosu, I., Rebedea, T.: Playing atari games with deep reinforcement learning and human checkpoint replay. CoRR abs/1607.05077 (2016)

    Google Scholar 

  19. Huang, R., Xu, B., Schuurmans, D., Szepesvári, C.: Learning with a strong adversary. arXiv preprint arXiv:1511.03034 (2015)

  20. Jakubovitz, D., Giryes, R.: Improving DNN robustness to adversarial attacks using jacobian regularization. In: Computer Vision - ECCV 2018–15th European Conference, Munich, Germany, 8–14 September 2018, Proceedings, Part XII, pp. 525–541 (2018)

    Google Scholar 

  21. Katz, G., Barrett, C., Dill, D.L., Julian, K., Kochenderfer, M.J.: Reluplex: an efficient SMT solver for verifying deep neural networks. In: Majumdar, R., Kunčak, V. (eds.) CAV 2017. LNCS, vol. 10426, pp. 97–117. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-63387-9_5

    Chapter  Google Scholar 

  22. Krizhevsky, A., Nair, V., Hinton, G.: The CIFAR-10 dataset home page (2009). https://www.cs.toronto.edu/~kriz/cifar.html

  23. Kurakin, A., Goodfellow, I.J., Bengio, S.: Adversarial examples in the physical world. In: 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, 24–26 April 2017, Workshop Track Proceedings (2017)

    Google Scholar 

  24. Kurakin, A., Goodfellow, I.J., Bengio, S.: Adversarial machine learning at scale. In: Proceedings International Conference on Learning Representations (ICLR), pp. 1–17 (2017)

    Google Scholar 

  25. LeCun, Y., Cortes, C., Burges, C.J.: The MNIST database of handwritten digits home page (1998). http://yann.lecun.com/exdb/mnist/

  26. Lin, J., Gan, C., Han, S.: Defensive quantization: when efficiency meets robustness. In: 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, 6–9 May 2019 (2019)

    Google Scholar 

  27. Madry, A., Makelov, A., Schmidt, L., Tsipras, D., Vladu, A.: Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083 (2017)

  28. Madry, A., Makelov, A., Schmidt, L., Tsipras, D., Vladu, A.: Towards deep learning models resistant to adversarial attacks. In: 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, 30 April–3 May 2018, Conference Track Proceedings (2018)

    Google Scholar 

  29. Maini, P., Wong, E., Kolter, J.Z.: Adversarial robustness against the union of multiple perturbation models. CoRR abs/1909.04068 (2019)

    Google Scholar 

  30. Moosavi-Dezfooli, S., Fawzi, A., Frossard, P.: DeepFool: a simple and accurate method to fool deep neural networks. In: IEEE Conference on Computer Vision and Pattern Recognition(CVPR), pp. 2574–2582 (2016)

    Google Scholar 

  31. Moosavi-Dezfooli, S., Fawzi, A., Frossard, P.: DeepFool: a simple and accurate method to fool deep neural networks. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, 27–30 June 2016, pp. 2574–2582 (2016)

    Google Scholar 

  32. Mosbach, M., Andriushchenko, M., Trost, T.A., Hein, M., Klakow, D.: Logit pairing methods can fool gradient-based attacks. CoRR abs/1810.12042 (2018)

    Google Scholar 

  33. Na, T., Ko, J.H., Mukhopadhyay, S.: Cascade adversarial machine learning regularized with a unified embedding. In: Proceedings International Conference on Learning Representations (ICLR) (2018)

    Google Scholar 

  34. Qian, H., Wegman, M.N.: L2-nonexpansive neural networks. In: 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, 6–9 May 2019 (2019)

    Google Scholar 

  35. Silver, D., et al.: Mastering the game of go with deep neural networks and tree search. Nature 529(7587), 484–489 (2016)

    Article  Google Scholar 

  36. Song, Y., Kim, T., Nowozin, S., Ermon, S., Kushman, N.: PixelDefend: leveraging generative models to understand and defend against adversarial examples. In: 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, 30 April–3 May 2018, Conference Track Proceedings (2018)

    Google Scholar 

  37. Sulam, J., Muthukumar, R., Arora, R.: Adversarial robustness of supervised sparse coding. In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M.F., Lin, H. (eds.) Advances in Neural Information Processing Systems, vol. 33, pp. 2110–2121. Curran Associates, Inc., New York (2020)

    Google Scholar 

  38. Szegedy, C., et al.: Intriguing properties of neural networks. In: Proceedings of the International Conference on Learning Representations (ICLR 2014) (2014)

    Google Scholar 

  39. Tjeng, V., Tedrake, R.: Verifying neural networks with mixed integer programming. CoRR abs/1711.07356 (2017)

    Google Scholar 

  40. Tramèr, F., Boneh, D.: Adversarial training and robustness for multiple perturbations. In: Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, Vancouver, BC, Canada, 8–14 December 2019, pp. 5858–5868 (2019)

    Google Scholar 

  41. Tramèr, F., Kurakin, A., Papernot, N., Goodfellow, I.J., Boneh, D., McDaniel, P.D.: Ensemble adversarial training: attacks and defenses. In: 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, 30 April–3 May, 2018, Conference Track Proceedings (2018)

    Google Scholar 

  42. Wong, E., Kolter, J.Z.: Provable defenses against adversarial examples via the convex outer adversarial polytope, vol. 12, Stockholm, Sweden, pp. 8405–8423 (2018)

    Google Scholar 

  43. Wong, E., Rice, L., Kolter, J.Z.: Fast is better than free: Revisiting adversarial training. In: 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, 26–30 April 2020 (2020)

    Google Scholar 

  44. Xiao, K.Y., Tjeng, V., Shafiullah, N.M.M., Madry, A.: Training for faster adversarial robustness verification via inducing reLU stability. In: International Conference on Learning Representations (2019)

    Google Scholar 

  45. Xie, C., Wu, Y., van der Maaten, L., Yuille, A.L., He, K.: Feature denoising for improving adversarial robustness. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, 16–20 June 2019, pp. 501–509 (2019)

    Google Scholar 

  46. Yang, Y., Zhang, G., Xu, Z., Katabi, D.: ME-Net: towards effective adversarial robustness with matrix estimation. In: Proceedings of the 36th International Conference on Machine Learning, ICML 2019, Long Beach, California, USA, 9–15 June 2019, pp. 7025–7034 (2019)

    Google Scholar 

  47. Zhang, H., Yu, Y., Jiao, J., Xing, E.P., Ghaoui, L.E., Jordan, M.I.: Theoretically principled trade-off between robustness and accuracy. arXiv preprint arXiv:1901.08573 (2019)

Download references

Acknowledgment

This research is supported by the National Natural Science Foundation of China under Grant 12171159 and Shanghai Trusted Industry Internet Software Collaborative Innovation Center.

Author information

Authors and Affiliations

  1. Shanghai Key Lab of Trustworthy Computing, East China Normal University, Shanghai, China

    Xiaochao Tang, Zhengfeng Yang & Xuanming Fu

  2. School of Computer and Information Engineering, Henan University, Kaifeng, China

    Jianlin Wang

  3. Department of Mathematics, Shanghai University, Shanghai, China

    Zhenbing Zeng

Authors
  1. Xiaochao Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhengfeng Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xuanming Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jianlin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhenbing Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhengfeng Yang .

Editor information

Editors and Affiliations

  1. IRIT, Toulouse, France

    Yamine Aït-Ameur

  2. Babeș-Bolyai University, Cluj-Napoca, Romania

    Florin Crăciun

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Tang, X., Yang, Z., Fu, X., Wang, J., Zeng, Z. (2022). Improving Adversarial Robustness of Deep Neural Networks via Linear Programming. In: Aït-Ameur, Y., Crăciun, F. (eds) Theoretical Aspects of Software Engineering. TASE 2022. Lecture Notes in Computer Science, vol 13299. Springer, Cham. https://doi.org/10.1007/978-3-031-10363-6_22

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-10363-6_22

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-10362-9

  • Online ISBN: 978-3-031-10363-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation