Skip to main content
SpringerLink
Log in

DeepMC: DNN test sample optimization method jointly guided by misclassification and coverage

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Large-scale and high-quality test samples are extremely scarce in deep neural networks(DNN) testing. Existing test sample optimization methods exhibit the problem of low efficiency and low neuron coverage of optimized test samples, which consistently fail to expose erroneous behaviors of DNNs with corner-case inputs. In this paper, we propose DeepMC, an image classification DNN test sample optimization method jointly guided by misclassification and coverage. Specifically, we select the seed sample from the original test samples according to the misclassification probability. To maximize the misclassification probability and neuron coverage, we construct the joint optimization problem for the seed samples and use the gradient ascent to solve the joint optimization problem. We evaluate this method on two well-known datasets and prevalent image classification DNN models. Compare with DeepXplore, a DL white-box testing framework, DeepMC does not require multiple DNN models with similar functions for cross-referencing, saves 90% time consumption on MNIST, averagely covers 1.87% more neurons, and optimized test samples with more than 69% attack success rate. In addition, the test sample optimized by DeepMC can also be applied to optimize the robustness of the corresponding DNN with an average 3% improvement of the model’s accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Algorithm 1
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Wei K T, Ismail M, Husin Z, Yasruddin M L (2022) Automated tomato grading system using computer vision (CV) and deep neural network (DNN) algorithm. In: Proceedings of the 2022 IEEE 12th symposium on computer applications & industrial electronics (ISCAIE), pp 22–27. https://doi.org/10.1109/ISCAIE54458.2022.9794557

  2. Tang H, Liu H, Xiao W, Sebe N (2021) When dictionary learning meets deep learning: deep dictionary learning and coding network for image recognition with limited data. IEEE Trans Neural Netw Learn Syst 32(5):2129–2141. https://doi.org/10.1109/TNNLS.2020.2997289

    Article  MathSciNet  Google Scholar 

  3. Shelke N, Chaudhury S, Chakrabarti S, Bangare SL, Yogapriya G, Pandey P (2022) An efficient way of text-based emotion analysis from social media using LRA-DNN. Neurosci Inf 2(3):100048. https://doi.org/10.1016/j.neuri.2022.100048

    Article  Google Scholar 

  4. Kahn G, Villaflor A, Ding B, Abbeel P, Levine S (2018) Self-supervised deep reinforcement learning with generalized computation graphs for robot navigation. In: Proceeding of the 2018 IEEE international conference on robotics and automation, pp 5129–5136. https://doi.org/10.1109/ICRA.2018.8460655

  5. Dhamija T, Gupta A, Gupta S, Anjum KR (2022) Semantic segmentation in medical images through transfused convolution and transformer networks. Appl Intell. https://doi.org/10.1007/s10489-022-03642-w

  6. Lee DH, Chen KL, Liou KH, Liu CH, Liu JL (2021) Deep learning and control algorithms of direct perception for autonomous driving. Appl Intell 51:237–247. https://doi.org/10.1007/s10489-020-01827-9

    Article  Google Scholar 

  7. Kadhim TA, Zghal NS, Hariri D, Aissa DB (2022) Face recognition in multiple variations using deep learning and convolutional neural networks. In: Proceeding of the 2022 IEEE 9th international conference on sciences of electronics, Technologies of Information and Telecommunications (SETIT), pp 305–311. https://doi.org/10.1109/SETIT54465.2022.9875530

  8. Golwalkar R, Mehendale N (2022) Masked-face recognition using deep metric learning and FaceMaskNet-21. Appl Intell. https://doi.org/10.1007/s10489-021-03150-3

  9. Zhang X, Mahadevan S (2019) Ensemble machine learning models for aviation incident risk prediction. Decis Support Syst 116:48–63. https://doi.org/10.1016/j.dss.2018.10.009

    Article  Google Scholar 

  10. Sun H, Chen J, Lei L, Ji K, Kuang G (2021) Adversarial robustness of deep convolutional neural network-based image recognition models: a review. J Radars 10(4):571–594. https://doi.org/10.12000/JR21048

    Article  Google Scholar 

  11. Berghoff C, Neu M, Twickel AV (2020) Vulnerabilities of connectionist AI applications: evaluation and defense. Frontiers Big Data 3:23. https://doi.org/10.3389/fdata.2020.00023

    Article  Google Scholar 

  12. Newaz AI, Haque NI, Sikder AK, Rahman MA, Uluagac AS (2020) Adversarial attacks to machine learning-based smart healthcare systems. In: Proceeding of the 2020–2020 IEEE Global Communications Conference, pp 1–6. https://doi.org/10.1109/GLOBECOM42002.2020.9322472

  13. Kumar K, Nair S, Roy DG, Rajalingam B, Kumar RS (2021) Security and privacy-aware artificial intrusion detection system using federated machine learning. Comput Electr Eng 96:107440. https://doi.org/10.1016/j.compeleceng.2021.107440

    Article  Google Scholar 

  14. Ramanagopal MS, Anderson C, Vasudevan R, Johnson-Roberson M (2018) Failing to learn: autonomously identifying perception failures for Selfdriving cars. IEEE Robot Autom Lett 3(4):3860–3867. https://doi.org/10.1109/LRA.2018.2857402

  15. Hyun K, Yongchul K, Ki-Woong P, Hyunsoo Y, Choi D (2018) Friend-SafeEvasion Attack: An Adversarial Example That is Correctly Recognized by A Friendly Classifier. Comput Secur 78:380–397. https://doi.org/10.1016/j.cose.2018.07.015

    Article  Google Scholar 

  16. Wang Z, Yan M, Liu S, Chen J, Zhang D, Wu Z et al (2020) Survey on Testing of Deep Neural Networks. J Softw 31(5):1255–1275. https://doi.org/10.13328/j.cnki.jos.005951

    Article  Google Scholar 

  17. Wang S Y, Zhang Z H, Sun J Z (2020) Test case prioritization based on coverage compaction. Journal of Xi’an University of Posts and Telecommunications 03:82–87. https://doi.org/10.13682/j.issn.2095-6533.2020.03.014

  18. Wang Z, Zheng Y, Hai Z, Chang Y, Chen T (2022) Transferable adversarial examples can efficiently fool topic models. Comput Secur 118:102749. https://doi.org/10.1016/j.cose.2022.102749

    Article  Google Scholar 

  19. Ma L, Juefei-Xu F, Zhang F, Sun J, Xue M, Li B et al (2018) Deepgauge: multi-granularity testing criteria for deep learning systems. In: Proceeding of the 2018 33rd IEEE/ACM international conference on automated software engineering, pp 120–131. https://doi.org/10.1145/3238147.3238202

  20. Serban A, Poll E, Visser J (2020) Adversarial examples on object recognition: a comprehensive survey. ACM Comput Surv 53(3):1–38. https://doi.org/10.1145/3398394

    Article  Google Scholar 

  21. Yuan X, He P, Zhu Q, Li X (2019) Adversarial examples: attacks and defenses for deep learning. IEEE Trans Neural Netw Learn Syst 30(9):2805–2824. https://doi.org/10.1109/TNNLS.2018.2886017

    Article  MathSciNet  Google Scholar 

  22. Zhang S, Zuo X, Liu J (2019) The problem of the adversarial examples in deep learning. Chin J Comput 2(8):1886–1904. https://doi.org/10.11897/SP.J.1016.2019.01886

    Article  Google Scholar 

  23. Zhang J, Qian W, Nie R, Cao J, Xu D (2022) Generate adversarial examples by adaptive moment iterative fast gradient sign method. Appl Intell. https://doi.org/10.1007/s10489-022-03437-z

  24. Liu J, Tian Y, Zhang R, Sun Y, Wang C (2020) A two-stage generative adversarial networks with semantic content constraints for adversarial example generation. IEEE 8:205766–205777. https://doi.org/10.1109/ACCESS.2020.3037329

    Article  Google Scholar 

  25. He Z, Lan X, Yuan J, Wen C (2022) Multi-layer noise reshaping and perceptual optimization for effective adversarial attack of images. Appl Intell 52:1289–1305. https://doi.org/10.1007/s10489-022-03838-0

    Article  Google Scholar 

  26. Li C, Zhang X, Yin F, Liu C (2022) Decision-based adversarial attack with frequency Mixup. EEE Trans Inf Forensic Secur 17:1038–1052. https://doi.org/10.1109/TIFS.2022.3156809

    Article  Google Scholar 

  27. Hayes J, Danezis G (2018) Learning universal adversarial perturbations with generative models. In: Proceeding of the 2018 IEEE security and privacy workshops, pp 43–49. https://doi.org/10.1109/SPW.2018.00015

  28. Pei K, Cao Y, Yang J, Jana S (2019) Deepxplore: automated Whitebox testing of deep learning systems. Commun ACM 62(11):137–145. https://doi.org/10.1145/3361566

    Article  Google Scholar 

  29. Ma L, Juefei-Xu F, Xue M, Li B, Li L, Liu Y, et al (2019) Deepct: tomographic combinatorial testing for deep learning systems. In: Proceeding of the 2019 IEEE 26th international conference on software analysis, pp 614−618. https://doi.org/10.1109/SANER.2019.8668044

  30. Tian Y, Pei K, Jana S, Ray B (2018) DeepTest: automated testing of deep-neural-network-driven autonomous cars. In: Proceeding of the 2018 IEEE/ACM 40th international conference on software engineering, pp 303–314. https://doi.org/10.1145/3180155.3180220

  31. SudKul (2018) Self-driving-car Datasets.[updated 17 May 2018; cited 10 February 2022]. Available from: https://github.com/udacity/self-driving-car/tree/master/datasets. Accessed 12 Nov 2021

  32. Guo J, Jiang Y, Zhao Y, Chen Q, Sun J (2018) DLFuzz: differential fuzzing testing of deep learning systems. In: Proceeding of the 2018 26th ACM joint meeting on European software engineering Conf. And Symp. On the foundations of software engineering, pp 739−743. https://doi.org/10.1145/3236024.3264835

  33. Le Cun Y, Cortes C, Burges C J (2018) Handwritten Digital Image Dataset MNIST. [updated 9 February 2018; cited 10 February 2022]. Available from: http://yann.lecun.com/exdb/mnist. Accessed 12 Nov 2021

  34. An LF Image Database ImageNet. [updated 11 March 2021; cited 10 February 2022]. Available from: http://www.image-net.org. Accessed 12 Nov 2021

  35. Feng Y, Shi Q, Gao X, Wan J, Fang C, Chen Z (2020) Deepgini: prioritizing massive tests to enhance the robustness of deep neural networks. In proceeding of the 29th ACM Sigsoft international symposium on software testing and analysis, pp.177-188. https://doi.org/10.1145/3395363.3397357

  36. Tan S, Tan Z (2019) Improved LeNet-5 model based on handwritten numeral recognition. In: Proceeding of the 2019 Chinese control and decision conference, pp 6396–6399. https://doi.org/10.1109/CCDC.2019.8833112

  37. Zhang X (2021) The AlexNet, LeNet-5 and VGG NET applied to CIFAR-10. In: Proceeding of the 2021 2nd international conference on big Data & Artificial Intelligence & software engineering, pp 414–419. https://doi.org/10.1109/ICBASE.53849.2021.00083

  38. Mascarenhas S, Agarwal M (2021) A comparison between VGG16, VGG19 and ResNet50 architecture frameworks for image classification. In: Proceeding of the 2021 international conference on disruptive Technologies for Multi-Disciplinary Research and Applications (CENTCON), pp 96–99. https://doi.org/10.1109/CENTCON52345.2021.9687944

  39. Guo J, Zhao Y, Jiang Y, Song H, Jiang Y (2021) Coverage guided differential adversarial testing of deep learning systems. IEEE Trans Netw Sci Eng 8(2):933–942. https://doi.org/10.1109/TNSE.2020.2997359

    Article  Google Scholar 

  40. Huang S (2020) Influence of different convolutional neural network settings on the performance of MNIST handwritten digits recognition. In: Proceeding of the 2020 international conference on artificial intelligence and education (ICAIE), pp 1–6. https://doi.org/10.1109/ICAIE50891.2020.00008

Download references

Acknowledgements

The work is supported by the National Natural Science Foundation of China (Grant No. 61876138, No.62272387), the Key R & D Project of Shaanxi Province (2020GY-010), the Key Industrial Chain Core Technology Research Project of Xi’an (Grant No.2022JH-RGZN-0028), and the Special Fund for Key Discipline Construction of General Institutions of Higher Learning from Shaanxi Province.

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Xi’an University of Posts and Telecommunications, Xi’an, 710121, China

    Jiaze Sun, Juan Li & Sulei Wen

  2. Shaanxi Key Laboratory of Network Data Analysis and Intelligent Processing, Xi’an, 710121, China

    Jiaze Sun

  3. Xi ‘an Key Laboratory of Big Data and Intelligent Computing, Xi’an, 710127, China

    Jiaze Sun

Authors
  1. Jiaze Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Juan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sulei Wen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Juan Li.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sun, J., Li, J. & Wen, S. DeepMC: DNN test sample optimization method jointly guided by misclassification and coverage. Appl Intell 53, 15787–15801 (2023). https://doi.org/10.1007/s10489-022-04323-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-04323-4

Keywords

Search

Navigation