Skip to main content
SpringerLink
Log in

Detection and robustness evaluation of android malware classifiers

  • Original Paper
  • Published:
Journal of Computer Virology and Hacking Techniques Aims and scope Submit manuscript

Abstract

Android malware attacks are tremendously increasing, and evasion techniques become more and more effective. For this reason, it is necessary to continuously improve the detection performances. With this paper, we wish to pursue this purpose with two contributions. On one hand, we aim at evaluating how improving machine learning-based malware detectors, and on the other hand, we investigate to which extent adversarial attacks can deteriorate the performances of the classifiers. Analysis of malware samples is performed using static and dynamic analysis. This paper proposes a framework for integrating both static and dynamic features trained on machine learning methods and deep neural network. On employing machine learning algorithms, we obtain an accuracy of 97.59% with static features using SVM, and 95.64% is reached with dynamic features using Random forest. Additionally, a 100% accuracy was obtained with CART and SVM using hybrid attributes (on combining relevant static and dynamic features). Further, using deep neural network models, experimental results showed an accuracy of 99.28% using static features, 94.61% using dynamic attributes, and 99.59% by combining both static and dynamic features (also known as multi-modal attributes). Besides, we evaluated the robustness of classifiers against evasion and poisoning attack. In particular comprehensive analysis was performed using permission, APIs, app components and system calls (especially n-grams of system calls). We noticed that the performances of the classifiers significantly dropped while simulating evasion attack using static features, and in some cases 100% of adversarial examples were wrongly labelled by the classification models. Additionally, we show that models trained using dynamic features are also vulnerable to attack, however they exhibit more resilience than a classifier built on static features.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Notes

  1. https://www.virustotal.com/gui/.

References

  1. Abou-Assaleh, T., Cercone, N., Keselj, V., Sweidan, R.: N-gram-based detection of new malicious code. In: Proceedings of the 28th Annual International Computer Software and Applications Conference, 2004. COMPSAC 2004., vol. 2, pp. 41–42. IEEE (2004)

  2. Afonso, V.M., de Amorim, M.F., Grégio, A.R.A., Junquera, G.B., de Geus, P.: Identifying Android malware using dynamically obtained features. J. Comput. Virol. Hacking Tech. 11(1), 9–17 (2015)

    Article  Google Scholar 

  3. Agarap, A.F.: Towards building an intelligent anti-malware system: a deep learning approach using support vector machine (svm) for malware classification. arXiv preprint arXiv:1801.00318 (2017)

  4. Almin, S.B., Chatterjee, M.: A novel approach to detect android malware. Procedia Comput. Sci. 45, 407–417 (2015)

    Article  Google Scholar 

  5. Aonzo, S., Georgiu, G.C., Verderame, L., Merlo, A.: Obfuscapk: an open-source black-box obfuscation tool for Android apps. SoftwareX 11, 100403 (2020)

    Article  Google Scholar 

  6. APKTool: https://ibotpeaches.github.io/Apktool/install/

  7. Arp, D., Spreitzenbarth, M., Hubner, M., Gascon, H., Rieck, K., Siemens, C.E.R.T.: Drebin: effective and explainable detection of android malware in your pocket. In: Ndss, vol. 14, pp. 23–26. (2014)

  8. Arshad, S., Shah, M.A., Wahid, A., Mehmood, A., Song, H., Hongnian, Y.: Samadroid: a novel 3-level hybrid malware detection model for android operating system. IEEE Access 6, 4321–4339 (2018)

    Article  Google Scholar 

  9. Bernardi, M.L., Cimitile, M., Distante, D., Martinelli, F., Mercaldo, F.: Dynamic malware detection and phylogeny analysis using process mining. Int. J. Inf. Secur. 18(3), 257–284 (2019)

    Article  Google Scholar 

  10. Biggio, B., Fabio, R.: Wild patterns: ten years after the rise of adversarial machine learning. Pattern Recognit. 84, 317–331 (2018)

    Article  Google Scholar 

  11. Burguera, I., Zurutuza, U., Nadjm-Tehrani, S.: Crowdroid: behavior-based malware detection system for android. In: Proceedings of the 1st ACM Workshop on Security and Privacy in Smartphones and Mobile Devices, pp. 15–26. (2011)

  12. Chavan, N., Di Troia, F., Stamp, M.: A comparative analysis of android malware. arXiv preprint arXiv:1904.00735 (2019)

  13. Chen, L., Hou, S., Ye, Y., Xu, S.: Droideye: fortifying security of learning-based classifier against adversarial android malware attacks. In: 2018 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), pp. 782–789. IEEE (2018)

  14. Chen, S., Xue, M., Fan, L., Hao, S., Xu, L., Zhu, H., Li, B.: Automated poisoning attacks and defenses in malware detection systems: an adversarial machine learning approach. Comput. Secur. 73, 326–344 (2018)

    Article  Google Scholar 

  15. Chuang, H.Y., Wang, S.D.: Machine learning based hybrid behavior models for Android malware analysis. In: 2015 IEEE International Conference on Software Quality, Reliability and Security, pp. 201–206. IEEE (2015)

  16. Damodaran, A., Di Troia, F., Visaggio, C.A., Austin, T.H., Stamp, M.: A comparison of static, dynamic, and hybrid analysis for malware detection. J. Comput. Virol. Hacking Techn. 13(1), 1–12 (2017)

    Article  Google Scholar 

  17. Demonits, A., Melis, M., Biggio, B., Maiorca, D.A., Rieck,K., Corona, I., Giacinto, G., Roli, F.: Yes, machine learning can be more secure! a case study on android malware detection. In: IEEE Transactions on Dependable and Secure Computing, vol.16, pp. 711–723. IEEE (2019)

  18. Dimjašević, M., Atzeni, S., Ugrina, I., Rakamaric, Z.: Evaluation of android malware detection based on system calls. In: Proceedings of the 2016 ACM on International Workshop on Security And Privacy Analytics, pp. 1–8. (2016)

  19. Dong, Y., Liao, F., Pang, T., Su, H., Zhu, J., Hu, X., Li, J.: Boosting adversarial attacks with momentum. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9185–9193. (2018)

  20. Gandotra, E., Bansal, Di., Sofat, S.: Malware analysis and classification: a survey. J. Inf. Secur. 2014 (2014)

  21. Garcia, J., Hammad, M., Malek, S.: Lightweight, obfuscation-resilient detection and family identification of android malware. ACM Trans. Softw. Eng. Methodol. (TOSEM) 26(3), 1–29 (2018)

    Article  Google Scholar 

  22. Greengard, S.: Cybersecurity gets smart. Commun. ACM 59(5), 29–31 (2016)

    Article  Google Scholar 

  23. Grosse, K., Papernot, N., Manoharan, P., Backes, M.l, McDaniel, P.: Adversarial examples for malware detection. In: European Symposium on Research in Computer Security, pp. 62–79. Springer, Cham (2017)

  24. Karbab, E.B., Debbabi, M., Derhab, A., Mouheb, D.: MalDozer: automatic framework for android malware detection using deep learning. Digit. Investig. 24, S48–S59 (2018)

    Article  Google Scholar 

  25. Kim, T.G., Kang, B.J., Rho, M., Sezer, S., Im, E.G.: A multimodal deep learning method for android malware detection using various features. IEEE Trans. Inf. Forensics Secur. 14(3), 773–788 (2018)

    Article  Google Scholar 

  26. Le, Q., Boydell, O., Namee, B.M., Scanlon, M.: Deep learning at the shallow end: malware classification for non-domain experts. Digit. Investig. 26, S118–S126 (2018)

    Article  Google Scholar 

  27. Li, J., Sun, L., Yan, Q., Li, Z., Srisa-An, W., Ye, H.: Significant permission identification for machine-learning-based android malware detection. IEEE Trans. Ind. Inf. 14(7), 3216–3225 (2018)

    Article  Google Scholar 

  28. Martinelli, F., Marulli, F., Mercaldo, F.: Evaluating convolutional neural network for effective mobile malware detection. Procedia Comput. Sci. 112, 2372–2381 (2017)

    Article  Google Scholar 

  29. McLaughlin, N., del Rincon, J.M., Kang, B.J., Yerima, S., Miller, S., Sakir, S., et al.: Deep android malware detection. In: Proceedings of the Seventh ACM on Conference on Data and Application Security and Privacy, pp. 301–308. (2017)

  30. MonkeyRunner:https://developer.android.com/studio/test/monkey

  31. Nataraj, L., Karthikeyan, S., Jacob, G., Manjunath, B.S.: Malware images: visualization and automatic classification. In: Proceedings of the 8th International Symposium on Visualization for Cyber Security, pp. 1–7. (2011)

  32. Ni, S., Qian, Q., Zhang, R.: Malware identification using visualization images and deep learning. Comput. Secur. 77, 871–885 (2018)

    Article  Google Scholar 

  33. Patel, K., Buddadev, B.: Detection and mitigation of android malware through hybrid approach. In International symposium on Security in Computing and Communication, pp. 455–463. Springer, Cham, (2015)

  34. Pendlebury, F., Pierazzi, F., Jordaney, R., Kinder, J., Cavallaro, L.: TESSERACT: eliminating experimental bias in malware classification across space and time. In: 28th USENIX Security Symposium (USENIX Security 19), pp. 729–746. (2019)

  35. Pierazzi, F., Pendlebury, F., Cortellazzi, J., Cavallaro, L.: Intriguing properties of adversarial ML attacks in the problem space. In: Proceedings of IEEE Symposium on Security and Privacy, 2020, pp.1332–1349. IEEE (2020)

  36. SL, S.D., Jaidhar, C.D.: Windows malware detector using convolutional neural network based on visualization images. IEEE Trans. Emerg. Top. Comput. (2019)

  37. Santos, I., Penya, Y.K., Devesa, J., Bringas, P.G.: N-grams-based file signatures for malware detection. ICEIS 9, 317–320 (2009)

    Google Scholar 

  38. Saracino, A., Sgandurra, D., Dini, G., Martinelli, F.: Madam: effective and efficient behavior-based android malware detection and prevention. IEEE Trans. Dependable Secure Comput. 15(1), 83–97 (2016)

    Article  Google Scholar 

  39. Saxe, J., Berlin, K.: Deep neural network based malware detection using two dimensional binary program features. In: 2015 10th International Conference on Malicious and Unwanted Software (MALWARE), pp. 11–20. IEEE (2015)

  40. Sen, S., Aydogan, E., Aysan, A.I.: Coevolution of mobile malware and anti-malware. IEEE Trans. Inf. Forensics Secur. 13(10), 2563–2574 (2018)

    Article  Google Scholar 

  41. Sun, G., Qian, Q.: Deep learning and visualization for identifying malware families. IEEE Trans. Dependable Secure Comput. (2018)

  42. Surendran, R., Thomas, T., Emmanuel, S.: GSDroid: graph signal based compact feature representation for android malware detection. Expert Syst. Appl. 159, 113581 (2020)

    Article  Google Scholar 

  43. Tam, K., Khan, S.J., Fattori, A., Cavallaro, L.: Copperdroid: automatic reconstruction of android malware behaviors. In: Ndss. (2015)

  44. Ucci, D., Leonardo, A., Roberto, B.: Survey of machine learning techniques for malware analysis. Comput. Secur. 81, 123–147 (2019)

    Article  Google Scholar 

  45. Vinayakumar, R., Soman, K.P.: DeepMalNet: evaluating shallow and deep networks for static PE malware detection. ICT Express 4(4), 255–258 (2018)

    Article  Google Scholar 

  46. Wang, X., Yang, Y., Zeng, Y., Tang, C., Shi, J., Xu, K.: A novel hybrid mobile malware detection system integrating anomaly detection with misuse detection. In: Proceedings of the 6th International Workshop on Mobile Cloud Computing and Services, pp. 15–22. (2015)

  47. Wu, W.C., Hung, S.H.: DroidDolphin: a dynamic Android malware detection framework using big data and machine learning. In: Proceedings of the 2014 Conference on Research in Adaptive and Convergent Systems, pp. 247–252. (2014)

  48. Xiao, X., Wang, Z., Li, Q., Xia, S., Jiang, Y.: Back-propagation neural network on Markov chains from system call sequences: a new approach for detecting Android malware with system call sequences. IET Inf. Secur. 11(1), 8–15 (2017)

    Article  Google Scholar 

  49. Xu, W., Qi, Y., Evans, D.: Automatically evading classifiers. In: Proceedings of the 2016 Network and Distributed Systems Symposium, vol. 10. (2016)

  50. Xue, Y., Meng, G., Liu, Y., Tan, T.H., Chen, H., Sun, J., Zhang, J.: Auditing anti-malware tools by evolving android malware and dynamic loading technique. IEEE Trans. Inf. Forensics Secur. 12(7), 1529–1544 (2017)

    Article  Google Scholar 

  51. Zhang, S., Xiao, X.: Cscdroid: Accurately detect android malware via contribution-level-based system call categorization. In 2017 IEEE Trustcom/BigDataSE/ICESS, pp. 193–200. IEEE (2017)

  52. Zhou, M.: A hybrid feature selection method based on fisher score and genetic algorithm. J. Math. Sci. Adv. Appl. 37(1), 51–78 (2016)

    Article  Google Scholar 

Download references

Author information

Author notes

  1. M. L. Anupama

    Present address: Department of Computer Science and Engineering, SCMS School of Engineering and Technology, Cochin, India

Authors and Affiliations

  1. Department of Computer Science and Engineering, SCMS School of Engineering and Technology, Cochin, India

    M. A. Arya, Josna Philomina, Anson Pinhero & K. S. Ajith

  2. Department of Computer Applications, Cochin University of Science and Technology, Cochin, India

    P. Vinod

  3. Department of Engineering, University of Sannio, Benevento, Italy

    Corrado Aaron Visaggio

  4. Department of Computer Science and Engineering, Sri Ramakrishna Engineering College, Coimbatore, Affiliated by Anna University, Chennai, India

    Rincy Raphael & P. Mathiyalagan

Authors
  1. M. L. Anupama
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. P. Vinod
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Corrado Aaron Visaggio
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. M. A. Arya
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Josna Philomina
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Rincy Raphael
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Anson Pinhero
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. K. S. Ajith
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. P. Mathiyalagan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Corrado Aaron Visaggio.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Anupama, M.L., Vinod, P., Visaggio, C.A. et al. Detection and robustness evaluation of android malware classifiers. J Comput Virol Hack Tech 18, 147–170 (2022). https://doi.org/10.1007/s11416-021-00390-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11416-021-00390-2

Keywords

Search

Navigation