Skip to main content
SpringerLink
Log in

A similarity metric method of obfuscated malware using function-call graph

  • Original Paper
  • Published:
Journal of Computer Virology and Hacking Techniques Aims and scope Submit manuscript

Abstract

Code obfuscating technique plays a significant role to produce new obfuscated malicious programs, generally called malware variants, from previously encountered malwares. However, the traditional signature-based malware detecting method is hard to recognize the up-to-the-minute obfuscated malwares. This paper proposes a method to identify the malware variants based on the function-call graph. Firstly, the function-call graphs were created from the disassembled codes of program; then the caller–callee relationships of functions and the operational code (opcode) information about functions, combining the graph coloring techniques were used to measure the similarity metric between two function-call graphs; at last, the similarity metric was utilized to identify the malware variants from known malwares. The experimental results show that the proposed method is able to identify the obfuscated malicious softwares effectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Bilar, D.: On callgraphs and generative mechanisms. J. Comput. Virol. 3(4), 285–297 (2007)

    Article  MathSciNet  Google Scholar 

  2. Borello, J.M., Filiol, E., Me, L.: From the design of a generic metamorphic engine to a black-box classi cation of antivirus detection techniques. J. Comput. Virol. 6(3), 277–287 (2010)

    Article  Google Scholar 

  3. Borello, J.M., Me, L.: Code obfuscation techniques for metamorphic viruses. J. Comput. Virol. 4(3), 211–220 (2008)

    Article  Google Scholar 

  4. Borello, J.M., Me, L., Filiol, E.: Dynamic malware detection by similarity measures between behavioral profiles. In: Proceedings of the 2011 Conference on Network and Information Systems Security, IEEE (2011)

  5. Bruschi, D., Martignoni, L., Monga, M.: Using code normalization for fighting self-mutating malware. In: Proceedings of International Symposium on Secure Software Engineering Washington, DC (2006)

  6. Carrera, E., Erdelyi G.: Digital genome mapping-advanced binary malware analysis. In: Proceeding of the 2004 Virus Bulletin Confference, pp. 187–197 (2004)

  7. Christodorescu, M., Jha, S., Kruegel, C.: Mining specifications of malicious behavior. In: Proceedings of the 6th joint meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering, pp. 5–14. ACM, New York (2007)

  8. Fredrikson, M., Jha, S, Christodorescu, M., Sailer, R., Yan, X.F.: Synthesizing near-optimal malware specifications from suspicious behaviors. In: Proceedings of the IEEE Symposium on Security and Privacy, pp. 45–60. IEEE (2010)

  9. Gao, X.B., Xiao, B., Tao, D.C.: A survey of graph edit distance. Pattern Anal. Appl. 13(1), 113–129 (2010)

    Article  MathSciNet  Google Scholar 

  10. Garey, M.R., Johnson, D.S.: Computers and Intractability: A Guide to the Theory of NP-Completeness. WH Freeman & Co, New York (1979)

    MATH  Google Scholar 

  11. Gheorghescu, M.: An automated virus classification system. In: Proceedings of the Virus Bulletin Conference, pp. 294–300 (2005)

  12. Hex-Rays, S.A.: IDA Pro 5.5, http://www.hex-rays.com/products/ida/index.shtml (2010)

  13. Hu, X.: Large-scale malware indexing using function-call graphs. In: Proceedings of the 16th ACM Conference on Computer and Communications Security, pp. 611–620. ACM, New York (2009)

  14. Jeong, K., Lee, H.: Code graph for malware detection. In: Proceedings of the International Conference on Information Networking, IEEE, pp. 1–5 (2008)

  15. Kapoor, A., Spurlock J.: Binary feature extraction and comparison. In: Proceedings of the AVAR 2006, Auckland (2006)

  16. Karnik, A., Goswami, S., Guha, R.: Detecting obfuscated viruses using cosine similarity analysis. In: Proceedings of the First Asia International Conference on Modelling & Simulation (AMS’07), pp. 165170. IEEE Computer Society, Phuket (2007)

  17. Kaspersky.: Monthly malware statistics: May 2009, http://www.kaspersky.com/news?id=207575832 (2010)

  18. Kinable, J., Kostakis, O.: Malware classification based on call graph clustering. J. Comput. Virol. 7(4), 233–245 (2011)

    Article  Google Scholar 

  19. Kostakis, O.: Improved call graph comparison using simulated annealing. In: Proceedings of the 2011 ACM Symposium on Applied Computing, pp. 1516–1523. ACM, New York (2011)

  20. Kruegel, C., Kirda, E.: Polymorphic worm detection using structural information of executable. In: Proceedings of the 8th International Symposium on Recent Advances in Intrusion Detection (RAID 2005), pp. 207–226 (2005)

  21. Lakhotia, A., Kumar, E.U., Venable, M.: A method for detecting obfuscated calls in malicious binaries. IEEE Trans. Softw. Eng. 31(27), 955–967 (2005)

    Article  Google Scholar 

  22. Lee, J., Jeong, K., Lee H.: Detecting metamorphic malwares using code graphs. In: Proceedings of the 2010 ACM Symposium on Applied Computing, pp. 1970–197. ACM, New York (2010)

  23. Li, J., Xu M., Zheng N., Xu. : Malware obfuscation detection via maximal patterns. In: Proceedings of the Third International Symposium on Intelligent Information Technology Application, IEEE. pp. 324–328 (2009)

  24. PEiD 0.95, http://www.peid.info/ (2010)

  25. Scanspyware.http://spyware.scanspyware.net/spywareremoval/rootkit.kernelbot.html (2012)

  26. Securelist.http://www.securelist.com/en/descriptions/old79396 (2012)

  27. Shang, S. H., Zhen, N., Xu, J., Xu, M., Zhang, H. P.: Detecting malware variants via function-call graph similarity. In: Proceedings of the 5th Malicious and Unwanted Software, IEEE, pp. 113–120 (2010)

  28. Symantec.: Internet Security Threat Report, Volume 17. Technical report, Symantec Corporation. http://www.symantec.com/content/en/us/enterprise/other_resources/b-istr_main_report_2011_21239364.en-us.pdf (2011)

  29. Szor, P.: The Art of Computer: Virus Research and Defense, 1st edn. Symantec Press, NJ (2005)

    Google Scholar 

  30. Tabish, S.M., Shaq M.Z., Farooq M.: Limits of static analysis for malware detection. In: Proceedings of the ACSAC, IEEE Computer Society, pp. 421430 (2007)

  31. Tian, R., Batten, L.M., Versteeg, S.C.: Function length as a tool for malware classification. In: Proceedings of the 3rd Malicious and Unwanted Software (MALWARE), pp. 69–76 (2008)

  32. UPX 3.05, http://upx.sourceforge.net/ (2010)

  33. Viruslistjp.http://www.viruslistjp.com/viruses/encyclopedia/?virusid=20425 (2002)

  34. VX Heavens. http://vx.netlux.org/index.html (2010)

  35. Zhang, Q., Reeves. D.S.: MetaAware: identifying metamorphic malware. In: Proceedings of the 23th Annual Computer Security Applications Conference (ACSAC’07), pp. 411–420 (2007)

Download references

Acknowledgments

This paper is supported by NSFC of China (No. 61070212, 61003195); Natural Science Foundation of Zhejiang Province, China (No. Y1090114, No. LY12F02006); the State Key Program of Major Science and Technology (Priority Topics) of Zhejiang Province, China (No 2010C11050). We would like to thank the anonymous reviewers for their helpful comments, suggestions, explanations, and arguments.

Author information

Authors and Affiliations

  1. College of Computer, Hangzhou Dianzi University, Hangzhou, China

    Ming Xu, Lingfei Wu, Shuhui Qi, Jian Xu, Haiping Zhang, Yizhi Ren & Ning Zheng

Authors
  1. Ming Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lingfei Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shuhui Qi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jian Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Haiping Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yizhi Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Ning Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ming Xu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Xu, M., Wu, L., Qi, S. et al. A similarity metric method of obfuscated malware using function-call graph. J Comput Virol Hack Tech 9, 35–47 (2013). https://doi.org/10.1007/s11416-012-0175-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11416-012-0175-y

Keywords

Search

Navigation