skip to main content
10.1145/1966913.1966986acmconferencesArticle/Chapter ViewAbstractPublication Pagesasia-ccsConference Proceedingsconference-collections
short-paper

Deriving common malware behavior through graph clustering

Published:22 March 2011Publication History

ABSTRACT

Detection of malicious software (malware) continues to be a problem as hackers devise new ways to evade available methods. The proliferation of malware and malware variants requires methods that are both powerful, and fast to execute. This paper proposes a method to derive the common execution behavior of a family of malware instances. For each instance, a graph is constructed that represents kernel objects and their attributes, based on system call traces. The method combines these graphs to develop a supergraph for the family. This supergraph contains a subgraph, called the HotPath, which is observed during the execution of all the malware instances. The proposed method is scalable, identifies previously-unseen malware instances, shows high malware detection rates, and false positive rates close to 0%.

References

  1. M. Bailey, J. Oberheide, J. Andersen, Z. M. Mao, and F. J. and Jose Nazario. Automated classification and analysis of internet malware. In Proceedings of 10th International Symposium in Recent Advances in Intrusion Detection(RAID), volume 4637 of Lecture Notes in Computer Science, pages 178--197, Gold Goast, Australia, September 2007. Springer. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. U. Bayer, P. Milani Comparetti, C. Hlauscheck, C. Kruegel, and E. Kirda. Scalable, Behavior-Based Malware Clustering. In 16th Symposium on Network and Distributed System Security (NDSS), 2009.Google ScholarGoogle Scholar
  3. H. Bunke, P. Foggia, C. Guidobaldi, and M. Vento. Graph clustering using the weighted minimum common supergraph. In Graph Based Representations in Pattern Recognition, volume 2726 of Lecture Notes in Computer Science, pages 235--246. Springer, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. C. Cadar, V. Ganesh, P. M. Pawlowski, D. L. Dill, and D. R. Engler. Exe: automatically generating inputs of death. In Proceedings of the 13th ACM conference on Computer and communications security(CCS), pages 322--335, New York, NY, USA, 2006. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. M. Christodorescu and S. Jha. Testing malware detectors. In Proceedings of the 2004 ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2004), pages 34--44, Boston, MA, USA, July 2004. ACM Press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. M. Christodorescu, S. Jha, and C. Kruegel. Mining specifications of malicious behavior. In Proceedings of ACM SIGSOFT symposium on The foundations of software engineering (FSE), pages 5--14, New York, NY, USA, 2007. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. M. Christodorescu, S. Jha, S. A. Seshia, D. X. Song, and R. E. Bryant. Semantics-aware malware detection. In IN IEEE SYMPOSIUM ON SECURITY AND PRIVACY, pages 32--46, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. D. Conte, P. Foggia, and M. Vento. Challenging complexity of maximum common subgraph detection algorithms: A performance analysis of three algorithms on a wide database of graphs. Journal of Graph Algorithms Applications, 11(1):99--143, 2007.Google ScholarGoogle ScholarCross RefCross Ref
  9. D. J. Cook and L. B. Holder. Mining Graph Data. John Wiley & Sons, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. S. Corp. Symantec global internet security threat report, April 2008. http://www.symantec.com/.Google ScholarGoogle Scholar
  11. A. Dinaburg, P. Royal, M. I. Sharif, and W. Lee. Ether: malware analysis via hardware virtualization extensions. In ACM Conference on Computer and Communications Security, pages 51--62, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. M. Fredrikson, S. Jha, M. Christodorescu, R. Sailer, and X. Yan. Synthesizing near-optimal malware specifications from suspicious behaviors. In Proceedings of 31st IEEE Symposium on Security and Privacy (S&P), May 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. X. Hu, T.-c. Chiueh, and K. G. Shin. Large-scale malware indexing using function-call graphs. In Proceedings of the 16th ACM conference on Computer and communications security(CCS'09), pages 611--620, Chicago, Illinois, USA, 2009. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. J. Kinder, S. Katzenbeisser, C. Schallhart, H. Veith, and T. U. München. Detecting malicious code by model checking. In Proceedings of International Conference on Intrusion and Malware Detection and Vulnerability Assessment (DIMVA), pages 174--187. Springer Berlin, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. E. Kirda, C. Kruegel, G. Banks, G. Vigna, and R. Kemmerer. Behavior-based spyware detection. In Proceedings of the 15th Usenix Security Symposium, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. C. Kolbitsch, T. Holz, C. Kruegel, and E. Kirda. Inspector gadget: Automated extraction of proprietary gadgets from malware binaries. In Proceedings of the 31st IEEE Symposium on Security & Privacy (Oakland'10), May 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. C. Kolbitsch, P. Milani Comparetti, C. Kruegel, E. Kirda, X. Zhou, and X. Wang. Effective and Efficient Malware Detection at the End Host. In 18th Usenix Security Symposium, Montreal, Canada, August 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. C. Kruegel, W. Robertson, and G. Vigna. Detecting kernel-level rootkits through binary analysis. In Proceedings of the 20th Annual Computer Security Applications Conference (ACSAC), pages 91--100, Washington, DC, USA, 2004. IEEE Computer Society. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. C. Krugel, E. Kirda, D. Mutz, W. K. Robertson, and G. Vigna. Polymorphic worm detection using structural information of executables. In RAID, pages 207--226, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. M. Library. Kernel object. http://msdn.microsoft.com/en-us/library/ms724485(VS.85).aspx.Google ScholarGoogle Scholar
  21. A. Moser, C. Kruegel, and E. Kirda. Exploring multiple execution paths for malware analysis. In Proceedings of the 2007 IEEE Symposium on Security and Privacy (S&P), pages 231--245, Washington, DC, USA, 2007. IEEE Computer Society. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. A. Moser, C. Kruegel, and E. Kirda. Limits of static analysis for malware detection. Computer Security Applications Conference, Annual, 0:421--430, 2007.Google ScholarGoogle ScholarCross RefCross Ref
  23. Y. Park, Q. Zhang, D. Reeves, and V. Mulukutla. Antibot: Clustering common semantic patterns for bot detection. In Proceedings of 34th Annual IEEE International Computer Software and Applications Conference(COMPSAC), July 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. D. Perry. Here comes the flood or end of the pattern file. In Virus Bulletin, Ottawa, 2008.Google ScholarGoogle Scholar
  25. D. Wagner and R. Dean. Intrusion Detection via Static Analysis. In Proceedings 2001 IEEE Symposium on Security and Privacy(S&P), pages 156--168, Oakland, CA, USA, May 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. C. Willems, T. Holz, and F. Freiling. Toward automated dynamic malware analysis using cwsandbox. IEEE Security and Privacy, 5(2):32--39, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. B. Xin and X. Zhang. Memory slicing. In Proceedings of the eighteenth international symposium on Software testing and analysis (ISSTA '09), pages 165--176, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. H. Yin, D. Song, M. Egele, C. Kruegel, and E. Kirda. Panorama: capturing system-wide information flow for malware detection and analysis. In Proceedings of the 14th ACM conference on Computer and communications security(CCS), pages 116--127, New York, NY, USA, 2007. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Q. Zhang and D. S. Reeves. Metaaware: Identifying metamorphic malware. In 23rd Annual Computer Security Applications Conference (ACSAC), pages 411--420, 2007.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Deriving common malware behavior through graph clustering

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in
        • Published in

          cover image ACM Conferences
          ASIACCS '11: Proceedings of the 6th ACM Symposium on Information, Computer and Communications Security
          March 2011
          527 pages
          ISBN:9781450305648
          DOI:10.1145/1966913

          Copyright © 2011 ACM

          Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 22 March 2011

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • short-paper

          Acceptance Rates

          ASIACCS '11 Paper Acceptance Rate35of217submissions,16%Overall Acceptance Rate418of2,322submissions,18%

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader