Skip to main content
SpringerLink
Log in

Hybrid Analysis and Control of Malware

  • Conference paper
Recent Advances in Intrusion Detection (RAID 2010)

Part of the book series: Lecture Notes in Computer Science ((LNSC,volume 6307))

Included in the following conference series:

Abstract

Malware attacks necessitate extensive forensic analysis efforts that are manual-labor intensive because of the analysis-resistance techniques that malware authors employ. The most prevalent of these techniques are code unpacking, code overwriting, and control transfer obfuscations. We simplify the analyst’s task by analyzing the code prior to its execution and by providing the ability to selectively monitor its execution. We achieve pre-execution analysis by combining static and dynamic techniques to construct control- and data-flow analyses. These analyses form the interface by which the analyst instruments the code. This interface simplifies the instrumentation task, allowing us to reduce the number of instrumented program locations by a hundred-fold relative to existing instrumentation-based methods of identifying unpacked code. We implement our techniques in SD-Dyninst and apply them to a large corpus of malware, performing analysis tasks such as code coverage tests and call-stack traversals that are greatly simplified by hybrid analysis.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Computer economics 2007 malware report: The economic impact of viruses, spyware, adware, botnets, and other malicious code (2007)

    Google Scholar 

  2. Darkparanoid virus (1998)

    Google Scholar 

  3. Offensive computing, http://www.offensivecomputing.net

  4. Anckaert, B., Madou, M., Bosschere, K.D.: A model for self-modifying code. In: Information Hiding, Alexandria, VA, pp. 232–248 (2007)

    Google Scholar 

  5. Balakrishnan, G., Reps, T.: Analyzing memory accesses in x86 executables. In: International Conference on Compiler Construction, New York, NY, pp. 5–23 (2004)

    Google Scholar 

  6. Bayer, U., Moser, A., Kruegel, C., Kirda, E.: Dynamic analysis of malicious code. Journal in Computer Virology 2(1), 66–77 (2006)

    Article  Google Scholar 

  7. Bellard, F.: QEMU, a fast and portable dynamic translator. In: USENIX Annual Technical Conference, Anaheim, CA, pp. 41–46 (2005)

    Google Scholar 

  8. BitDefender: BitDefender anti-virus technology. White Paper (2007)

    Google Scholar 

  9. Bustamante, P.: Malware prevalence. Panda Research web article (2008)

    Google Scholar 

  10. Bustamante, P.: Packer (r)evolution. Panda Research web article (2008)

    Google Scholar 

  11. Bustamante, P.: Personal correspondence (2009)

    Google Scholar 

  12. Chiang, K., Lloyd, L.: A case study of the rustock rootkit and spam bot. In: First Conference on Hot Topics in Understanding Botnets, Cambridge, MA (2007)

    Google Scholar 

  13. Cifuentes, C., Emmerik, M.V.: UQBT: adaptable binary translation at low cost. Computer 33(3), 60–66 (2000)

    Article  Google Scholar 

  14. Collberg, C., Thomborson, C., Low, D.: Manufacturing cheap, resilient, and stealthy opaque constructs. In: Symposium on Principles of Programming Languages, San Diego, CA, pp. 184–196 (1998)

    Google Scholar 

  15. Coogan, K., Debray, S., Kaochar, T., Townsend, G.: Automatic static unpacking of malware binaries. In: Working Conference on Reverse Engineering, Antwerp, Belgium (2009)

    Google Scholar 

  16. Danehkar, A.: Inject your code into a portable executable file (2005), http://www.codeproject.com/KB/system/inject2exe.aspx

  17. Dinaburg, A., Royal, P., Sharif, M., Lee, W.: Ether: Malware analysis via hardware virtualization extensions. In: Conference on Computer and Communications Security, Alexandria, VA (2008)

    Google Scholar 

  18. Ferrie, P.: Anti-unpacker tricks. In: International CARO Workshop. Amsterdam, Netherlands (2008)

    Google Scholar 

  19. Garfinkel, T., Rosenblum, M.: A virtual machine introspection based architecture for intrusion detection. In: Network and Distributed System Security Symposium, San Diego, CA (2003)

    Google Scholar 

  20. Guo, F., Ferrie, P., Chiueh, T.: A study of the packer problem and its solutions. In: Lippmann, R., Kirda, E., Trachtenberg, A. (eds.) RAID 2008. LNCS, vol. 5230, pp. 98–115. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  21. Hind, M., Pioli, A.: Which pointer analysis should I use? In: International Symposium on Software Testing and Analysis, Portland, OR, pp. 113–123 (2000)

    Google Scholar 

  22. Hollingsworth, J.K., Miller, B.P., Cargille, J.: Dynamic program instrumentation for scalable performance tools. In: Scalable High Performance Computing Conference, Knoxville, TN (1994)

    Google Scholar 

  23. Kang, M.G., Poosankam, P., Yin, H.: Renovo: A hidden code extractor for packed executables. In: Workshop on Recurring Malcode, Alexandria, VA (2007)

    Google Scholar 

  24. Kruegel, C., Robertson, W., Valeur, F., Vigna, G.: Static disassembly of obfuscated binaries. In: USENIX Security Symposium, San Diego, CA (2004)

    Google Scholar 

  25. Linn, C., Debray, S.: Obfuscation of executable code to improve resistance to static disassembly. In: Conference on Computer and Communications Security, Washington, DC, pp. 290–299 (2003)

    Google Scholar 

  26. Linn, C., Debray, S., Andrews, G., Schwarz, B.: Stack analysis of x86 executables (2004) (manuscript)

    Google Scholar 

  27. Luk, C.K., Cohn, R., Muth, R., Patil, H., Klauser, A., Lowney, G., Wallace, S., Reddi, V.J., Hazelwood, K.: Pin: Building customized program analysis tools with dynamic instrumentation. In: Programming Language Design and Implementation, Chicago, IL, pp. 190–200 (2005)

    Google Scholar 

  28. Madou, M., Anckaert, B., de Sutter, B., Bosschere, K.D.: Hybrid static-dynamic attacks against software protection mechanisms. In: ACM Workshop on Digital Rights Management, Alexandria, VA, pp. 75–82 (2005)

    Google Scholar 

  29. Maebe, J., Bosschere, K.D.: Instrumenting self-modifying code. In: International Workshop on Automated and Algorithmic Debugging, Ghent, Belgium (2003)

    Google Scholar 

  30. Martignoni, L., Christodorescu, M., Jha, S.: Omniunpack: Fast, generic, and safe unpacking of malware. In: Annual Computer Security Applications Conference, Miami Beach, FL (2007)

    Google Scholar 

  31. Mirgorodskiy, A.V., Miller, B.P.: Autonomous analysis of interactive systems with self-propelled instrumentation. In: International Conference on Parallel Computing, San Jose, CA (2005)

    Google Scholar 

  32. Moser, A., Kruegel, C., Kirda, E.: Exploring multiple execution paths for malware analysis. In: Symposium on Security and Privacy, Oakland, CA, pp. 231–245 (2007)

    Google Scholar 

  33. Moser, A., Kruegel, C., Kirda, E.: Limits of static analysis for malware detection. In: Annual Computer Security Applications Conference, Miami Beach, FL (2007)

    Google Scholar 

  34. Nanda, S., Li, W., Lam, L.C., Cker Chiueh, T.: Bird: Binary interpretation using runtime disassembly. In: International Symposium on Code Generation and Optimization (CGO 2006), New York, NY, pp. 358–370 (2006)

    Google Scholar 

  35. Neumann, R.: Exepacker blacklisting part 2. Virus Bulletin pp. 10–13 (2007)

    Google Scholar 

  36. Nguyen, A.M., Schear, N., Jung, H., Godiyal, A., King, S.T., Nguyen, H.: Mavmm: A lightweight and purpose-built vmm for malware analysis. In: Annual Computer Security Applications Conference, Honolulu, HI (2009)

    Google Scholar 

  37. Perriot, F., Ferrie, P.: Principles and practise of x-raying. In: Virus Bulletin Conference, Chicago, IL, pp. 51–66 (2004)

    Google Scholar 

  38. Popov, I., Debray, S., Andrews, G.: Binary obfuscation using signals. In: USENIX Security Symposium, Boston, MA, pp. 275–290 (2007)

    Google Scholar 

  39. Porras, P., Saidi, H., Yegneswaran, V.: A multi-perspective analysis of the storm (peacomm) worm. SRI International Technical Report (2007)

    Google Scholar 

  40. Porras, P., Saidi, H., Yegneswaran, V.: An analysis of conficker’s logic and rendezvous points. SRI International Technical Report (2009)

    Google Scholar 

  41. Quist, D., Ames, C.: Temporal reverse engineering. In: Blackhat, USA, Las Vegas, NV (2008)

    Google Scholar 

  42. Quist, D.A., Liebrock, L.M.: Visualizing compiled executables for malware analysis. In: Workshop on Visualization for Cyber Security, Atlantic City, NJ (2009)

    Google Scholar 

  43. Rosenblum, N.E., Zhu, X., Miller, B.P., Hunt, K.: Learning to analyze binary computer code. In: Conference on Artificial Intelligence, Chicago, IL (2008)

    Google Scholar 

  44. Royal, P., Halpin, M., Dagon, D., Edmonds, R., Lee, W.: PolyUnpack: Automating the hidden-code extraction of unpack-executing malware. In: Annual Computer Security Applications Conference, Miami Beach, FL, pp. 289–300 (2006)

    Google Scholar 

  45. Security, P.: Annual report Pandalabs (2008)

    Google Scholar 

  46. Sharif, M., Lanzi, A., Giffin, J., Lee, W.: Impeding malware analysis using conditional code obfuscation. In: Network and Distributed System Security Symposium, San Diego, CA (2008)

    Google Scholar 

  47. Sites, R.L., Chernoff, A., Kirk, M.B., Marks, M.P., Robinson, S.G.: Binary translation. Communications of the ACM 36(2), 69–81 (1993)

    Article  Google Scholar 

  48. Srivastava, A., Edwards, A., Vo, H.: Vulcan: Binary transformation in a distributed environment. Technical Report MSR-TR-2001-50 (2001)

    Google Scholar 

  49. Srivastava, A., Eustace, A.: ATOM: a system for building customized program analysis tools. In: Programming Language Design and Implementation, Orlando, FL (1994)

    Google Scholar 

  50. Trilling, S.: Project green bay–calling a blitz on packers. In: CIO Digest: Strategies and Analysis from Symantec, p. 4 (2008)

    Google Scholar 

  51. Vigna, G.: Static disassembly and code analysis. In: Malware Detection. Advances in Information Security, vol. 35, pp. 19–42. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  52. Yegneswaran, V., Saidi, H., Porras, P.: Eureka: A framework for enabling static analysis on malware. Technical Report SRI-CSL-08-01 (2008)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Sciences Department, University of Wisconsin,  

    Kevin A. Roundy & Barton P. Miller

Authors
  1. Kevin A. Roundy
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Barton P. Miller
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Computer Sciences Department, University of Wisconsin, 53706, Madison, WI, USA

    Somesh Jha

  2. International Computer Science Institute, 1947 Center Street, Suite 600, 94704, Berkeley, CA, USA

    Robin Sommer  & Christian Kreibich  & 

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Roundy, K.A., Miller, B.P. (2010). Hybrid Analysis and Control of Malware. In: Jha, S., Sommer, R., Kreibich, C. (eds) Recent Advances in Intrusion Detection. RAID 2010. Lecture Notes in Computer Science, vol 6307. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15512-3_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-15512-3_17

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-15511-6

  • Online ISBN: 978-3-642-15512-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation