Abstract
Shellcode is malicious binary code whose execution is triggered after the exploitation of a vulnerability. The automated analysis of malicious shellcode is a challenging task, since encryption and evasion techniques are often used. This paper introduces Shellzer, a novel dynamic shellcode analyzer that generates a complete list of the API functions called by the shellcode, and, in addition, returns the binaries retrieved at run-time by the shellcode. The tool is able to modify on-the-fly the arguments and the return values of certain API functions in order to simulate specific execution contexts and the availability of the external resources needed by the shellcode. This tool has been tested with over 24,000 real-world samples, extracted from both web-based drive-by-download attacks and malicious PDF documents. The results of the analysis show that Shellzer is able to successfully analyze 98% of the shellcode samples.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Immunity debugger, http://www.immunityinc.com/products-immdbg.shtml
Ms debugging tools, http://www.microsoft.com/whdc/devtools/debugging/
Libemu (2007), http://libemu.carnivore.it
Anubis (2008), http://anubis.iseclab.org
Wepawet (2008), http://wepawet.cs.ucsb.edu/
Threat expert (2009), http://www.threatexpert.com
Amiri, P.: Pydbg (2005), http://pedram.redhive.com/PyDbg/
Bellard, F.: Qemu (2005), http://www.qemu.org
Borders, K., Prakash, A., Zielinski, M.: Spector: Automatically analyzing shell code. In: Proceeding of the Annual Computer Security Applications Conference, ACSAC (2007)
Cova, M., Kruegel, C., Vigna, G.: Detection and Analysis of Drive-by-Download Attacks and Malicious JavaScript Code. In: Proceedings of the World Wide Web Conference, WWW (2010)
Gu, B., Bai, X., Yang, Z., Champion, A.C., Xuan, D.: Malicious shellcode detection with virtual memory snapshots. In: Proceedings of the IEEE International Conference on Computer Communications, INFOCOM (2010)
Hex-Rays: Ida pro disassembler and debugger, http://www.hex-rays.com/idapro/
Payer, U., Teufl, P., Lamberger, M.: Hybrid engine for polymorphic shellcode detection. In: Julisch, K., Krügel, C. (eds.) DIMVA 2005. LNCS, vol. 3548, pp. 19–31. Springer, Heidelberg (2005)
Polychronakis, M., Anagnostakis, K., Markatos, E.: Network-level polymorphic shellcode detection using emulation. In: Büschkes, R., Laskov, P. (eds.) DIMVA 2006. LNCS, vol. 4064, pp. 54–73. Springer, Heidelberg (2006)
Polychronakis, M., Anagnostakis, K., Markatos, E.: Emulation-based detection of non-self-contained polymorphic shellcode. In: Kruegel, C., Lippmann, R., Clark, A. (eds.) RAID 2007. LNCS, vol. 4637, pp. 87–106. Springer, Heidelberg (2007)
Polychronakis, M., Anagnostakis, K., Markatos, E.: Comprehensive shellcode detection using runtime heuristics. In: Proceeding of the Annual Computer Security Applications Conference, ACSAC (2010)
Shields, T.: Anti-debugging - a developers view
Singh, A.: Identifying Malicious Code Through Reverse Engineering. Springer, Heidelberg (2009)
Skape: Understanding windows shellcode, http://www.hick.org/code/skape/papers/win32-shellcode.pdf
Yuschuk, O.: Ollydbg (2005), http://www.ollydbg.de/
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Fratantonio, Y., Kruegel, C., Vigna, G. (2011). Shellzer: A Tool for the Dynamic Analysis of Malicious Shellcode. In: Sommer, R., Balzarotti, D., Maier, G. (eds) Recent Advances in Intrusion Detection. RAID 2011. Lecture Notes in Computer Science, vol 6961. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-23644-0_4
Download citation
DOI: https://doi.org/10.1007/978-3-642-23644-0_4
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-23643-3
Online ISBN: 978-3-642-23644-0
eBook Packages: Computer ScienceComputer Science (R0)