Skip to main content

Toward Revealing Kernel Malware Behavior in Virtual Execution Environments

  • Conference paper
Recent Advances in Intrusion Detection (RAID 2009)

Part of the book series: Lecture Notes in Computer Science ((LNSC,volume 5758))

Included in the following conference series:

Abstract

Using a sandbox for malware analysis has proven effective in helping people quickly understand the behavior of unknown malware. This technique is also complementary to other malware analysis techniques such as static code analysis and debugger-based code analysis. This paper presents Rkprofiler, a sandbox-based malware tracking system that dynamically monitors and analyzes the behavior of Windows kernel malware. Kernel malware samples run inside a virtual machine (VM) that is supported and managed by a PC emulator. By building its monitoring component into the PC emulator, Rkprofiler is able to inspect each instruction executed by the kernel malware and therefore possesses a powerful weapon against the malware. Rkprofiler provides several capabilities that other malware tracking systems do not. First, it can detect the execution of malicious kernel code regardless of how the monitored kernel malware is loaded into the kernel and whether it is packed or not. Second, it captures all function calls made by the kernel malware and constructs call graphs from the trace files. Third, a technique called aggressive memory tagging (AMT) is proposed to track the dynamic data objects that the kernel malware visit. Last, Rkprofiler records and reports the hardware access events of kernel malware (e.g., MSR register reads and writes). Our evaluation results show that Rkprofiler can quickly expose the security-sensitive activities of kernel malware and thus reduces the effort exerted in conducting tedious manual malware analysis.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Anubis Project (2009), http://anubis.iseclab.org/?action=home

  2. Abadi, M., Budiu, M., Erlingsson, U., Ligatti, j.: Control-flow integiryt: Principles, implementations, and applications. In: Proceedings of the ACM Conference on Computer and Communications Security, CCS (2005)

    Google Scholar 

  3. BitBlaze Project (2009), http://bitblaze.cs.berkeley.edu/

  4. Baliga, A., Ganapathy, V., Iftode, L.: Automatic Inference and Enforcement of Kernel Data Structure Invariants. In: Proceedings of the 24th Annual Computer Security Applications Conference, ACSAC (2008)

    Google Scholar 

  5. Bellard, F.: QEMU and Kqemu (2009), http://fabrice.bellard.free.fr/qemu/

  6. CBS News. Conficker Wakes Up (2009), http://www.cbsnews.com/stories/2009/04/09/tech/cnettechnews/main4931360.shtml

  7. Chiang, K., Lloyd, L.: A case Study of the Rustock Rootkit and Spam Bot. In: First workshop on hot topics in understanding botnets (2007)

    Google Scholar 

  8. Dr.Web Company. Win32.Ntldrbot (aka Rustock.C) no longer a myth, no longer a threat. New Dr.Web scanner detects and cures it for real (2009), http://info.drweb.com/show/3342/en

  9. Garfinkel, T., Rosenblum, M.: A Virtual Machine Introspection Based Architecture for Intrusion Detection. In: Proceedings of the Symposium on Network and Distributed System Security, NDSS (2003)

    Google Scholar 

  10. Geeg Blog. The Conficker Worm Awakens (2009), http://geeg.info/blog4.php/2009/04/the-conficker-worm-awakens

  11. GraphViz Project (2009), http://www.graphviz.org/

  12. Hoglund, G., Butler, J.: Rootkits: Subverting the Windows Kernel. Addison-Wesley Professional, Reading (2005)

    Google Scholar 

  13. Kruegel, B.C., Robertson, W., Vigna, G.: Detecting Kernel-Level Rootkits through Binary Analysis. In: Proceedings of the 20th Annual Computer Security Applications Conference, ACSAC (2004)

    Google Scholar 

  14. Lanzi, A., Sharif, M., Lee, W.: K-Tracer: A System for Extracting Kernel Malware Behavior. In: Proceeding of the Annual Network and distributed System Security Symposium, NDSS (2009)

    Google Scholar 

  15. Microsoft Symbol Server (2009), http://msdl.microsoft.com/download/symbols

  16. Moser, A., Kruegel, C., Kirda, E.: Exploring multiple execution paths for malware analysis. In: Proceedings of the IEEE Symposium on Security and Privacy (2007)

    Google Scholar 

  17. Petroni, N.L., Fraser, T., Molinz, J., Arbaugh, W.A.: Copilot - a Coprocessor-based Kernel Runtime Integrity Monitor. In: Proceedings of the USENIX Security Symposium (2004)

    Google Scholar 

  18. Petroni, N.L., Fraser, T., Walters, A., Arbaugh, W.A.: An Architecture for Specification-Based Detection of Semantic Integrity Violations in Kernel Dynamic Data. In: Proceedings of the USENIX Security Symposium (2006)

    Google Scholar 

  19. Petroni, N.L., Hicks, M.: Automated Detection of Persistent Kernel Control-Flow Attacks. In: Proceedings of the ACM Conference on Computer and Communications Security, CCS (2007)

    Google Scholar 

  20. Offensivecomputing Website (2009), http://www.offensivecomputing.net/

  21. Rootkit website (2009), http://www.rootkit.com

  22. Riley, R., Jiang, X., Xu, D.: Guest-Transparent Prevention of Kernel Rootkits with VMM-based Memory Shadowing. In: Lippmann, R., Kirda, E., Trachtenberg, A. (eds.) RAID 2008. LNCS, vol. 5230, pp. 1–20. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  23. Riley, R., Jiang, X., Xu, D.: Multi-Aspect Profiling of Kernel Rootkit Behavior. In: Proceedings of the ACM SIGOPS European Conference on Computer Systems, EuroSys (2009)

    Google Scholar 

  24. Shacham, H.: The Geometry of Innocent Flesh on the Bone: Return-into-libc without Function Calls. In: Proceedings of the ACM Conference on Computer and Communications Security, CCS (2007)

    Google Scholar 

  25. Seshadri, A., Luk, M., Qu, N., Perrig, A.: SecVisor: A Tiny Hypervisor to Guarantee Lifetime Kernel Code Integrity for Commodity OSes. In: Proceedings of the ACM Symposium on Operating Systems Principles, SOSP (2007)

    Google Scholar 

  26. Wilhelm, J., Chiueh, T.: A Forced Sampled Execution Approach to Kernel Rootkit Identification. In: Kruegel, C., Lippmann, R., Clark, A. (eds.) RAID 2007. LNCS, vol. 4637, pp. 219–235. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  27. Wang, Z., Jiang, X., Cui, W., Wang, X.: Countering Persistent Kernel Rootkits Through systematic Hook Discovery. In: Lippmann, R., Kirda, E., Trachtenberg, A. (eds.) RAID 2008. LNCS, vol. 5230, pp. 21–38. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  28. Jiang, X., Wang, X., Xu, D.: Stealthy Malware Detection through VMM-Based “Out-of-the-Box” Semantic View Recontruction. In: Proceedings of the ACM Conference on Computer and Communications Security, CCS (2007)

    Google Scholar 

  29. Xuan, C., Copeland, J., Beyah, R.: Shepherding Loadable Kernel Modules through On-demand Emulation. In: Flegel, U., Bruschi, D. (eds.) DIMVA 2009. LNCS, vol. 5587, pp. 48–67. Springer, Heidelberg (2009)

    Google Scholar 

  30. Yin, H., Liang, Z., Song, D.: Hookfinder: Identifying and understanding malware hooking behaviors. In: Proceeding of the Annual Network and distributed System Security Symposium, NDSS (2008)

    Google Scholar 

  31. Yin, H., Song, D., Egele, M., Kruegel, C., Kirda, E.: Panorama: Captureing System-wide Information Flow for Malware Detection and Analysis. In: Proceedings of the ACM Conference on Computer and Communications Security, CCS (2007)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2009 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Xuan, C., Copeland, J., Beyah, R. (2009). Toward Revealing Kernel Malware Behavior in Virtual Execution Environments. In: Kirda, E., Jha, S., Balzarotti, D. (eds) Recent Advances in Intrusion Detection. RAID 2009. Lecture Notes in Computer Science, vol 5758. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-04342-0_16

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-04342-0_16

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-04341-3

  • Online ISBN: 978-3-642-04342-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics