Skip to main content
SpringerLink
Log in
Book cover

International Conference on Security, Privacy, and Applied Cryptography Engineering

SPACE 2016: Security, Privacy, and Applied Cryptography Engineering pp 281–300Cite as

VMI Based Automated Real-Time Malware Detector for Virtualized Cloud Environment

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNSC,volume 10076))

Abstract

The Virtual Machine Introspection (VMI) has evolved as a promising future security solution to performs an indirect investigation of the untrustworthy Guest Virtual Machine (GVM) in real-time by operating at the hypervisor in a virtualized cloud environment. The existing VMI techniques are not intelligent enough to read precisely the manipulated semantic information on their reconstructed high-level semantic view of the live GVM. In this paper, a VMI-based Automated-Internal-External (A-IntExt) system is presented that seamlessly introspects the untrustworthy Windows GVM internal semantic view (i.e. Processes) to detect the hidden, dead, and malicious processes. Further, it checks the detected, hidden as well as running processes (not hidden) as benign or malicious. The prime component of the A-IntExt is the Intelligent Cross-View Analyzer (ICVA), which is responsible for detecting hidden-state information from internally and externally gathered state information of the Monitored Virtual Machine (\(M_{ed-VM}\)). The A-IntExt is designed, implemented, and evaluated by using publicly available malware and Windows real-world rootkits to measure detection proficiency as well as execution speed. The experimental results demonstrate that A-IntExt is effective in detecting malicious and hidden-state information rapidly with maximum performance overhead of 7.2 %.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    http://libvmi.com/.

  2. 2.

    Dubious Processes (DPs) are current state of executable processes it includes both benign and malicious processes (not hidden) on the \(M_{ing-VM}\). Existing hypervisor-based VMI systems are not intelligent enough to detect and identify actual malicious processes that are running or attached to a benign one.

  3. 3.

    http://www.volatilityfoundation.org/.

  4. 4.

    LMD consists of 107520 MD5,SHA-1, and SHA-256 hash digest for all previously identified well-known families of malware which was obtained by using https://virusshare.com/ malware repository.

  5. 5.

    https://www.virustotal.com/.

  6. 6.

    http://openmalware.org/.

References

  1. Pearce, M., Zeadally, S., Hunt, R.: Virtualization: Issues, security threats, and solutions. ACM Comput. Surv. (CSUR) 45(2), 17 (2013)

    Article  Google Scholar 

  2. Barford, P., Yegneswaran, V.: An inside look at botnets. Malware Detection. Springer, New York (2007)

    Google Scholar 

  3. Lanzi, A., Sharif, M.I., Lee, W.: K-Tracer: a system for extracting kernel malware behavior. In: NDSS (2009)

    Google Scholar 

  4. Prakash, A., et al.: Manipulating semantic values in kernel data structures: attack assessments and implications. In: 2013 43rd Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN). IEEE (2013)

    Google Scholar 

  5. Jiang, X., Wang, X., Dongyan, X.: Stealthy malware detection through vmm-based out-of-the-box semantic view reconstruction. In: Proceedings of the 14th ACM Conference on Computer and Communications Security. ACM (2007)

    Google Scholar 

  6. Garfinkel, T., Rosenblum, M.: A virtual machine introspection based architecture for intrusion detection. In: NDSS. vol. 3 (2003)

    Google Scholar 

  7. Payne, B.D., Martim, D.D.A., Lee, W.: Secure and flexible monitoring of virtual machines. In: Twenty-Third Annual Computer Security Applications Conference, ACSAC 2007. IEEE (2007)

    Google Scholar 

  8. Srinivasan, D., et al.: Process out-grafting: an efficient out-of-VM approach for fine-grained process execution monitoring. In: Proceedings of the 18th ACM Conference on Computer and Communications Security. ACM (2011)

    Google Scholar 

  9. Dolan-Gavitt, B., et al.: Virtuoso: narrowing the semantic gap in virtual machine introspection. In: 2011 IEEE Symposium on Security and Privacy. IEEE (2011)

    Google Scholar 

  10. Jain, B., et al.: SoK: introspections on trust and the semantic gap. In: 2014 IEEE Symposium on Security and Privacy. IEEE (2014)

    Google Scholar 

  11. Fu, Y., Lin, Z.: Bridging the semantic gap in virtual machine introspection via online kernel data redirection. ACM Trans. Inf. Syst. Secur. (TISSEC) 16(2), 7 (2013)

    Article  MathSciNet  Google Scholar 

  12. Saberi, A., Yangchun, F., Lin, Z.: HYBRID-BRIDGE: Efficiently bridging the semantic gap in virtual machine introspection via decoupled execution and training memoization. In: Proceedings of the 21st Annual Network and Distributed System Security Symposium (NDSS-2014) (2014)

    Google Scholar 

  13. Bauman, E., Ayoade, G., Lin, Z.: A Survey on Hypervisor-Based Monitoring: approaches, applications, and evolutions. ACM Comput. Surv. (CSUR) 48(1), 10 (2015)

    Article  Google Scholar 

  14. Goudey, H.: Threat Report: Rootkits. https://www.microsoft.com/en-in/download/details.aspx?id=34797

  15. Xuan, C., Copeland, J., Beyah, R.: Toward revealing kernel malware behavior in virtual execution environments. In: Kirda, E., Jha, S., Balzarotti, D. (eds.) RAID 2009. LNCS, vol. 5758, pp. 304–325. Springer, Heidelberg (2009). doi:10.1007/978-3-642-04342-0_16

    Chapter  Google Scholar 

  16. Jones, S.T., Arpaci-Dusseau, A.C., Arpaci-Dusseau, R.H.: VMM-based hidden process detection and identification using Lycosid. In: Proceedings of the fourth ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments. ACM (2008)

    Google Scholar 

  17. Richer, T.J., Neale, G., Osborne, G.: On the effectiveness of virtualisation assisted view comparison for rootkit detection. In: Proceedings of the 13th Australasian Information Security Conference (AISC 2015), vol. 27, p. 30 (2015)

    Google Scholar 

  18. Wu, R., et al.: System call redirection: A practical approach to meeting real-world virtual machine introspection needs. In: 2014 44th Annual IEEE/IFIP International Conference on Dependable Systems and Networks. IEEE (2014)

    Google Scholar 

  19. Westphal, F., et al.: VMI-PL: a monitoring language for virtual platforms using virtual machine introspection. Digital Invest. 11, S85–S94 (2014)

    Article  Google Scholar 

  20. Fu, Y., Zeng, J., Lin, Z.: HYPERSHELL: a practical hypervisor layer guest OS shell for automated in-VM management. In: 2014 USENIX Annual Technical Conference (USENIX ATC 2014) (2014)

    Google Scholar 

  21. Jones, S.T., Arpaci-Dusseau, A.C., Arpaci-Dusseau, R.H.: Antfarm: tracking processes in a virtual machine environment. In: USENIX Annual Technical Conference, General Track (2006)

    Google Scholar 

  22. Litty, L., Andres Lagar-Cavilla, H., Lie, D.: Hypervisor support for identifying covertly executing binaries. In: USENIX Security Symposium (2008)

    Google Scholar 

  23. Wang, Y.-M., et al.: Detecting stealth software with strider ghostbuster. 2005 International Conference on Dependable Systems and Networks (DSN 2005). IEEE (2005)

    Google Scholar 

  24. Lamps, J., Palmer, I., Sprabery, R.: WinWizard: expanding Xen with a LibVMI intrusion detection tool. In: 2014 IEEE 7th International Conference on Cloud Computing. IEEE (2014)

    Google Scholar 

  25. Vmware, 2011. Vmware, inc. vprobes programming reference. http://www.vmware.com/pdf/ws8_f4_vprobes_reference.pdf

  26. Aneja, A.: Xen hypervisor case study-designing embedded virtualized Intel architecture platforms. Intel, March 2011. https://www.intel.in/content/dam/www/public/us/en/documents/white-papers/ia-embedded-virtualized-hypervisor-paper.pdf

Download references

Author information

Authors and Affiliations

  1. Department of Information Technology, National Institute of Technology Karnataka, Surathkal, India

    M. A. Ajay Kumara & C. D. Jaidhar

Authors
  1. M. A. Ajay Kumara
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. C. D. Jaidhar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to M. A. Ajay Kumara .

Editor information

Editors and Affiliations

  1. Universities of Paris 8 and Paris 13, LAGA, Paris, France

    Claude Carlet

  2. University of Waterloo, Waterloo, Ontario, Canada

    M. Anwar Hasan

  3. CRRao AIMSCS, Hyderabad, India

    Vishal Saraswat

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing AG

About this paper

Cite this paper

Ajay Kumara, M.A., Jaidhar, C.D. (2016). VMI Based Automated Real-Time Malware Detector for Virtualized Cloud Environment. In: Carlet, C., Hasan, M., Saraswat, V. (eds) Security, Privacy, and Applied Cryptography Engineering. SPACE 2016. Lecture Notes in Computer Science(), vol 10076. Springer, Cham. https://doi.org/10.1007/978-3-319-49445-6_16

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-49445-6_16

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-49444-9

  • Online ISBN: 978-3-319-49445-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation