Abstract
Virtual machine introspection (VMI) describes the method of monitoring and analyzing the state of a virtual machine from the hypervisor level. This lends itself well to security applications, though the hardware virtualization support from Intel and AMD was not designed with VMI in mind. This results in many challenges for developers of hardware-supported VMI systems. This paper describes the design and implementation of our prototype framework, Nitro, for system call tracing and monitoring. Since Nitro is a purely VMI-based system, it remains isolated from attacks originating within the guest operating system and is not directly visible from within the guest. Nitro is extremely flexible as it supports all three system call mechanisms provided by the Intel x86 architecture and has been proven to work in Windows, Linux, 32-bit, and 64-bit environments. The high performance of our system allows for real-time capturing and dissemination of data without hindering usability. This is supported by extensive testing with various guest operating systems. In addition, Nitro is resistant to circumvention attempts due to a construction called hardware rooting. Finally, Nitro surpasses similar systems in both performance and functionality.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bahram, S., Jiang, X., Wang, Z., Grace, M., Li, J., Xu, D.: DKSM: Subverting virtual machine introspection for fun and profit. In: Proc. of 29th IEEE Int. Symp. on Reliable Distributed Systems (SRDS 2010), New Delhi, India (October 2010)
Bayer, U., Kruegel, C., Kirda, E.: TTAnalyze: A tool for analyzing malware. In: 15th European Inst. for Computer Antivirus Research (EICAR 2006) Conf., Hamburg, Germany (April 2006)
Chen, P.M., Noble, B.D.: When virtual is better than real. In: Proc. of the 8th Workshop on Hot Topics in Op. Sys., p. 133. IEEE, Washington, DC, USA (2001)
Dinaburg, A., Royal, P., Sharif, M., Lee, W.: Ether: malware analysis via hardware virtualization extensions. In: Proc. of the 15th ACM Conf. on Computer and Communications Security, pp. 51–62. ACM, New York (2008)
Garfinkel, T., Rosenblum, M.: A virtual machine introspection based architecture for intrusion detection. In: Proc. of Network and Distributed Systems Security Symp., pp. 191–206 (2003)
Hofmeyr, S.A., Forrest, S., Somayaji, A.: Intrusion detection using sequences of system calls. Journal of Computer Security 6(3), 151–180 (1998)
Holz, T., Freiling, F., Willems, C.: Toward automated dynamic malware analysis using CWSandbox. IEEE Security & Privacy 5(2), 32–39 (2007)
Jones, S.T., Arpaci-Dusseau, A.C., Arpaci-Dusseau, R.H.: VMM-based hidden process detection and identification using Lycosid. In: Proc. of the 4th Int. conf. on Virtual Execution Environments, pp. 91–100. ACM, New York (2008)
Kosoresow, A.P., Hofmeyr, S.A.: Intrusion detection via system call traces. IEEE Softw. 14(5), 35–42 (1997)
Martignoni, L., Fattori, A., Paleari, R., Cavallaro, L.: Live and trustworthy forensic analysis of commodity production systems. In: Jha, S., Sommer, R., Kreibich, C. (eds.) RAID 2010. LNCS, vol. 6307, pp. 297–316. Springer, Heidelberg (2010)
Payne, B.D., Carbone, M., Sharif, M., Lee, W.: Lares: An architecture for secure active monitoring using virtualization. In: Proc. of 2008 IEEE Symp. on Security and Privacy, pp. 233–247. IEEE, Washington, DC, USA (2008)
Pfoh, J., Schneider, C., Eckert, C.: A formal model for virtual machine introspection. In: Proc. of the 2nd ACM Workshop on Virtual Machine Security. ACM, New York (2009)
Pfoh, J., Schneider, C., Eckert, C.: Exploiting the x86 architecture to derive virtual machine state information. In: Proc. of the 4th Int. Conf. on Emerging Security Information, Systems and Technologies. IEEE, Venice (2010)
Rieck, K., Trinius, P., Willems, C., Holz, T.: Automatic analysis of malware behavior using machine learning. Tech. Rep. 18-2009, Berlin Inst. of Technology (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Pfoh, J., Schneider, C., Eckert, C. (2011). Nitro: Hardware-Based System Call Tracing for Virtual Machines. In: Iwata, T., Nishigaki, M. (eds) Advances in Information and Computer Security. IWSEC 2011. Lecture Notes in Computer Science, vol 7038. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-25141-2_7
Download citation
DOI: https://doi.org/10.1007/978-3-642-25141-2_7
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-25140-5
Online ISBN: 978-3-642-25141-2
eBook Packages: Computer ScienceComputer Science (R0)