ABSTRACT
Virtualization is increasingly being used to address server management and administration issues like flexible resource allocation, service isolation and workload migration. In a virtualized environment, the virtual machine monitor (VMM) is the primary resource manager and is an attractive target for implementing system features like scheduling, caching, and monitoring. However, the lackof runtime information within the VMM about guest operating systems, sometimes called the semantic gap, is a significant obstacle to efficiently implementing some kinds of services.In this paper we explore techniques that can be used by a VMM to passively infer useful information about a guest operating system's unified buffer cache and virtual memory system. We have created a prototype implementation of these techniques inside the Xen VMM called Geiger and show that it can accurately infer when pages are inserted into and evicted from a system's buffer cache. We explore several nuances involved in passively implementing eviction detection that have not previously been addressed, such as the importance of tracking disk block liveness, the effect of file system journaling, and the importance of accounting for the unified caches found in modern operating systems.Using case studies we show that the information provided by Geiger enables a VMM to implement useful VMM-level services. We implement a novel working set size estimator which allows the VMM to make more informed memory allocation decisions. We also show that a VMM can be used to drastically improve the hit rate in remote storage caches by using eviction-based cache placement without modifying the application or operating system storage interface. Both case studies hint at a future where inference techniques enable a broad new class of VMM-level functionality.
- A.C. Arpaci-Dusseau and R.H. Arpaci-Dusseau. Information and Control in Gray-Box Systems. In Proceedings of the 18th ACM Symposium on Operating Systems Principles (SOSP '01), pages 43--56, Banff, Canada, October 2001. Google ScholarDigital Library
- L.N. Bairavasundaram, M. Sivathanu, A.C. Arpaci-Dusseau, and R.H. Arpaci-Dusseau. X-RAY: A Non-Invasive Exclusive Caching Mechanism for RAIDs. In Proceedings of the 31st Annual International Symposium on Computer Architecture (ISCA '04), Munich, Germany, June 2004. Google ScholarDigital Library
- S. Ballmer. Keynote address. Microsoft Management Summit, April 2005.Google Scholar
- S. Best. JFS Overview. www.ibm.com/developerworks/library/ljfs. html, 2000.Google Scholar
- T.C. Bressoud and F.B. Schneider. Hypervisor-based fault tolerance. In SOSP '95: Proceedings of the fifteenth ACM symposium on Operating systems principles, pages 1--11. ACM Press, 1995. Google ScholarDigital Library
- E. Bugnion, S. Devine, and M. Rosenblum. Disco: Running commodity operating systems on scalable multiprocessors. In Proceedings of the 16th ACM Symposium on Operating Systems Principles (SOSP '97), pages 143--156, Saint-Malo, France, October 1997. Google ScholarDigital Library
- P.M. Chen and B.D. Noble. When Virtual Is Better Than Real. In HOTOS '01: Proceedings of the Eighth Workshop on Hot Topics in Operating Systems, page 133. IEEE Computer Society, 2001. Google ScholarDigital Library
- Z. Chen, Y. Ahang, Y. Zhou, H. Scott, and B. Schiefer. Empirical Evaluation of Multi-level Buffer Cache Collaboration for Storage Systems. In Proceedings of the 2005 ACM SIGMETRICS Conference on Measurement and Modeling of Computer Systems (SIGMETRICS '05), Banff, Canada, June 2005. Google ScholarDigital Library
- Z. Chen, Y. Zhou, and K. Li. Eviction-based Placement for Storage Caches. In Proceedings of the USENIX Annual Technical Conference (USENIX '03), pages 269--282, San Antonio, Texas, June 2003.Google Scholar
- C. Clark, K. Fraser, S. Hand, J.G. Hansen, E. Jul, C. Limpach, I. Pratt, and A. Warfield. Live Migration of Virtual Machines. In Proceedings of the 2nd Symposium on Networked Systems Design and Implementation (NSDI '05), Boston, Massachusetts, May 2005. Google ScholarDigital Library
- P.J. Denning. The Working Set Model for Program Behavior. Communications of the ACM, 11(5): 323--333, May 1968. Google ScholarDigital Library
- P.J. Denning. Working Sets: Past and Present. IEEE Transactions on Software Engineering, SE-6(1):64--84, January 1980.Google ScholarDigital Library
- B. Dragovic, K. Fraser, S. Hand, T. Harris, A. Ho, I. Pratt, A.Warfield, P. Barham, and R. Neugebauer. Xen and the Art of Virtualization. In Proceedings of the 19th ACM Symposium on Operating Systems Principles (SOSP '03), Bolton Landing (Lake George), New York, October 2003. Google ScholarDigital Library
- R. Figueriredo, P. Dinda, and J. Fortes. A Case for Grid Computing on Virtual Machines. In Proceedings of the International Conference on Distributed Computing Systems (ICDCS), May 2003. Google ScholarDigital Library
- K. Fraser, S. Hand, R. Neugebauer, I. Pratt, A. Warfield, and M. Williamson. Safe Hardware Access with the Xen Virtual Machine Monitor. In OASIS ASPLOS 2004 workshop, 2004.Google Scholar
- T. Garfinkel, B. Pfaff, J. Chow, M. Rosenblum, and D. Boneh. Terra: A Virtual Machine-Based Platform for Trusted Computing. In Proceedings of the 19th ACM Symposium on Operating Systems Principles (SOSP '03), Bolton Landing (Lake George), New York, October 2003. Google ScholarDigital Library
- R. Goldberg. Survey of Virtual Machine Research. IEEE Computer, 7(6):34--45, 1974.Google ScholarDigital Library
- P. Gum. System/370 Extended Architecture: Facilities for Virtual Machines. IBM Journal of Research and Development, 27(6):530--544, November 1983.Google ScholarDigital Library
- ImageMagick Studio LLC. ImageMagick image processing software. http://www.imagemagick.org.Google Scholar
- Intel Corporation. Intel Virtualization Technology Specification. ftp://download.intel.com/technology/computing/vptech/C97063.pdf,2005.Google Scholar
- T. Johnson and D. Shasha. 2Q: A Low-Overhead High Performance Buffer Management Replacement Algorithm. In Proceedings of the 20th International Conference on Very Large Databases (VLDB 20), pages 439--450, Santiago, Chile, September 1994. Google ScholarDigital Library
- S.T. Jones, A.C. Arpaci-Dusseau, and R.H. Arpaci-Dusseau. Antfarm: Tracking Processes in a Virtual Machine Environment. In Proceedings of the USENIX Annual Technical Conference (USENIX '06), Boston, Massachusetts, June 2006. Google ScholarDigital Library
- N.P. Jouppi. Improving Direct-Mapped Cache Performance by the Addition of a Small Fully-Associative Cache and Prefetch Buffers. In Proceedings of the 17th Annual International Symposium on Computer Architecture (ISCA '90), pages 364--373, Seattle, Washington, May 1992. Google ScholarDigital Library
- S.T. King and P.M. Chen. Backtracking Intrusions. In Proceedings of the 18th ACM Symposium on Operating Systems Principles (SOSP '01), Banff, Canada, October 2001.Google Scholar
- D. Muntz and P. Honeyman. Multi-Level Caching in Distributed File Systems - or - Your Cache Ain't Nuthin' But Trash. Proceedings of the USENIX Winter Conference, pages 305--313, January 1992.Google Scholar
- Open Source Development Labs. OSDL Database Test Suite. http://www.osdl.org/lab activities/kernel testing/osdl database test suite.Google Scholar
- R. H. Patterson, G. A. Gibson, E. Ginting, D. Stodolsky, and J. Zelenka. Informed Prefetching and Caching. In Proceedings of the 15th ACM Symposium on Operating Systems Principles (SOSP '95), pages 79--95, Copper Mountain Resort, Colorado, December 1995. Google ScholarDigital Library
- H. Reiser. ReiserFS. www.namesys.com, 2004.Google Scholar
- C.P. Sapuntzakis, R. Chandra, B. Pfaff, J. Chow, M.S. Lam, and M. Rosenblum. Optimizing the Migration of Virtual Computers. In Proceedings of the 5th Symposium on Operating Systems Design and Implementation (OSDI '02), pages 377--390, Boston, Massachusetts, December 2002. Google ScholarDigital Library
- M. Sivathanu, L.N. Bairavasundaram, A.C. Arpaci-Dusseau, and R.H. Arpaci-Dusseau. Life or Death at Block Level. In Proceedings of the 6th Symposium on Operating Systems Design and Implementation (OSDI '04), pages 379--394, San Francisco, California, December 2004. Google ScholarDigital Library
- M. Sivathanu, V. Prabhakaran, A C. Arpaci-Dusseau, and R.H. Arpaci-Dusseau. Improving Storage System Availability with DGRAID. In Proceedings of the Third USENIX Symposium on File and Storage Technologies (FAST '04), San Francisco, CA, March 2004. Google ScholarDigital Library
- Storage Performance Council. SPC web search engine storage traces. http://traces.cs.umass.edu/storage.Google Scholar
- J. Sugerman, G. Venkitachalam, and B.-H. Lim. Virtualizing I/O Devices on VMware Workstation's Hosted Virtual Machine Monitor. In Proceedings of the USENIX Annual Technical Conference (USENIX '01), Boston, Massachusetts, June 2001. Google ScholarDigital Library
- A. Sweeney, D. Doucette, W. Hu, C. Anderson, M. Nishimoto, and G. Peck. Scalability in the XFS File System. In Proceedings of the USENIX Annual Technical Conference (USENIX '96), San Diego, California, January 1996. Google ScholarDigital Library
- A. Tridgell. Dbench filesystem benchmark. http://samba.org/ftp/tridge/dbench.Google Scholar
- T. Ts'o and S. Tweedie. Future Directions for the Ext2/3 Filesystem. In Proceedings of the USENIX Annual Technical Conference (FREENIX Track), Monterey, California, June 2002. Google ScholarDigital Library
- S.C. Tweedie. EXT3, Journaling File System. olstrans.sourceforge.net/release/OLS2000-ext3/OLS2000-ext3.html, July 2000.Google Scholar
- C.A. Waldspurger. Memory Resource Management in VMware ESX Server. In Proceedings of the 5th Symposium on Operating Systems Design and Implementation (OSDI '02), Boston, Massachusetts, December 2002. Google ScholarDigital Library
- A. Whitaker, M. Shaw, and S.D. Gribble. Scale and Performance in the Denali Isolation Kernel. In Proceedings of the 5th Symposium on Operating Systems Design and Implementation (OSDI '02), Boston, Massachusetts, December 2002. Google ScholarDigital Library
- T.M. Wong and J. Wilkes. My Cache or Yours? Making Storage More Exclusive. In Proceedings of the USENIX Annual Technical Conference (USENIX '02), Monterey, California, June 2002. Google ScholarDigital Library
- M. Zhao, J. Zhang, and R. Figueriredo. Distributed File System Support for Virtual Machines in Grid Computing. In Proceedings of High Performance Distributed Computing (HPDC), July 2004. Google ScholarDigital Library
- Y. Zhou, J.F. Philbin, and K. Li. The Multi-Queue Replacement Algorithm for Second Level Buffer Caches. In Proceedings of the USENIX Annual Technical Conference (USENIX '01), pages 91--104, Boston, Massachusetts, June 2001. Google ScholarDigital Library
Index Terms
- Geiger: monitoring the buffer cache in a virtual machine environment
Recommendations
Geiger: monitoring the buffer cache in a virtual machine environment
Proceedings of the 2006 ASPLOS ConferenceVirtualization is increasingly being used to address server management and administration issues like flexible resource allocation, service isolation and workload migration. In a virtualized environment, the virtual machine monitor (VMM) is the primary ...
Geiger: monitoring the buffer cache in a virtual machine environment
Proceedings of the 2006 ASPLOS ConferenceVirtualization is increasingly being used to address server management and administration issues like flexible resource allocation, service isolation and workload migration. In a virtualized environment, the virtual machine monitor (VMM) is the primary ...
Geiger: monitoring the buffer cache in a virtual machine environment
Proceedings of the 2006 ASPLOS ConferenceVirtualization is increasingly being used to address server management and administration issues like flexible resource allocation, service isolation and workload migration. In a virtualized environment, the virtual machine monitor (VMM) is the primary ...
Comments