Asim Kadav
Abstract
Virtual machine migration greatly aids management by allowing flexible provisioning of resources and decommissioning of hardware for maintenance. However, efforts to improve network performance by granting virtual machines direct access to hardware currently prevent migration. This occurs because (1) theVMMcannot migrate the state of the device, and (2) the source and destination machines may have different network devices, requiring different drivers to run in the migrated virtual machine.
In this paper, we describe a lightweight software mechanism for migrating virtual machines with direct hardware access. We base our solution on shadow drivers, an agent in the guest OS kernel that efficiently captures and restores the state of a device driver. On the source machine, the shadow driver monitors the state of the driver and device. After migration, the shadow driver uses this information to configure a driver for the corresponding device on the destination machine. We implement shadow driver migration for Linux network drivers running on the Xen hypervisor. Shadow driver migration requires a migration downtime similar to the driver initialization time, short enough to avoid disrupting active TCP connections. We find that the performance overhead, compared to direct hardware access, is negligible and is much lower than using a virtual NIC.
- Darren Abramson, Je Jackson, Sridhar Muthrasanallur, Gil Neiger, Greg Regnier, Rajesh Sankran, Ionnis Schoinas, Rich Uhlig, Balaji Vembu, and JohnWeigert. Intel virtualization technology for directed I/O. Intel Technology Journal, 10(3):179--191, August 2006.Google Scholar
Cross Ref
- Advanced Micro Devices, Inc. AMD I/O virtualization technology (IOMMU) specification. www.amd.com/us-en/assets/content type/white papers and tech docs/34434.pdf, February 2007. Publication # 34434.Google Scholar
- Michael Armbrust, Armando Fox, Rean Griffth, Anthony D. Joseph, Randy H. Katz, Andrew Konwinski, Gunho Lee, David A. Patterson, Ariel Rabkin, Ion Stoica, and Matei Zaharia. Above the clouds: A berkeley view of cloud computing. Technical Report UCB/EECS-2009-28, EECS Department, University of California, Berkeley, Feb 2009.Google Scholar
- Christopher Clark, Keir Fraser, Steven Hand, Jacob Gorm Hansen, Eric Jul, Christian Limpach, Ian Pratt, and Andrew Warfield. Live migration of virtual machines. In Proc. of the 2nd Symp. on Networked Systems Design and Implementation, May 2005. Google ScholarDigital Library
- Information Networks Division. Netperf: A network performance benchmark. http://www.netperf.org.Google Scholar
- Keir Fraser, Steven Hand, Rolf Neugebauer, Ian Pratt, Andrew Warfield, and Mark Williamson. Safe hardware access with the Xen virtual machine monitor. In Proceedings of the Workshop on Operating System and Architectural Support for the On-Demand IT Infrastructure, October 2004.Google Scholar
- Wei Huang, Jiuxing Liu, Bulent Abali, and Dhabaleswar K. Panda. A case for high performance computing with virtual machines. In ICS '06: Proceedings of the 20th annual international conference on Supercomputing, pages 125--134, New York, NY, USA, 2006. ACM. Google ScholarDigital Library
- M. Kozuch and M. Satyanarayanan. Internet suspend/resume. In Proceedings of the 4th IEEE Workshop on Mobile Computing Systems and Applications, Callicoon, NY, June 2002. Google ScholarDigital Library
- Pengcheng Liu, Ziye Yang, Xiang Song, Yixun Zhou, Haibo Chen, and Binyu Zang. Heterogeneous live migration of virtual machines. In International Workshop on Virtualization Technology (IWVT'08), 2008.Google Scholar
- Mallik Mahalingam. I/o architectures for virtualization. In VMWORLD, 2006.Google Scholar
- Aravind Menonand, Alan L. Cox, and Willy Zwaenepoel. Optimizing network virtualization in Xen. In Proceedings of the 2006 USENIX ATC, pages 15--28, May 2006. Google ScholarDigital Library
- Mike Neil. Hypervisor, virtualization stack, and device virtualization architectures. Technical Report Win-Hec 2006 Presentation VIR047, Microsoft Corporation, May 2006.Google Scholar
- Michael Nelson, Beng-Hong Lim, and Greg Hutchins. Fast transparent migration for virtual machines. In Proceedings of the annual conference on USENIX Annual Technical Conference, 2005. Google ScholarDigital Library
- Qumranet Inc. KVM: Kernel-based virtualization driver. www.qumranet.com/wp/kvm wp.pdf, 2006.Google Scholar
- Quramnet Inc. KVM: Migrating a VM. http://kvm.qumranet.com/kvmwiki/Migration, 2008.Google Scholar
- Kaushik Kumar Ram, Jose Renato Santos, Yoshio Turner, Alan L. Cox, and Scott Rixner. Achieving 10 gb/s using safe and transparent network interface virtualization. In VEE '09: Proceedings of the 2009 ACM SIGPLAN/SIGOPS international conference on Virtual execution environments, pages 61--70,Washington, DC, USA, 2009. Google ScholarDigital Library
- Jose Renato Santos, Yoshio Turner, G. John Janakiraman, and Ian Pratt. Bridging the gap between software and hardware techniques for I/O virtualization. In Proceedings of the annual conference on USENIX Annual Technical Conference, 2008. Google ScholarDigital Library
- Ashley Saulsbury. Your OS on the T1 hypervisor. www.opensparc.net/publications/presentations/your-os-on-the-t1-hypervisor.html, March 2006.Google Scholar
- Jeremy Sugerman, Ganesh Venkitachalam, and Beng-Hong Lim. Virtualizing I/O devices on VMware workstation's hosted virtual machine monitor. In Proceedings of the 2001 USENIX ATC, Boston, Massachusetts, June 2001. Google ScholarDigital Library
- Michael Swift, Muthukaruppan Annamalau, Brian N. Bershad, and Henry M. Levy. Recovering device drivers. ACM Transactions on Computer Systems, 24(4), November 2006. Google ScholarDigital Library
- Michael M. Swift, Damien Martin-Guillerez, Muthukaruppan Annamalai, Brian N. Bershad, and Henry M. Levy. Live update for device drivers. Technical Report CS-TR-2008-1634, March 2008.Google Scholar
- Sean Varley and Howie Xu. I/O pass-through methologies for mainstream virtualization usage. http://intel.wingateweb.com/US08/published/sessions/IOSS003/SF08 IOSS003 101r.pdf, August 2008.Google Scholar
- VMware Inc. I/O compatibility guide for ESX server 3.x. http://www.vmware.com/pdf/vi3 io guide.pdf, June 2007.Google Scholar
- Andrew Whitaker, Marianne Shaw, and Steven D. Gribble. Denali: Lightweight virtual machines for distributed and networked applications. In Proceedings of the 5th USENIX OSDI, pages 195--209, Boston, MA, December 2002.Google Scholar
- Xen Wiki. Xen virtual hosting server providers. http://wiki.xensource.com/xenwiki/VirtualPrivateServerProviders.Google Scholar
- Paul Willmann, Scott Rixner, and Alan L. Cox. Protection strategies for direct access to virtualized I/O devices. In Proceedings of the annual conference on USENIX Annual Technical Conference, 2008. Google ScholarDigital Library
- WireShark: A network protocol analyzer. http://www.wireshark.org.Google Scholar
- Edwin Zhai, Gregory D. Cummings, and Yaozu Dong. Live migration with pass-through device for Linux VM. In Proceedings of the Ottawa Linux Symposium, pages 261--267, 2008.Google Scholar
Index Terms
- Live migration of direct-access devices
Recommendations
Live migration of direct-access devices
WIOV'08: Proceedings of the First conference on I/O virtualizationVirtual machine migration greatly aids management by allowing flexible provisioning of resources and decommissioning of hardware for maintenance. However, efforts to improve network performance by granting virtual machines direct access to hardware ...
A Hypervisor Approach to Enable Live Migration with Passthrough SR-IOV Network Devices
Special TopicsSingle-Root I/O Virtualization (SR-IOV) is a specification that allows a single PCI Express (PCIe) device (physical function or PF) to be used as multiple PCIe devices (virtual functions or VF). In a virtualization system, each VF can be directly ...
Traffic-sensitive live migration of virtual machines
CCGRID '15: Proceedings of the 15th IEEE/ACM International Symposium on Cluster, Cloud, and Grid ComputingIn this paper we address the problem of network contention between the migration traffic and the Virtual Machine (VM) application traffic for the live migration of co-located Virtual Machines. When VMs are migrated with pre-copy, they run at the source ...
Comments