ABSTRACT
One of the biggest challenges of virtualization today is to efficiently share and manage network devices among different virtual machines (VMs). Software-based network virtualization solutions like device emulation and split driver device models have advantages of resource sharing and fine grained hypervisor resource control. However, software based approaches have performance and scalability impediments due to the software interventions for every I/O activity. Recent hardware advancements in network devices allow in-device partitioning and assignment of network functions to different guest operating systems. The nature of the assignment is static which gives rise to inflexibility in efficient network resource management. Additionally, fine grained hypervisor control on the network device is compromised because of the direct hardware assignment to the guest virtual machine.
In this work, we propose Vagabond, an alternate network virtualization model that supports flexible and dynamic assignment of network resources to guest VMs. The most significant advantage of the proposed model is to facilitate a framework to manage network resources efficiently when the number of hardware in-device partitions are less than the number of VMs. Additionally the advantages of software based network virtualization are kept intact to overcome the difficulties posed due to direct hardware assignment to the guest. Our experimental evaluation shows that the CPU resource overhead with Vagabond is up to 2x lower than the software approach. Compared to the direct hardware assignment, Vagabond incurs an additional overhead of 17% in the best case. We demonstrate the applicability and usage of Vagabond with two use cases: live migration of VMs connected to hardware in-device partitions and a fair-share network resource allocation scheme.
- Linux integration services version 3.4 for hyper-v. URL http://www.microsoft.com/.Google Scholar
- Intel 64 and IA-32 Architectures Developer's Manual: Vol. 3B. URL www.intel.com.Google Scholar
- Linux new api (napi). URL http://www.linuxfoundation.org/.Google Scholar
- Intel documentation, PCI-SIG SR-IOV Primer: An Introduction to SR-IOV Technology. URL http://www.intel.com/.Google Scholar
- Intel Gigabit ET/ET2/EF Multi-Port Server Adapters. URL http://www.intel.com/.Google Scholar
- Intel Virtualization Technology for Directed I/O (VT-d): Enhancing Intel platforms for efficient virtualization of I/O devices. URL www.software.intel.com/.Google Scholar
- P. Barham, B. Dragovic, K. Fraser, S. Hand, T. Harris, A. Ho, R. Neugebauer, I. Pratt, and A. Warfield. Xen and the art of virtualization. SIGOPS Operating Systems Review, 37(5): 164--177, Oct. 2003. Google ScholarDigital Library
- F. Bellard. QEMU, a Fast and Portable Dynamic Translator. In Proceedings of the USENIX Annual Technical Conference, pages 41--46, 2005. Google ScholarDigital Library
- D. Chisnall. The Definitive Guide to the Xen Hypervisor. Prentice Hall Press, first edition, 2007. ISBN 9780132349710. Google ScholarDigital Library
- C. Clark, K. Fraser, S. Hand, J. G. Hansen, E. Jul, C. Limpach, I. Pratt, and A. Warfield. Live Migration of Virtual Machines. In Proceedings of the 2nd Conference on Symposium on Networked Systems Design and Implementation (NSDI), pages 273--286, 2005. Google ScholarDigital Library
- B. Cully, G. Lefebvre, D. Meyer, M. Feeley, N. Hutchinson, and A. Warfield. Remus: High Availability via Asynchronous Virtual Machine Replication. In Proceedings of the 5th USENIX Symposium on Networked Systems Design and Implementation (NSDI), pages 161--174, 2008. Google ScholarDigital Library
- Y. Dong, D. Xu, Y. Zhang, and G. Liao. Optimizing Network I/O Virtualization with Efficient Interrupt Coalescing and Virtual Receive Side Scaling. In Conference on Cluster Computing (CLUSTER), pages 26--34, Sept 2011. Google ScholarDigital Library
- Y. Dong, Y. Chen, Z. Pan, J. Dai, and Y. Jiang. ReNIC: Architectural Extension to SR-IOV I/O Virtualization for Efficient Replication. ACM Transactions on Architecture and Code Optimization, 8(4):40:1--40:22, Jan 2012. Google ScholarDigital Library
- Y. Dong, X. Yang, J. Li, G. Liao, K. Tian, and H. Guan. High performance network virtualization with SR-IOV. Journal of Parallel and Distributed Computing, 72(11):1471--1480, 2012. Google ScholarDigital Library
- K. Fraser, S. H, R. Neugebauer, I. Pratt, A. Warfield, and M. Williamson. Safe hardware access with the Xen virtual machine monitor. In Proceedings of the 1st Workshop on Operating System and Architectural Support for the on demand IT InfraStructure (OASIS), pages 1--10, 2004.Google Scholar
- E. Keller, J. Szefer, J. Rexford, and R. B. Lee. NoHype: Virtualized Cloud Infrastructure without the Virtualization. In Proceedings of the 37th Annual International Symposium on Computer Architecture (ICSA), pages 350--361, 2010. Google ScholarDigital Library
- A. Kivity. kvm: the Linux virtual machine monitor. In The Ottawa Linux Symposium (OLS), pages 225--230, July 2007.Google Scholar
- J. Liu. Evaluating standard-based self-virtualizing devices: A performance study on 10 GbE NICs with SR-IOV support. In Parallel Distributed Processing (IPDPS), pages 1--12, April 2010.Google ScholarCross Ref
- D. E. Lowell, Y. Saito, and E. J. Samberg. Devirtualizable virtual machines enabling general, single-node, online maintenance. In Proceedings of the 11th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), pages 211--223, 2004. Google ScholarDigital Library
- K. Mansley, G. Law, D. Riddoch, G. Barzini, N. Turton, and S. Pope. Getting 10 Gb/s from Xen: Safe and Fast Device Access from Unprivileged Domains. In Proceedings of the 2007 Conference on Parallel Processing, Euro-Par, pages 224--233, 2007. Google ScholarDigital Library
- A. Menon, J. R. Santos, Y. Turner, G. J. Janakiraman, and W. Zwaenepoel. Diagnosing Performance Overheads in the Xen Virtual Machine Environment. In Proceedings of the 1st International Conference on Virtual Execution Environments (VEE), pages 13--23, 2005. Google ScholarDigital Library
- A. Menon, A. L. Cox, and W. Zwaenepoel. Optimizing Network Virtualization in Xen. In Proceedings of the USENIX Annual Technical Conference, 2006. Google ScholarDigital Library
- K. Menychtas, K. Shen, and M. L. Scott. Disengaged Scheduling for Fair, Protected Access to Fast Computational Accelerators. In Proceedings of the 19th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), pages 301--316, 2014. Google ScholarDigital Library
- R. Niranjan Mysore, G. Porter, and A. Vahdat. FasTrak: Enabling Express Lanes in Multi-tenant Data Centers. In Proceedings of the Ninth ACM Conference on Emerging Networking Experiments and Technologies, (CoNEXT), pages 139--150, 2013. Google ScholarDigital Library
- Z. Pan, Y. Dong, Y. Chen, L. Zhang, and Z. Zhang. CompSC: Live Migration with Pass-through Devices. SIGPLAN Notices, 47(7):109--120, Mar 2012. Google ScholarDigital Library
- H. Raj and K. Schwan. High Performance and Scalable I/O Virtualization via Self-virtualized Devices. In Proceedings of the 16th International Symposium on High Performance Distributed Computing, HPDC, pages 179--188, 2007. Google ScholarDigital Library
- K. K. Ram, J. R. Santos, Y. Turner, A. L. Cox, and S. Rixner. Achieving 10 Gb/s Using Safe and Transparent Network Interface Virtualization. In Proceedings of the 5th International Conference on Virtual Execution Environments (VEE), pages 61--70, 2009. Google ScholarDigital Library
- K. K. Ram, J. R. Santos, and Y. Turner. Redesigning Xen's Memory Sharing Mechanism for Safe and Efficient I/O Virtualization. In Proceedings of the 2nd Workshop on I/O Virtualization (WIOV), 2010. Google ScholarDigital Library
- S. Rixner. Network Virtualization: Breaking the Performance Barrier. ACM Queue, 6(1):37:36--37:, 2008. ISSN 1542-7730. Google ScholarDigital Library
- R. Russell. Virtio: Towards a De-facto Standard for Virtual I/O Devices. SIGOPS Operating Systems Review, 42(5):95--103, Jul 2008. Google ScholarDigital Library
- J. R. Santos, Y. Turner, G. Janakiraman, and I. Pratt. Bridging the Gap Between Software and Hardware Techniques for I/O Virtualization. In Proceedings of the USENIX Annual Technical Conference, pages 29--42, 2008. Google ScholarDigital Library
- J. Sugerman, G. Venkitachalam, and B.-H. Lim. Virtualizing I/O Devices on VMware Workstation's Hosted Virtual Machine Monitor. In Proceedings of the USENIX Annual Technical Conference, pages 1--14, 2001. Google ScholarDigital Library
- S. Tripathi, N. Droux, T. Srinivasan, and K. Belgaied. Cross-bow: From Hardware Virtualized NICs to Virtualized Networks. In Proceedings of the 1st ACM Workshop on Virtualized Infrastructure Systems and Architectures, (VISA), pages 53--62, 2009. Google ScholarDigital Library
- C. A. Waldspurger. Memory resource management in VMware ESX server. SIGOPS Operating Systems Review, 36(SI):181--194, Dec. 2002. Google ScholarDigital Library
- P. Willmann, J. Shafer, D. Carr, S. Rixner, A. Cox, W. Zwaenepoel, and W. Zwaenepoel. Concurrent Direct Network Access for Virtual Machine Monitors. In Proceedings of the 13th International Symposium on High Performance Computer Architecture, (HPCA), pages 306--317, Feb 2007. Google ScholarDigital Library
Index Terms
- Vagabond: Dynamic Network Endpoint Reconfiguration in Virtualized Environments
Recommendations
SRVM: Hypervisor Support for Live Migration with Passthrough SR-IOV Network Devices
VEE '16Single-Root I/O Virtualization (SR-IOV) is a specification that allows a single PCI Express (PCIe) device (ysical function or PF) to be used as multiple PCIe devices (virtual functions or VF). In a virtualization system, each VF can be directly assigned ...
SRVM: Hypervisor Support for Live Migration with Passthrough SR-IOV Network Devices
VEE '16: Proceedings of the12th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution EnvironmentsSingle-Root I/O Virtualization (SR-IOV) is a specification that allows a single PCI Express (PCIe) device (ysical function or PF) to be used as multiple PCIe devices (virtual functions or VF). In a virtualization system, each VF can be directly assigned ...
Comments