Abstract
Advances in virtualization technology have focused mainly on strengthening the isolation barrier between virtual machines (VMs) that are co-resident within a single physical machine. At the same time, a large category of communication intensive distributed applications and software components exist, such as web services, high performance grid applications, transaction processing, and graphics rendering, that often wish to communicate across this isolation barrier with other endpoints on co-resident VMs. State of the art inter-VM communication mechanisms do not adequately address the requirements of such applications. TCP/UDP based network communication tends to perform poorly when used between co-resident VMs, but has the advantage of being transparent to user applications. Other solutions exploit inter-domain shared memory mechanisms to improve communication latency and bandwidth, but require applications or user libraries to be rewritten against customized APIs—something not practical for a large majority of distributed applications. In this paper, we present the design and implementation of a fully transparent and high performance inter-VM network loopback channel, called XenLoop, in the Xen virtual machine environment. XenLoop does not sacrifice user-level transparency and yet achieves high communication performance between co-resident guest VMs. XenLoop intercepts outgoing network packets beneath the network layer and shepherds the packets destined to co-resident VMs through a high-speed inter-VM shared memory channel that bypasses the virtualized network interface. Guest VMs using XenLoop can migrate transparently across machines without disrupting ongoing network communications, and seamlessly switch between the standard network path and the XenLoop channel. In our evaluation using a number of unmodified benchmarks, we observe that XenLoop can reduce the inter-VM round trip latency by up to a factor of 5 and increase bandwidth by a up to a factor of 6.
Similar content being viewed by others
References
Barham, P., Dragovic, B., Fraser, K., Hand, S. et al.: Xen and the art of virtualization. In: SOSP, Oct. 2003
Chisnall, D.: The Definitive Guide to the Xen Hypervisor, 2nd edn. Prentice-Hall, Englewood Cliffs (2007)
Hensbergen, E.V., Goss, K.: Prose i/o. In: First International Conference on Plan 9, Madrid, Spain, 2006
Huang, W., Koop, M., Gao, Q., Panda, D.K.: Virtual machine aware communication libraries for high performance computing. In: Proc. of SuperComputing (SC’07), Reno, NV, Nov. 2007
Kim, K., Kim, C., Jung, S.-I., Shin, H., Kim, J.-S.: Inter-domain socket communications supporting high performance and full binary compatibility on Xen. In: Proc. of Virtual Execution Environments, 2008
Kourai, K., Chiba, S.: HyperSpector: Virtual distributed monitoring environments for secure intrusion detection. In: Proc. of Virtual Execution Environments, 2005
McVoy, L., Staelin, C.: lmbench: portable tools for performance analysis. In: Proc. of USENIX Annual Technical Symposium, 1996
Menon, A., Cox, A.L., Zwaenepoel, W.: Optimizing network virtualization in Xen. In: Proc. of USENIX Annual Technical Conference, 2006
Menon, A., Santos, J.R., Turner, Y., Janakiraman, G.J., Zwaenepoel, W.: Diagnosing performance overheads in the Xen virtual machine environment. In: Proc. of Virtual Execution Environments, 2005
Muir, S., Peterson, L., Fiuczynski, M., Cappos, J., Hartman, J.: Proper: privileged operations in a virtualised system environment. In: USENIX Annual Technical Conference, 2005
Netfilter. http://www.netfilter.org/
Netperf. http://www.netperf.org/
Snell, Q.O., Mikler, A.R., Gustafson, J.L.: NetPIPE: a network protocol independent performance evaluator. In: Proc. of IASTED International Conference on Intelligent Information Management and Systems, 1996
Turner, D., Chen, X.: Protocol-dependent message-passing performance on Linux clusters. In: Proc. of Cluster Computing, 2002
Turner, D., Oline, A., Chen, X., Benjegerdes, T.: Integrating new capabilities into NetPIPE. In: Proc. of 10th European PVM/MPI conference, Venice, Italy, 2003
XenLoop Source Code. http://osnet.cs.binghamton.edu/projects/xenloop.html
Zhang, X., McIntosh, S., Rohatgi, P., Griffin, J.L.: Xensocket: a high-throughput interdomain transport for virtual machines. In: Proc. of Middleware, 2007
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Wang, J., Wright, KL. & Gopalan, K. XenLoop: a transparent high performance inter-VM network loopback. Cluster Comput 12, 141–152 (2009). https://doi.org/10.1007/s10586-009-0079-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10586-009-0079-x