Abstract
Nowadays, virtual machines are becoming widely used and their range of applications include a large number of scientific fields. From HPC to IaaS, communication between co-located VMs is a critical factor of efficiency. In our paper, we examine communication methods between VMs located in the same physical node, optimizing communication cost without sacrificing upper-layer API compatibility. We present YASMIN (Yet Another Shared Memory Implementation for Intra-Node), a generic socket-compliant framework for intra-node communication in the Xen hypervisor. We build on the concept of Vchan, a Xen library for intra-node communication between different VMs and we use Xen granting and signaling mechanisms to provide an efficient communication framework. The key of our design is the transport layer which runs underneath the AF_VSOCK protocol family, implemented as a dynamically inserted module. We are able to achieve 4.4x higher bandwidth rate and 65% lower latency without the need of application binary recompilation.
This is a preview of subscription content, log in via an institution.
Notes
- 1.
grant-entry index and event-channel port number.
- 2.
This value refers to a 1 Byte message.
References
Huang, W., Liu, J., Abali, B., Panda, D.K.: A case for high performance computing with virtual machines. In: ICS 2006, Cairns, Queensland, Australia, 28–30 June (2006)
Reuther, A., Michaleas, P., Prout, A., Kepner, J.: HPC-VMs: virtual machines in high performance computing systems. In: IEEE High Performance Extreme Computing (HPEC) Conference, Waltham, MA, 10–12 September (2012)
Tikotekar, A., Ong, H., Alam, S., Vallee, G., Naughton, T., Engelmann, C., Scott, S.L.: Performance comparison of two virtual machine scenarios using an HPC application. a case study using molecular dynamics simulations. In: HPCVirt 2009 Proceedings of the 3rd ACM Workshop on System-level Virtualization for High Performance Computing, Nuremburg, Germany, 31 March (2009)
Strazdins, P., Alexander, R., Barr, D.: Performance enhancement of SMP clusters with multiple network interfaces using virtualization. In: Min, G., Di Martino, B., Yang, L.T., Guo, M., Rünger, G. (eds.) ISPA 2006. LNCS, vol. 4331, pp. 452–463. Springer, Heidelberg (2006). doi:10.1007/11942634_47
Shehabi, A., Smith, S.J., Horner, N., Azevedo, I., Brown, R., Koomey, J., Masanet, E., Sartor, D., Herrlin, M., Lintner, W.: United States Data Center Energy Usage Report. Lawrence Berkeley National Laboratory, Berkeley, California. LBNL-1005775 (2016)
Yang, C.-T., Tseng, C.-H., Chou, K.-Y., Tsaur, S.-C., Hsu, C.-H., Chen, S.-C.: A xen-based paravirtualization system toward efficient high performance computing environments. In: Hsu, C.-H., Malyshkin, V. (eds.) MTPP 2010. LNCS, vol. 6083, pp. 126–135. Springer, Heidelberg (2010). doi:10.1007/978-3-642-14822-4_14
Network Functions Virtualisation, An Introduction, Benefits, Enablers, Challenges & Call for Action, https://portal.etsi.org/nfv/nfv_white_paper.pdf
DiGiglio, J., Ricci, D.: High Performance, Open Standard Virtualization with NFV and SDN, A Joint Hardware and Software Platform for Next-Generation NFV and SDN Deployments
Network Function Virtualisation (FV); Use Cases, ETSI GS NFV 001, http://www.etsi.org/deliver/etsi_gs/NFV/001_099/001/01.01.01_60/gs_NFV001v010101p.pdf
Apache Hadoop, https://wiki.apache.org/hadoop
List of institutions that are using Apache Hadoop for educational or production uses, https://wiki.apache.org/hadoop/PoweredBy, https://wiki.apache.org/hadoop/Distributions
Kang, H., Chen, Y., Wong, J., Sion, R., Jason, W.: Enhancement of Xen’s scheduler for MapReduce workloads. In: HPDC 2011, San Jose, California, USA, 8–11 June (2011)
Park, J., Lee, D., Kim, B., Huh, J., Maeng, S.: Locality-aware dynamic VM reconfiguration on MapReduce clouds. In: HPDC 2012, Delft, The Netherlands, 18–22 June (2012)
Ibrahim, S., Jin, H., Lu, L., He, B., Wu, S.: Adaptive Disk I/O Scheduling for MapReduce in Virtualized Environment. In: 2011 International Conference on Parallel Processing (2011)
Xen Project Hypervisor, https://wiki.xen.org/wiki/Xen
Vchan Xen Library, https://github.com/mirage/xen/tree/master/tools/libvchan
VMware vSockets, https://pubs.vmware.com/vsphere-65/index.jsp#com.vmware.vmci.pg.doc/vsockAbout.3.2.html#1023121
Virtual Machine Communication Interface, https://pubs.vmware.com/vmci-sdk/
Virtio - IO Virtualization in KVM, http://www.linux-kvm.org/page/Virtio
NetPIPE - Network Protocol Independent Performance Evaluator, https://linux.die.net/man/1/netpipe
iPerf Benchmark, https://iperf.fr/
netperf - a network performance benchmark, https://linux.die.net/man/1/netperf
McCalpin, J.D.: Memory bandwidth and machine balance in current high performance computers. In: IEEE Computer Society Technical Committee on Computer Architecture (TCCA) Newsletter, December 1995
Diakhaté, F., Pérache, M., Namyst, R., Jourdren, H.: Efficient shared memory message passing for inter-VM communications (2008)
Kernel Virtual Machine, https://www.linux-kvm.org/page/Main
QEMU, http://www.qemu.org/
Huang, W., Koop, M.J., Gao, Q., Panda, D.K.: Virtual machine aware communication libraries for high performance computing. In: SC 2007: Proceedings of the 2007 ACM/IEEE Conference on Supercomputing, NY, USA (2007)
Zhang, X., McIntosh, S., Rohatgi, P., Griffin, J.L.: XenSocket: a high-throughput interdomain transport for virtual machines. In: Cerqueira, R., Campbell, R.H. (eds.) Middleware 2007. LNCS, vol. 4834, pp. 184–203. Springer, Heidelberg (2007). doi:10.1007/978-3-540-76778-7_10
PV Calls: A new paravirtualized protocol for POSIX syscalls
Wang, J., Wright, K.-L., Gopalan, K.: XenLoop: a transparent high performance inter-vm network loopback. In: HPDC 2008: Proceedings of the 17th International Symposium on High Performance Distributed Computing (2008)
Nanos, A., Gerangelos, S., Alifieraki, I., Koziris, N.: V4VSockets: low-overhead intra-node communication in Xen. In: CloudDP 2015, Bordeaux, France, 21–24 April (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Rozis, M., Gerangelos, S., Koziris, N. (2017). YASMIN: Efficient Intra-node Communication Using Generic Sockets. In: Kunkel, J., Yokota, R., Taufer, M., Shalf, J. (eds) High Performance Computing. ISC High Performance 2017. Lecture Notes in Computer Science(), vol 10524. Springer, Cham. https://doi.org/10.1007/978-3-319-67630-2_43
Download citation
DOI: https://doi.org/10.1007/978-3-319-67630-2_43
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-67629-6
Online ISBN: 978-3-319-67630-2
eBook Packages: Computer ScienceComputer Science (R0)