Abstract
Inter-VM traffics are witnessed as one of the dominating costs for massive parallel computation jobs in virtualized clouds nowadays. One way to improve the efficiency is to exploit shared memory facilities provided by the hypervisor or the host domain to bypass traditional network path for co-located VMs. Thus it is important to be capable of determining whether two VMs are co-located on the same physical node or not. Existing approaches employ either static or dynamic method to maintain co-located VM membership. Since static methods are not adaptive to support runtime detection of VM existence changes, polling based dynamic methods are adopted, with which the response time to membership change is hold up by pre-configured polling cycle. Moreover, the overhead for the membership change and the scalability over the number of co-located VMs are not considered. In this paper, we propose CoKeeper, a dynamic event driven co-located VM detection and membership update approach for residency aware inter-VM communication. CoKeeper responses faster than polling based methods since the membership updates are immediately visible after the events of VM existence changes. Experimental results show that the response time of CoKeeper is more than 10 times lower than that of polling based method and more stable. It achieves lower CPU and network overhead as well as co-located VM scalability. With real network intensive application workloads, we validate that CoKeeper can ensure user level and Linux kernel transparency even in cases of dynamic VM existence changes.
Z. Wang and Y. Ren—These authors contributed equally to this work.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Zhang, J., Lu, X., Panda, D.K.: Designing locality and NUMA aware MPI runtime for nested virtualization based HPC cloud with SR-IOV enabled InfiniBand. In: Proceedings of 13th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE 2017), Xi’an, China, 7–8 April 2017, pp. 187–200 (2017)
Zhang, Q., Liu, L.: Workload adaptive shared memory management for high performance network I/O in virtualized cloud. IEEE Trans. Comput. 65(11), 3480–3494 (2016)
Ahmad, F., Chakradhar, S.T., Raghunathan, A., Vijaykumar, T.N.: Shufflewatcher: shuffle-aware scheduling in multi-tenant MapReduce clusters. In: Proceedings of the 2014 USENIX Conference on USENIX Annual Technical Conference, Philadelphia, PA, USA, 19–20 June 2014, pp. 1–12. USENIX Association (2014)
Li, J., Xue, S., Zhang, W., Ma, R., Qi, Z., Guan, H.: When I/O interrupt becomes system bottleneck: efficiency and scalability enhancement for SR-IOV network virtualization. IEEE Trans. Cloud Comput. 7(4), 1183–1196 (2019)
Zhou, F., Ma, R., Li, J., Chen, L., Qiu, W., Guan, H.: Optimizations for high performance network virtualization. J Comput. Sci. Technol. 31(1), 107–116 (2016)
Zhang, J., Lu, X., Panda, D.K.: Performance characterization of hypervisor and container based virtualization for HPC on SR-IOV enabled InfiniBand clusters. In: Proceedings of IEEE International Symposium on Parallel and Distributed Processing, Workshops and PHD Forum (IPDPSW), Chicago, IL, USA, pp. 1777–1784 (2016)
Ren, Y., et al.: Shared-memory optimizations for inter virtual machine communication. ACM Comput. Surv. 48(4), 49:1–49:42 (2016)
Martini, B., Gharbaoui, M., Adami, D., Castoldi, P., Giordano, S.: Experimenting SDN and cloud orchestration in virtualized testing facilities: performance results and comparison. IEEE Trans. Netw. Service Manage. 16(3), 965–979 (2019)
Linguaglossa, L.: Survey of performance acceleration techniques for network function virtualization. Proc. IEEE 107(4), 746–764 (2019)
Overview of single root I/O virtualization (SR-IOV). http://msdn.microsoft.com/en-us/library/windows/hardware/hh440148%28v=vs.85%29.aspx. Accessed 28 Mar 2020
Xu, X., Davda, B.: A hypervisor approach to enable live migration with passthrough SR-IOV network devices. ACM SIGOPS Operating Syst. Rev. Spec. Top. 51(1), 15–23 (2017)
Russell, R.: Virtio: towards a de-facto standard for virtual I/O devices. ACM SIGOPS Operating Syst. Rev. 42(5), 95–103 (2008)
Intel Corporation. Intel data plane development kit: getting started guide for Linux. http://doc.dpdk.org/guides/linux_gsg/index.html. Accessed 15 Jul 2019
Hwang, J., Ramakrishnan, K.K., Wood, T.: NetVM: high performance and flexible networking using virtualization on commodity platforms. In: Proceedings of the 11th USENIX Symposium on Networked Systems Design and Implementation (NSDI), Seattle, WA, USA, 2–4 April 2014, pp. 445–458 (2014)
Wang, J., Wright, K.L., Gopalan, K.: XenLoop: a transparent high performance inter-VM network loopback. In: Proceedings of 17th International Symposium on High Performance Distributed Computing (HPDC), pp. 109–118. ACM, New York (2008)
Ren, Y., et al.: An efficient and transparent approach for adaptive intra- and inter-node virtual machine communication in virtualized clouds. In: Proceedings of 25th IEEE International Conference on Parallel and Distributed Systems (ICPADS 2019), Tianjin, China, 4–6 December 2019, pp. 35-44 (2019)
Radhakrishnan, P., Srinivasan, K.: MMNet: an efficient inter-VM communication mechanism. In: Proceedings of Xen Summit, Boston, USA, 23–24 June 2008
Huang, W., Koop, M., Gao, Q., Panda, D.K.: Virtual machine aware communication libraries for high performance computing. In: Proceedings of the 2007 ACM/IEEE Conference on Supercomputing (SC). ACM, New York. Article No. 9 (2007)
Kim, K., Kim, C., Jung, S., Shin, H., Kim, J.: Inter-domain socket communications supporting high performance and full binary compatibility on Xen. In: Proceedings of the 4th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE), Seattle, WA, USA, 5–7 March, pp. 1–10 (2008)
Zhang, X., McIntosh, S., Rohatgi, P., Griffin, J.L.: XenSocket: a high-throughput interdomain transport for virtual machines. In: Cerqueira, R., Campbell, R.H. (eds.) Middleware 2007. LNCS, vol. 4834, pp. 184–203. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-76778-7_10
Diakhaté, F., Perache, M., Namyst, R., Jourdren, H.: Efficient shared memory message passing for Inter-VM communications. In: César, E., et al. (eds.) Euro-Par 2008. LNCS, vol. 5415, pp. 53–62. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-00955-6_7
Macdonell, A.C.: Shared-memory optimizations for virtual machines. PhD thesis, Department of Computing Science, University of Alberta, Canada (2011)
Zhang, J., Lu, X., Jose, J., Li, M., Shi, R., Panda, D.K.: High performance MPI library over SR-IOV enabled InfiniBand clusters. In: Proceedings of 21st Annual IEEE International Conference on High Performance Computing (HiPC 2014), Goa, India, 17–20 December 2014
Zhang, Y., Juels, A., Oprea, A., Reiter, M.K.: HomeAlone: co-residency detection in the cloud via side-channel analysis. In: Proceedings of the 2011 IEEE Symposium on Security and Privacy (SP2011), Berkeley, CA, USA, 22–25 May 2011, pp. 313–328 (2011)
Bates, A., Mood, B., Pletcher, J., Pruse, H., Valafar, M., Butler, K.: Detecting co-residency with active traffic analysis techniques. In: Proceedings of the 2012 ACM Workshop on Cloud Computing Security Workshop (CCSW2012), Raleigh, North Carolina, USA, 19 October 2012, pp. 1–12 (2012)
Bates, A., Mood, B., Pletcher, J., Pruse, H., Valafar, M., Butler, K.: On detecting co-resident cloud instances using network flow watermarking techniques. Int. J. Inf. Secur. 13(2), 171–189 (2013). https://doi.org/10.1007/s10207-013-0210-0
Atya, A.O.F., Qian, Z., Krishnamurthy, S.V., Porta, T.L., McDaniel, P., Marvel, L.: Malicious co-residency on the cloud: attacks and defense. In: 36th IEEE Conference on Computer Communications (INFOCOM 2017), Atlanta, GA, USA, 1–4 May 2017, pp. 1–9 (2017)
Xenstore protocol specification. http://xenbits.xen.org/docs/4.6-testing/misc/xenstore.txt. Accessed 21 Mar 2020
Acknowledgment
The authors would like to thank Renshi Liu for his previous partial work on XenVMC. The work in this paper is supported by Natural Science Foundation of China (Under Grant NO.: 61872444 and U19A2060). Zhe Wang and Yi Ren contributed equally to this work.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, Z., Ren, Y., Guan, J., You, Z., Yang, S., Tan, Y. (2020). Dynamic Co-located VM Detection and Membership Update for Residency Aware Inter-VM Communication in Virtualized Clouds. In: Qiu, M. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2020. Lecture Notes in Computer Science(), vol 12454. Springer, Cham. https://doi.org/10.1007/978-3-030-60248-2_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-60248-2_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-60247-5
Online ISBN: 978-3-030-60248-2
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)