skip to main content
10.1145/3184407.3184414acmconferencesArticle/Chapter ViewAbstractPublication PagesicpeConference Proceedingsconference-collections
research-article

Virtualization Techniques Compared: Performance, Resource, and Power Usage Overheads in Clouds

Published: 30 March 2018 Publication History

Abstract

Virtualization solutions based on hypervisors or containers are enabling technologies for scalable, flexible, and cost-effective resource sharing. As the fundamental limitations of each technology are yet to be understood, they need to be regularly reevaluated to better understand the trade-off provided by latest technological advances. This paper presents an in-depth quantitative analysis of virtualization overheads in these two groups of systems and their gaps relative to native environments based on a diverse set of workloads that stress CPU, memory, storage, and networking resources. KVM and XEN are used to represent hypervisor-based virtualization, and LXC and Docker for container-based platforms. The systems were evaluated with respect to several cloud resource management dimensions including performance, isolation, resource usage, energy efficiency, start-up time, and density. Our study is useful both to practitioners to understand the current state of the technology in order to make the right decision in the selection, operation and/or design of platforms and to scholars to illustrate how these technologies evolved over time.

References

[1]
Dec 2016. KVM {online}. (Dec 2016). https://www.linux-kvm.org/page/Main_Page.
[2]
Feb 2016. STREAM Benchmark {online}. (Feb 2016). http://www.cs.virginia.edu/stream/.
[3]
Feb 2017. Bonnie++ {online}. (Feb 2017). http://www.coker.com.au/bonnie++/.
[4]
Jan 2015. Optimizing memory bandwidth on stream triad {on-line}. (Jan 2015). https://software.intel.com/en-us/articles/optimizing-memory-bandwidth-on-stream-triad.
[5]
Jan 2017. Google containers {online}. (Jan 2017). https://cloudplatform.googleblog.com/2014/06/an-update-on-container-support-on-google-cloud-platform.html.
[6]
June 2015. Mesos {online}. (June 2015). http://mesos.apache.org.
[7]
June 2016. Window container {online}. (June 2016). https://docs.microsoft.com/en-us/virtualization/windowscontainers/index.
[8]
March 2014. YARN {online}. (March 2014). https://www.ibm.com/developerworks/library/bd-yarn-intro/.
[9]
March 2015. Bootchart {online}. (March 2015). http://www.bootchart.org.
[10]
March 2016. Fio Benchmark {online}. (March 2016). https://github.com/axboe/fio.
[11]
March 2017. Blogbench {online}. (March 2017). https://www.pureftpd.org/project/blogbench.
[12]
March 2017. Isolation Benchmark Suite (IBS) {online}. (March 2017). http://web2.clarkson.edu/class/cs644/isolation.
[13]
March 2017. MallocBomb {online}. (March 2017). http://web2.clarkson.edu/class/cs644/isolation/download.html.
[14]
May 2015. Sysbench {online}. (May 2015). https://www.howtoforge.com/how-to-benchmark-your-system-cpu-file-io-mysql-with-sysbench.
[15]
Nov 2015. Openstack {online}. (Nov 2015). https://www.openstack.org/.
[16]
Nov 2016. Docker {online}. (Nov 2016). https://www.docker.com/.
[17]
Nov 2016. Linux VServer {online}. (Nov 2016). http://linux-vserver.org/Welcome_to_Linux-VServer.org.
[18]
Nov 2016. Lookbusy:A synthetic load generator {online}. (Nov 2016). http://www.devin.com/lookbusy/.
[19]
Nov 2016. LXC {online}. (Nov 2016). https://linuxcontainers.org/.
[20]
Nov 2016. Memtester:Memory-stresser benchmark {online}. (Nov 2016). https://linux.die.net/man/8/memtester.
[21]
Nov 2016. OpenVZ {online}. (Nov 2016). https://openvz.org/Main_Page.
[22]
Nov 2016. Rocket Container {online}. (Nov 2016). https://coreos.com/rkt.
[23]
Nov 2016. RSS:Resident set size {online}. (Nov 2016). http://www.unixlore.net/ articles/quick-easy-way-monitor-process-memory-usage.html.
[24]
Nov 2016. Type1vsType2 {online}. (Nov 2016). http://searchservervirtualization.techtargt.com/news/2240034817/KVM-reignites-Type-1-vs-Type-2-hypervisor-debate.
[25]
Sep 2017. Kubernetes 1.8 {online}. (Sep 2017). https://www.mirantis.com/blog/expect-kubernetes-1-8/.
[26]
T. Adufu, J. Choi, and Y. Kim. 2015. Is container-based technology a winner for high performance scientific applications? 17th Asia-Pacific Network Operations and Management Symposium (APNOMS). 507--510.
[27]
K. Agarwal, B. Jain, and D. E. Porter. 2015. Containing the hype The 6th Asia-Pacific Workshop on Systems. 8.
[28]
P. Barham, B. Dragovic, K. Fraser, S. Hand, T. Harris, A. Ho, R. Neugebauer, I. Pratt, and A. Warfield. 2003. Xen and the Art of Virtualization. SIGOPS Oper. Syst. Rev. Vol. 37, 5 (Oct. . 2003), 164--177.
[29]
D. Bernstein. 2014. Containers and cloud: From LXC to Docker to kubernetes. IEEE Cloud Computing 1, 3 (2014), 81?84.
[30]
L. Chen, S. Patel, H. Shen, and Z. Zhou. 2015. Profiling and understanding virtualization overhead in cloud. In ICPP. 31?40.
[31]
L. Cherkasova, D. Gupta, and A. Vahdat. 2007. Comparison of the three CPU schedulers in Xen. SIGMETRICS Perform. Eval. Rev. 35, 2 (2007), 42?51.
[32]
C. Delimitrou and C. Kozyrakis. 2013. ibench: Quantifying interference for datacenter applications. In Workload Characterization (IISWC). 23?33.
[33]
C. Delimitrou and C. Kozyrakis. 2014. Quasar: resource-efficient and QoS-aware cluster management. In ACM SIGPLAN Notices, Vol. 49. 127?144.
[34]
Z. J. Estrada, Z. Stephens, C. Pham, Z. Kalbarczyk, and R. K. Iyer. 2014. A performance evaluation of sequence alignment software in virtualized environments. In CCGrid. 730?737.
[35]
W. Felter, A. Ferreira, R. Rajamony, and J. Rubio. 2015. An updated performance comparison of virtual machines and linux containers. In ISPASS. 171?172.
[36]
Y. Gao, H. Guan, Z. Qi, B. Wang, and L. Liu. 2013. Quality of service aware power management for virtualized data centers. JSA 59, 4 (2013), 245?259.
[37]
C. D. Graziano. 2011. A performance analysis of Xen and KVM hypervisors for hosting the Xen Worlds Project. (2011).
[38]
R. Jones et al. 1996. NetPerf: a network performance benchmark. Information Networks Division, Hewlett-Packard Company (1996).
[39]
P.H. Kamp and R. NM. Watson. 2000. Jails: Confining the omnipotent root. In The 2nd International SANE Conference, Vol. 43. 116.
[40]
R. Morabito, J. Kjällman, and M. Komu. 2015. Hypervisors vs. lightweight virtualization: a performance comparison. In IC2E. 386?393.
[41]
C. Ruiz, E. Jeanvoine, and L. Nussbaum. 2015. Performance evaluation of containers for HPC. In VHPC. 12.
[42]
R. Schöne, D. Hackenberg, and D. Molka. 2012. Memory Performance at Reduced CPU Clock Speeds: An Analysis of Current x86_64 Processors. HotPower 12.
[43]
P. Sharma, L. Chaufournier, P. J Shenoy, and YC Tay. 2016. Containers and Virtual Machines at Scale: A Comparative Study. In Middleware. 1?1.
[44]
S. Soltesz, H. Pötzl, M. E. Fiuczynski, A. Bavier, and L. Peterson. 2007. Container-based operating system virtualization: a scalable, high-performance alternative to hypervisors. In ACM SIGOPS Operating Systems Review, Vol. 41. 275?287.
[45]
V. Spiliopoulos, S. Kaxiras, and G. Keramidas. 2011. Green governors: A framework for continuously adaptive DVFS. In GCC. 1?8.
[46]
I. Tafa, E. Zanaj, E. Kajo, A. Bejleri, and A. Xhuvani. 2011. The comparison of virtual machine migration performance between xen-hvm, xen-pv, open-vz, kvm-fv, kvm-pv. IJCSMS Int. J. Comput. Sci.: Manag. Stud 11, 2 (2011), 65?75.
[47]
S.K. Tesfatsion, L. Tomás, and J. Tordsson. 2017. OptiBook: Optimal resource booking for energy-efficient datacenters. In IWQoS.
[48]
S.K. Tesfatsion, E. Wadbro, and J. Tordsson. 2016. Autonomic Resource Management for Optimized Power and Performance in Multi-tenant Clouds. In ICAC.
[49]
M. G. Xavier, M. V. Neves, and C. A. F. De Rose. 2014. A performance comparison of container-based virtualization systems for mapreduce clusters. In PDP.
[50]
M. G Xavier, M. V. Neves, F. D Rossi, T. C Ferreto, T. Lange, and C. AF De Rose. 2013. Performance evaluation of container-based virtualization for high performance computing environments. In PDP. IEEE, 233?240.
[51]
J. Zhi. 2015. Literature Survey on Virtual Machine Performance Isolation. (2015).

Cited By

View all
  • (2024)Towards AI Onboard EO Satellites: Assessment of Virtualization Techniques for Extreme Edge ComputingIGARSS 2024 - 2024 IEEE International Geoscience and Remote Sensing Symposium10.1109/IGARSS53475.2024.10642768(1727-1732)Online publication date: 7-Jul-2024
  • (2024)Analysis of Virtualization Performance on Resource Efficiency Using Containers and Unikernel2024 International Conference on Artificial Intelligence, Blockchain, Cloud Computing, and Data Analytics (ICoABCD)10.1109/ICoABCD63526.2024.10704518(125-130)Online publication date: 20-Aug-2024
  • (2024)Enhanced Cloud Virtualization Based on Offloading Dynamic Switching Scheduling Algorithm Using Optimal Time Desk Task Allocation ModelIntelligent Computing, Smart Communication and Network Technologies10.1007/978-3-031-75957-4_5(46-57)Online publication date: 20-Nov-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ICPE '18: Proceedings of the 2018 ACM/SPEC International Conference on Performance Engineering
March 2018
328 pages
ISBN:9781450350952
DOI:10.1145/3184407
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 March 2018

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. VMS
  2. containers
  3. isolation
  4. performance
  5. power
  6. resource usage
  7. virtualization overhead

Qualifiers

  • Research-article

Conference

ICPE '18

Acceptance Rates

Overall Acceptance Rate 252 of 851 submissions, 30%

Upcoming Conference

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)75
  • Downloads (Last 6 weeks)3
Reflects downloads up to 14 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Towards AI Onboard EO Satellites: Assessment of Virtualization Techniques for Extreme Edge ComputingIGARSS 2024 - 2024 IEEE International Geoscience and Remote Sensing Symposium10.1109/IGARSS53475.2024.10642768(1727-1732)Online publication date: 7-Jul-2024
  • (2024)Analysis of Virtualization Performance on Resource Efficiency Using Containers and Unikernel2024 International Conference on Artificial Intelligence, Blockchain, Cloud Computing, and Data Analytics (ICoABCD)10.1109/ICoABCD63526.2024.10704518(125-130)Online publication date: 20-Aug-2024
  • (2024)Enhanced Cloud Virtualization Based on Offloading Dynamic Switching Scheduling Algorithm Using Optimal Time Desk Task Allocation ModelIntelligent Computing, Smart Communication and Network Technologies10.1007/978-3-031-75957-4_5(46-57)Online publication date: 20-Nov-2024
  • (2024)Kubernetes-in-the-Loop: Enriching Microservice Simulation Through Authentic Container OrchestrationPerformance Evaluation Methodologies and Tools10.1007/978-3-031-48885-6_6(82-98)Online publication date: 3-Jan-2024
  • (2023)Putting a Padlock on Lambda – Integrating vTPMs into AWS Firecracker2023 IEEE 22nd International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom)10.1109/TrustCom60117.2023.00188(1377-1384)Online publication date: 1-Nov-2023
  • (2023)PM $^{2}$ VE: Power Metering Model for Virtualization Environments in Cloud Data CentersIEEE Transactions on Cloud Computing10.1109/TCC.2023.3262648(1-14)Online publication date: 2023
  • (2023)An Empirical Study of Container Image Configurations and Their Impact on Start Times2023 IEEE/ACM 23rd International Symposium on Cluster, Cloud and Internet Computing (CCGrid)10.1109/CCGrid57682.2023.00019(94-105)Online publication date: May-2023
  • (2023)MonintainerJournal of Systems Architecture: the EUROMICRO Journal10.1016/j.sysarc.2023.103035145:COnline publication date: 1-Dec-2023
  • (2022)Radio: Reconciling Disk I/O Interference in a Para-virtualized Cloud2022 IEEE 15th International Conference on Cloud Computing (CLOUD)10.1109/CLOUD55607.2022.00034(144-156)Online publication date: Jul-2022
  • (2022)ShareOn: Shared Resource Dynamic Container Migration Framework for Real-Time Support in Mobile Edge CloudsIEEE Access10.1109/ACCESS.2022.318312210(66045-66060)Online publication date: 2022
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media