skip to main content
10.1145/2151024.2151030acmconferencesArticle/Chapter ViewAbstractPublication PagesveeConference Proceedingsconference-collections
research-article

Virtualization challenges: a view from server consolidation perspective

Published: 03 March 2012 Publication History

Abstract

Server consolidation, by running multiple virtual machines on top of a single platform with virtualization, provides an efficient solu-tion to parallelism and utilization of modern multi-core processors system. However, the performance and scalability of server con-solidation solution on modern massive advanced server is not well addressed. In this paper, we conduct a comprehensive study of Xen per-formance and scalability characterization running SPECvirt_sc2010, and identify that large memory and cache footprint, due to the unnecessary high frequent context switch, introduce additional challenges to the system performance and scalability. We propose two optimizations (dynamically-allocable tasklets and context-switch rate controller) to improve the performance. The results show the improved memory and cache efficiency with a reduction of the overall CPI, resulting in an improvement of server consolidation capability by 15% in SPECvirt_sc2010. In the meantime, our optimization achieves an up to 50% acceleration of service response, which greatly improves the QoS of Xen virtualization solution.

References

[1]
Intel Corporation. Terascale computing. http://www.intel.com/research/platform/terascale/index.htm.
[2]
Intel Corporation. Intel Develops Tera-Scale Research Chips. http://www.intel.com/pressroom/archive/releases/20060926corpb.htm.
[3]
VMware. http://www.vmware.com/.
[4]
Microsoft Virtual Server, http://www.microsoft.com/hyper-v-server/
[5]
Kernel Based Virtual Machine http://www.linux-kvm.org/
[6]
P. Barham, B. Dragovic, K. Fraser, S. Hand, T. Harris, A. Ho, R. Neugebauer, I. Pratt, and A. Warfield. Xen and the art of virtualization. In Proceedings of the Symposium on Operating Systems Principles (SOSP), Oct. 2003
[7]
SPECvirt_sc2010 http://www.spec.org/virt_sc2010/
[8]
P. ApparaoNewell, Towards Modeling & Analysis of Consolidated CMP Servers, Workshop on the Design, Analysis, and Simulation of Chip Multi-Processors (dasCMP), 2007
[9]
Jeffrey P. Casazza, Redefining server performance characterization for virtualization benchmarking, Intel Technology journal August 2006
[10]
I. M. Leslie, D. Mcauley, R. Black, T. Roscoe, P. T. Barham, D. Evers, R. Fairbairns, and E. Hyden. The Design and Implementation of an Operating System to Support Distributed Multimedia Applications. IEEE Journal of Selected Areas in Communications, 1996
[11]
VMmark http://www.vmware.com/products/vmmark/
[12]
M. Rosenblum. VMware's Virtual Platform: A virtual machine monitor for commodity PCs. In Hot Chips 11: Stanford University, Stanford, CA, August 15--17, 1999
[13]
K. Fraser, S. Hand, R. Neugebauer, I. Pratt, A. War_eld, and M. Williamson. Safe hardware access with the Xen virtual machine monitor. In 1st Workshop on Operating System and Architectural Support for the on demand IT InfraStructure (OASIS), Oct 2004
[14]
A. Menon, J. R. Santos, Y. Turner, G. J. Janakiraman, and W. Zwaenepoel. Diagnosing performance overheads in the Xen virtual machine environment. In Proceedings of the First ACM/USENIX International Conference on Virtual Execution Environments (VEE), pages 13--23, June 2005
[15]
Selvamuthukumar Senthilvelan and Murugappan Senthilvelan. Study of content-based sharing on the xen virtual machine monitor http://www.cs.wisc.edu/~remzi/Classes/736/Spring2005/Projects/Muru-Selva/cs736-report.pdf.
[16]
Wiegert, J., et al.: Challenges for Scalable Networking in a Virtualized Server. In: 16th International Conference on Computer Communications and Networks
[17]
A. Menon, A.L. Cox, and W. Zwaenepoel. Optimizing network virtualization in Xen. In Proceedings of the 2006 USENIX Annual Technical Conference, pages 15--28, June 2006
[18]
L. Cherkasova and R. Gardner. Measuring CPU overhead for I/O processing in the Xen virtual machine monitor. In USENIX Annual Technical Conference, Apr. 2005
[19]
Diego Ongaro, Alan L. Cox, Scott Rixner, Scheduling I/O in virtual machine monitors, Proceedings of the fourth ACM SIG-PLAN/SIGOPS international conference on Virtual execution environments, March 05-07, 2008, Seattle, WA, USA
[20]
Susan J. Eggers, Joel S. Emer, Henry M. Levy, Jack L. Lo, Rebecca L. Stamm, Dean M. Tullsen, Simultaneous Multithreading: A Platform for Next-Generation Processors, IEEE Micro, v.17 n.5, p.12--19, September 1997
[21]
Xudong Zheng, Jiangang Duan, Shameem F Akhter, Zhidong Yu, Hui Lv, A Consolidation Workload Characterization Study on Modern Platform, CMG'09
[22]
UHLIG, R., NEIGER, G., RODGERS, D., SANTONI, A. L., MARTINS, F. C. M., ANDERSON, A. V., BENNETT, S. M., KAGI, A., LEUNG, F. H., AND SIMTH, L. 2005. Intel Virtualization Technology. IEEE Computer. 38, 5, 48--56
[23]
Yaozu Dong, Zhao Yu, Greg Rose: SR-IOV Networking in Xen: Architecture, Design and Implementation. Workshop on I/O Virtualization 2008
[24]
DONG, Y., YANG, X., LI, X., LI, J., TIAN, K., AND GUAN, H. 2010. High Performance Network Virtualization with SR-IOV. In Proceeding of the 16th IEEE International Symposium on High-Performance Computer Architecture (HPCA'10). IEEE, Bangalore, India, 271--280
[25]
L. Cherkasova, D. Gupta, and A. Vahdat. Comparison of the Three CPU Schedulers in Xen. ACM SIGMETRICS Performance Evaluation Review, 35(2):42--51, 2007
[26]
Credit Scheduler. http://wiki.xensource.com/xenwiki/CreditScheduler
[27]
RAJ, H., AND SCHWAN, K. 2007. High performance and scalable I/O virtualization via self-virtualized devices. In Proceeding of the 16th international symposium on high performance distributed computing (HPDC'07). ACM, Monterrey, CA, 179--188
[28]
K. J. Duda and D. R. Cheriton. Borrowed-virtual-time (BVT) scheduling: supporting latency-sensitive threads in a general-purpose scheduler. In Proceedings of the 17th ACM SOSP, 1999.
[29]
GUO, D., LIAO, G., AND BHUYAN, L. N. 2009. Performance characterization and cache-aware core scheduling in a virtualized multi-core server under 10GbE. In Proceeding of 2009 IEEE International Symposium on Workload Characterization (IISWC'09). IEEE, Austin, TX, 168--177
[30]
Yaozu Dong, Xiaowei Yang, Xiaoyong Li, Jianhui Li, Kun Tian, Haibing Guan. High performance network virtualization with SR-IOV. HPCA'2010. pp.1~10
[31]
SANTOS, J. R., TURNER, Y. JANAKIRAMAN, G., AND PRATT, I. 2008. Bridging the gap between software and hardware techniques for I/O virtualization, In Proceeding of the USENIX Annual Technical Conference (USENIX'08). USENIX, Boston, MA, 29--42.
[32]
Hui Lv, Xudong Zheng, Zhiteng Huang, Jiangang Duan, Tackling the Challenges of Server Consolidation on Multi-Core Systems. IISWC'2010
[33]
Open-iscsi project http://www.open-iscsi.org/
[34]
J. Calandrino and J. Anderson. On the design and implementation of a cache-aware multicore real-time scheduler. In Proceedings of the 21st Euromicro Conference on Real-Time Systems, July 2009.
[35]
Arcangeli, Andrea ; Eidus, Izik ; Wright, Chris: Increasing memory density by using KSM. http://www.kernel.org/doc/ols/2009/#19--28. Version: 2009
[36]
LIAO, G., BHUYAN, L. N., WU, W., YU, H., AND KING, S. R. 2010. A new TCB cache to efficiently manage TCP sessions for Web servers. In Proceeding of the 6th ACM/IEEE Symposium on Architecture for Networking and Communication Systems (ANCS'10). ACM, San Diego, CA, 1--10.
[37]
Jun Nakajima, Qian Lin, Sheng Yang, Min Zhu, Shang Gao, Mingyuan Xia, Peijie Yu, Yaozu Dong, Zhengwei Qi, Kai Chen, Haibing Guan: Optimizing virtual machines using hybrid virtualization. SAC 2011: 573--578
[38]
A. J. Bernstein, Program Analysis for Parallel Processing, IEEE Trans. on Electronic Computers". EC-15, pp. 757--62, 1966
[39]
X. Zhang, A. E. Eichenberger, Y. Luo, K. O'Brien, K. O'Brien, Exploiting Parallelism with Dependence-Aware Scheduling, In Proceedings of the 18th International Conference on Parallel Architecture and Compilation Techniques (PACT'09) (Raleigh, NC, USA, Sept. 2009), ACM, pp. 193--202

Cited By

View all
  • (2023)Bulut Bilişim Sanal Sunucu Ürün Seçiminde Çok Kriterli Bir Karar Destek ModeliA Multi-Criteria Decision Support Model for Cloud Computing Virtual Server Product SelectionÇukurova Üniversitesi Mühendislik Fakültesi Dergisi10.21605/cukurovaumfd.141026938:4(939-953)Online publication date: 28-Dec-2023
  • (2022)PM-AIO: An Effective Asynchronous I/O System for Persistent MemoryIEEE Transactions on Emerging Topics in Computing10.1109/TETC.2021.310904710:3(1558-1574)Online publication date: 1-Jul-2022
  • (2018)Optimum Utilization of Resources Through Restricted Virtual Machine Migration and Efficient VM Placement in Cloud Data CenterInternational Journal of Distributed Systems and Technologies10.4018/IJDST.20181001019:4(1-19)Online publication date: 1-Oct-2018
  • Show More Cited By

Index Terms

  1. Virtualization challenges: a view from server consolidation perspective

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    VEE '12: Proceedings of the 8th ACM SIGPLAN/SIGOPS conference on Virtual Execution Environments
    March 2012
    248 pages
    ISBN:9781450311762
    DOI:10.1145/2151024
    • cover image ACM SIGPLAN Notices
      ACM SIGPLAN Notices  Volume 47, Issue 7
      VEE '12
      July 2012
      229 pages
      ISSN:0362-1340
      EISSN:1558-1160
      DOI:10.1145/2365864
      Issue’s Table of Contents
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 03 March 2012

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. performance optimization
    2. scheduler
    3. server consolidation
    4. specvirt_sc2010
    5. virtualization

    Qualifiers

    • Research-article

    Conference

    VEE '12

    Acceptance Rates

    Overall Acceptance Rate 80 of 235 submissions, 34%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)14
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 20 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2023)Bulut Bilişim Sanal Sunucu Ürün Seçiminde Çok Kriterli Bir Karar Destek ModeliA Multi-Criteria Decision Support Model for Cloud Computing Virtual Server Product SelectionÇukurova Üniversitesi Mühendislik Fakültesi Dergisi10.21605/cukurovaumfd.141026938:4(939-953)Online publication date: 28-Dec-2023
    • (2022)PM-AIO: An Effective Asynchronous I/O System for Persistent MemoryIEEE Transactions on Emerging Topics in Computing10.1109/TETC.2021.310904710:3(1558-1574)Online publication date: 1-Jul-2022
    • (2018)Optimum Utilization of Resources Through Restricted Virtual Machine Migration and Efficient VM Placement in Cloud Data CenterInternational Journal of Distributed Systems and Technologies10.4018/IJDST.20181001019:4(1-19)Online publication date: 1-Oct-2018
    • (2018)Software consolidation as an efficient energy and cost saving solutionFuture Generation Computer Systems10.1016/j.future.2015.11.02758:C(1-12)Online publication date: 30-Dec-2018
    • (2018)Enforcing CPU allocation in a heterogeneous IaaSFuture Generation Computer Systems10.1016/j.future.2015.05.01353:C(1-12)Online publication date: 30-Dec-2018
    • (2016)Billing system CPU time on individual VMProceedings of the 16th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing10.1109/CCGrid.2016.76(493-496)Online publication date: 16-May-2016
    • (2015)VMcSimProceedings of the 2015 IEEE 8th International Conference on Cloud Computing10.1109/CLOUD.2015.35(195-202)Online publication date: 27-Jun-2015
    • (2014)Enhancing the I/O system for virtual machines using high performance SSDs2014 IEEE 33rd International Performance Computing and Communications Conference (IPCCC)10.1109/PCCC.2014.7017096(1-8)Online publication date: Dec-2014
    • (2014)Managing Performance Overhead of Virtual Machines in Cloud Computing: A Survey, State of the Art, and Future DirectionsProceedings of the IEEE10.1109/JPROC.2013.2287711102:1(11-31)Online publication date: Jan-2014
    • (2014)Workload Prediction of Virtual Machines for Harnessing Data Center ResourcesProceedings of the 2014 IEEE International Conference on Cloud Computing10.1109/CLOUD.2014.76(522-529)Online publication date: 27-Jun-2014
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media