skip to main content
10.1145/1996130.1996163acmconferencesArticle/Chapter ViewAbstractPublication PageshpdcConference Proceedingsconference-collections
research-article

Dynamic adaptive scheduling for virtual machines

Published: 08 June 2011 Publication History

Abstract

With multi-core processors becoming popular, exploiting their computational potential becomes an urgent matter. The functionality of multiple standalone computer systems can be aggregated into a single hardware computer by virtualization, giving efficient usage of the hardware and decreased cost for power. Some principles of operating systems can be applied directly to virtual machine systems, however virtualization disrupts the basis of spinlock synchronization in the guest operating system, which results in performance degradation of concurrent workloads such as parallel programs or multi-threaded programs in virtual machines.
Eliminating this negative influence of virtualization on synchronization seems to be a non-trivial challenge, especially for concurrent workloads. In this work, we first demonstrate with parallel benchmarks that virtualization can cause long waiting times for spinlock synchronization in the guest operating system, resulting in performance degradation of parallel programs in the virtualized system. Then we propose an adaptive dynamic coscheduling approach to mitigate the performance degradation of concurrent workloads running in virtual machines, while keeping the performance of non-concurrent workloads. For this purpose, we build an adaptive scheduling framework with a series of algorithms to dynamically detect the occurrence of spinlocks with long waiting times, and determine and execute coscheduling of virtual CPUs on physical CPUs in the virtual machine monitor. We have implemented a prototype (ASMan) based on Xen and Linux. Experiments show that ASMan achieves better performance for concurrent workloads, while maintaining the performance for non-concurrent workloads. ASMan coscheduling depends directly on the dynamic behavior of virtual CPUs, unlike other approaches which depend on static properties of workloads and manual setting of rules. Therefore, ASMan achieves a better trade-off between coscheduling and non-coscheduling in the virtual machine monitor, and is an effective solution to this open issue.

References

[1]
P. H. Gum. System/370 extended architecture: Facilities for virtual machines. IBM Journal of Research and Development, 27(6):530--544, 1983.
[2]
J. E. Smith and R. Nair. Virtual Machines: Versatile platforms for systems and processes. Elsevier, USA, 2005.
[3]
C. A. Waldspurger. Memory resource management in \textscVMware \textscESX server. In Proceedings of the 5th symposium on Operating Systems Design and Implementation (OSDI), December 2002.
[4]
\textscVMware. http://www.vmware.com/.
[5]
B. Dragovic, K. Fraser, S. Hand, T. Harris, A. Ho, I. Pratt, A.Warfield, P. Barham, and R. Neugebauer. Xen and the art of virtualization. In Proceedings of the ACM Symposium on Operating Systems Principles (SOSP), pages 164--177, October 2003.
[6]
Xen. http://www.xen.org/.
[7]
KVM. http://www.linux-kvm.org/page/main\_page.
[8]
Hyper-\textscV. http://www.microsoft.com/windowsserver2008/en/ us/hyperv-main.aspx.
[9]
Virtualbox. http://www.virtualbox.org/.
[10]
J. Dike. User Mode Linux. Prentice Hall, 2006.
[11]
D. G. Feitelson, L. Rudolph, and U. Schwiegelshohn. Parallel job scheduling - a status report. In Proceedings of the Workshop on Job Scheduling Strategies for Parallel Processing, pages 1--16, 2004.
[12]
C. Weng, Z. Wang, M. Li, and X. Lu. The hybrid scheduling framework for virtual machine systems. In Proceedings of the 2009 ACM SIGPLAN/SIGOPS international conference on Virtual Execution Environments (VEE), pages 111--120, 2009.
[13]
VMware. \textscVMware v\textscSphere 4: The \textscCPU scheduler in \textscVMware \textscESX 4, 2009. http://www.vmware.com/files/pdf/perf-vsphere-cpu\_scheduler.pdf.
[14]
D. H. Bailey, E. Barszcz, J. T. Barton, D. S. Browning, R. L. Carter, L. Dagum, R. A. Fatoohi, P. O. Frederickson, T. A. Lasinski, R. S. Schreiber, H. D. Simon, V. Venkatakrishnan, and S. K. Weeratunga. The \textscNAS parallel benchmarks--summary and preliminary results. In Proceedings of the 1991 ACM/IEEE conference on Supercomputing, pages 158--165, 1991.
[15]
P. M. Chen and B. D. Noble. When virtual is better than real. In Proceedings of the Eighth Workshop on Hot Topics in Operating Systems (HotOS), 2001.
[16]
Credit Scheduler. http://wiki.xensource.com/xenwiki/credit scheduler.
[17]
P. J. Denning. The locality principle. Communications of the ACM, 48(7):19--24, 2005.
[18]
A. Aho, M. Lam, R. Sethi, and J. Ullman. Compilers: Principles, Techniques, and Tools (2nd Edition). Addison Wesley, 2006.
[19]
J. R. Spirn and P. J. Denning. Experiments with program locality. In Proceedings of the Fall Joint Computer Conference, pages 612--621, 1972.
[20]
A. Roth and I. Erev. Learning in extensive form games: Experimental data and simple dynamic models in the intermediate term. Games and econmic behavior, (8):164--212, 1995.
[21]
NAS Parallel Benchmarks. http://www.nas.nasa.gov/resources/software /npb.html.
[22]
Standard Performance Evaluation Corporation. http://www. spec.org/.
[23]
L. Cherkasova, D. Gupta, and A. Vahdat. Comparison of the three \textscCPU schedulers in \textscXen. ACM SIGMETRICS Performance Evaluation Review, 35(2):42--51, 2007.
[24]
Amazon EC2. http://aws.amazon.com/ec2/.
[25]
J. M. Mellor-Crummey and M. L. Scott. Algorithms for scalable synchronization on shared-memory multiprocessors. ACM Transactions on Computer Systems, 9(1):21--65, 1991.
[26]
D. Dice and A. Garthwaite. Mostly lock-free malloc. In Proceedings of the 3rd international symposium on Memory management, pages 163--174, 2002.
[27]
V. Uhlig, J. LeVasseur, E. Skoglund, and U. Dannowski. Towards scalable multiprocessor virtual machines. In Proceedings of the 3rd Virtual Machine Research and Technology Symposium, 2004.
[28]
T. Friebel and S. Biemueller. How to deal with lock holder preemption, 2008. http://www.amd64.org/fileadmin/user\_upload/pub/ 2008-Friebel-LHP-GI\_OS.pdf.
[29]
S. Govindan, J. Choi, A. R. Nath, A. Das, B. Urgaonkar, and A. Sivasubramaniam. \textscXen and \textscCo.: Communication-aware \textscCPU management in consolidated \textscXen-based hosting platforms. IEEE Transactions on Computers, 58(8):1111--1125, 2009.
[30]
H. Kim, H. Lim, J. Jeong, H. Jo, and J. Lee. Task-aware virtual machine scheduling for \textscI/O performance. In Proceedings of the 2009 ACM SIGPLAN/SIGOPS international conference on Virtual Execution Environments (VEE), pages 101--110, 2009.
[31]
J. K. Ousterhout. Scheduling techniques for concurrent systems. In Proceedings of the 3rd International Conference on Distributed Computing Systems (ICDCS), pages 22--30, 1982.
[32]
D. G. Feitelson and L. Rudolph. Gang scheduling performance benefits for fine-grain synchronization. Journal of Parallel and Distributed Computing, 16(4):306--318, 1992.
[33]
D. G. Feitelson and M. A. Jette. Improved utilization and responsiveness with gang scheduling. In Proceedings of the Job Scheduling Strategies for Parallel Processing, pages 238--261, 1997.
[34]
Z. Sergey, B. Sergey, and F. Alexandra. Addressing shared resource contention in multicore processors via scheduling. In Proceedings of the 15th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), pages 129--142, 2010.\endthebibliography

Cited By

View all
  • (2023)Revisiting VM-Agnostic KVM vCPU Scheduler for Mitigating Excessive vCPU SpinningIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2023.329768834:10(2615-2628)Online publication date: Oct-2023
  • (2023)Performance Analysis of Virtual Machine Monitoring System2023 IEEE Green Energy and Smart Systems Conference (IGESSC)10.1109/IGESSC59090.2023.10321757(1-6)Online publication date: 13-Nov-2023
  • (2021)Mitigating excessive vCPU spinning in VM-agnostic KVMProceedings of the 17th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments10.1145/3453933.3454020(139-152)Online publication date: 7-Apr-2021
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
HPDC '11: Proceedings of the 20th international symposium on High performance distributed computing
June 2011
296 pages
ISBN:9781450305525
DOI:10.1145/1996130
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 June 2011

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. adaptive strategy
  2. performance
  3. scheduling
  4. synchronization
  5. virtualization

Qualifiers

  • Research-article

Conference

HPDC '11
Sponsor:

Acceptance Rates

Overall Acceptance Rate 166 of 966 submissions, 17%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)8
  • Downloads (Last 6 weeks)3
Reflects downloads up to 23 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2023)Revisiting VM-Agnostic KVM vCPU Scheduler for Mitigating Excessive vCPU SpinningIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2023.329768834:10(2615-2628)Online publication date: Oct-2023
  • (2023)Performance Analysis of Virtual Machine Monitoring System2023 IEEE Green Energy and Smart Systems Conference (IGESSC)10.1109/IGESSC59090.2023.10321757(1-6)Online publication date: 13-Nov-2023
  • (2021)Mitigating excessive vCPU spinning in VM-agnostic KVMProceedings of the 17th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments10.1145/3453933.3454020(139-152)Online publication date: 7-Apr-2021
  • (2021)Accelerating Parallel Applications in Cloud Platforms via Adaptive Time-Slice ControlIEEE Transactions on Computers10.1109/TC.2020.299961970:7(992-1005)Online publication date: 1-Jul-2021
  • (2020)Doris: An Adaptive Soft Real-Time Scheduler in Virtualized EnvironmentsIEEE Transactions on Services Computing10.1109/TSC.2017.272073213:5(815-828)Online publication date: 1-Sep-2020
  • (2020)Locality-Aware Scheduling for Containers in Cloud ComputingIEEE Transactions on Cloud Computing10.1109/TCC.2018.27943448:2(635-646)Online publication date: 1-Apr-2020
  • (2019)A lock-aware virtual machine scheduling scheme for synchronization performanceThe Journal of Supercomputing10.1007/s11227-015-1557-y75:1(20-32)Online publication date: 1-Jan-2019
  • (2018)Hard real-time scheduling for parallel run-time systemsProceedings of the 27th International Symposium on High-Performance Parallel and Distributed Computing10.1145/3208040.3208052(14-26)Online publication date: 11-Jun-2018
  • (2017)Back-end scheduling methods of virtual clusters for energy conservation in event-driven WSNsInternational Journal of Ad Hoc and Ubiquitous Computing10.1504/IJAHUC.2017.08512225:4(194-203)Online publication date: 1-Jan-2017
  • (2017)Scheduler activations for interference-resilient SMP virtual machine schedulingProceedings of the 18th ACM/IFIP/USENIX Middleware Conference10.1145/3135974.3135975(222-234)Online publication date: 11-Dec-2017
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media