skip to main content
10.1145/3275219.3275234acmotherconferencesArticle/Chapter ViewAbstractPublication PagesinternetwareConference Proceedingsconference-collections
short-paper

A Fair Scheduling Algorithm for Adaptive Heterogeneous Resources in Data Centers

Published: 16 September 2018 Publication History

Abstract

The resource scheduling problem of data center clusters has always been a hot topic in the field of cloud computing. Existing research efforts focus on fairness, resource utilization and energy efficiency, and lack of research on heterogeneous clustering issues. To solve the problem that the traditional DRF algorithm does not consider the classification of machine performance and task type, this paper proposes a fair scheduling algorithm X-DRF that adapts to heterogeneous resources in the data center. The algorithm mainly classifies the performance of physical machines, increases the machine performance scoring factor, and increases the training and job type judgment classification of the XGBoost model. The experiments show that CPU utilization and memory usage increased by 10% and 6%, respectively. The normalized ratio is increased by about 3% compared to the original DRF system. Therefore, the presented fair scheduling algorithm for heterogeneous resources is more fair and reasonable in terms of resource allocation.

References

[1]
Benjamin Hindman, Andy Konwinski, Matei Zaharia, Ali Ghodsi, Anthony D. Joseph, Randy Katz, Scott Shenker, and Ion Stoica. 2013. Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center. In Proceedings of the 8th USENIX conference on Networked systems design and implementation. USENIX Association, 429--483.
[2]
Christina Delimitrou and Christos Kozyrakis. 2014. Quasar: resource-efficient and QoS-aware cluster management. In International Conference on Architectural Support for Programming Languages & Operating Systems ACM, 127--144.
[3]
Ying Li, Jing Zhang, Wei zhang, and Qing Liu. 2017. Cluster resource adjustment based on an improved artificial fish swarm algorithm in Mesos. In International Conference on Signal Processing. IEEE, 1843--1847.
[4]
Jalal Khamse Ashari, Ioannis Lambadaris, George Kesidis, Bhuvan Urgaonkar, and Yiqiang Zhao. 2017. Per-Server Dominant-Share Fairness (PS-DSF): A multiresource fair allocation mechanism for heterogeneous servers. In ICC 2017 - 2017 IEEE International Conference on Communications. IEEE, 1--7.
[5]
Ali Ghodsi, Matei Zaharia, Benjamin Hindman, Andy Konwinski, Scott Shenker, and Ion Stoica. 2011. Dominant resource fairness: fair allocation of multiple resource types. Usenix Conference on Networked Systems Design and Implementation. USENIX Association, 323--336.
[6]
Wei Wang, Ben Liang, and Baochun Li. 2015. Multi-Resource Fair Allocation in Heterogeneous Cloud Computing Systems. In Transactions on Parallel & Distributed Systems, IEEE 26(10), 2822--2835.
[7]
Wei Wang, Baochun Li, and Ben Liang. 2014. Dominant resource fairness in cloud computing systems with heterogeneous servers. In INFOCOM, 2014 Proceedings IEEE. IEEE, 583--591.
[8]
Tang Hongyan, Li Ying, Wang Long, Gu Jing, and Wu Zhonghai. 2017. Predicting Misconfiguration-Induced Unsuccessful Executions of Jobs in Big Data System. In Computer Software and Applications Conference. IEEE, 772--777.
[9]
Tianqi Chen, and Carlos Guestrin. 2016. XGBoost: A Scalable Tree Boosting System ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 785--794.
[10]
Manuel Fernández-Delgado, Eva Cernadas and Senén Barro. 2014. Do we need hundreds of classifiers to solve real world classification problems. Journal of Machine Learning Research 15(1), 3133--3181.
[11]
Haydar Ali Ismail and Mardhani Riasetiawan. 2017. CPU and memory performance analysis on dynamic and dedicated resource allocation using XenServer in Data Center environment. In International Conference on Science and Technology-Computer. IEEE, 17--22.
[12]
Yuri Torres, Arturo González-Escribano, Diego R, and Llanos Ferraris. 2013. UBench: Exposing the impact of CUDA block geometry in terms of performance. Journal of Supercomputing 65(3), 1150--1163.
[13]
Michael Isard, Vijayan Prabhakaran, Jon Currey, Udi Wieder, Kunal Talwar and Andrew Goldberg. 2009. Quincy: fair scheduling for distributed computing clusters. In ACM Sigops, Symposium on Operating Systems Principles. ACM, 261--276.
[14]
Ali Ghodsi, Matei Zaharia, Scott Shenker, and Ion Stoica. 2013. Choosy: Maxmin fair sharing for datacenter jobs with constraints. In Proceedings of the 8th ACM European Conference on Computer Systems. ACM, 365--378
[15]
Kay Ousterhout, Patrick Wendell, Matei Zaharia, and Ion Stoica. 2013. Sparrow: distributed, low latency scheduling. In Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles. ACM, 69--84.
[16]
Xiaoxu Chen, Heng Wu, Yuewen Wu, Zhigang Lu, and Wenbo Zhang. 2017. Large-Scale Resource Scheduling Based on Minimum Cost Maximum Flow. Journal of Software 28(3), 598--610.
[17]
Yash Ukidave, Xiangyu Li, and David Kaeli. 2016. Mystic:Predictive Scheduling for GPU Based Cloud Servers Using Machine Learning. In IEEE International Parallel and Distributed Processing Symposium. IEEE, 353--362.

Index Terms

  1. A Fair Scheduling Algorithm for Adaptive Heterogeneous Resources in Data Centers

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Other conferences
      Internetware '18: Proceedings of the 10th Asia-Pacific Symposium on Internetware
      September 2018
      167 pages
      ISBN:9781450365901
      DOI:10.1145/3275219
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      In-Cooperation

      • Institute of Software, Chinese Academy of Sciences
      • CCF: China Computer Federation

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 16 September 2018

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. Data center
      2. Fairness
      3. Heterogeneous cluster
      4. Machine learning
      5. Mesos
      6. Resource scheduling

      Qualifiers

      • Short-paper
      • Research
      • Refereed limited

      Funding Sources

      • the Natural Science Foundation of China

      Conference

      Internetware '18

      Acceptance Rates

      Internetware '18 Paper Acceptance Rate 20 of 26 submissions, 77%;
      Overall Acceptance Rate 55 of 111 submissions, 50%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • 0
        Total Citations
      • 101
        Total Downloads
      • Downloads (Last 12 months)1
      • Downloads (Last 6 weeks)0
      Reflects downloads up to 14 Feb 2025

      Other Metrics

      Citations

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media