Skip to main content

Advertisement

Log in

A new rule-based power-aware job scheduler for supercomputers

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

The fast processing speeds of the current generation of supercomputers provide a great convenience to scientists dealing with extremely large data sets. The next generation of exascale supercomputers could provide accurate simulation results for the automobile industry, aerospace industry, and even nuclear fusion reactors for the very first time. However, the energy cost of super-computing is extremely high, with a total electricity bill of 9 million dollars per year. Thus, conserving energy and increasing the energy efficiency of supercomputers have become critical in recent years. Many researchers have studied this problem and are trying to conserve energy by incorporating the dynamic voltage frequency scaling technique into their methods. However, this approach is limited, especially when the workload is high. In this paper, we developed a power-aware job scheduler by applying a rule-based control method and taking into consideration real-world power and speedup profiles to improve power efficiency while adhering to predetermined power constraints. The intensive simulation results showed that our proposed method is able to achieve the maximum utilization of computing resources as compared to baseline scheduling algorithms while keeping the energy cost under the threshold. Moreover, by introducing a power performance factor based on the real-world power and speedup profiles, we are able to increase the power efficiency by up to 75%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Niu S, Zhai J, Ma X, Liu M, Zhai Y, Chen W, Zheng W (2013) Employing checkpoint to improve job scheduling in large-scale systems. In: Job Scheduling Strategies for Parallel Processing. Springer, pp 36–55

  2. Supercomputer ’titans’ face huge energy costs. http://www.livescience.com/18072-rise-titans-exascale-supercomputers-leap-power-hurdle.html. Accessed 23 Jan 2012

  3. Reducing energy consumption and cost in the data center. http://www.datacenterknowledge.com/archives/2014/12/11/reducing-energy-consumption-cost-data-center/. Accessed 11 Dec 2014

  4. America’s data centers consuming and wasting growing amounts of energy. http://www.nrdc.org/energy/data-center-efficiency-assessment.asp. Accessed 6 Feb 2015

  5. Wang Y, Lu P (2013) DDS: a deadlock detection-based scheduling algorithm for workflow computations in HPC systems with storage constraints. Parallel Comput 39(8):291–305

    Article  Google Scholar 

  6. Chan H-L, Chan W-T, Lam T-W, Lee L-K, Mak K-S, Wong PW (2007) Energy efficient online deadline scheduling. In: Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms. Society for Industrial and Applied Mathematics, pp 795–804

  7. Wang X, Wang Y, Zhu H (2012) Energy-efficient task scheduling model based on mapreduce for cloud computing using genetic algorithm. J Comput 7(12):2962–2970

    Article  Google Scholar 

  8. Deore SS, Patil AN (2013) Energy-efficient job scheduling and allocation scheme for virtual machines in private clouds. Energy 5(1):56–60

    Google Scholar 

  9. Varsamopoulos G, Banerjee A, Gupta SK (2009) Energy efficiency of thermal-aware job scheduling algorithms under various cooling models. In: Contemporary Computing. Springer, pp 568–580

  10. Wang L-X (1999) A course in fuzzy systems. Prentice-Hall press, Upper Saddle River

    Google Scholar 

  11. Shaocheng T, Changying L, Yongming L (2009) Fuzzy adaptive observer backstepping control for mimo nonlinear systems. Fuzzy Sets Syst 160(19):2755–2775

    Article  MathSciNet  MATH  Google Scholar 

  12. Tong S, Li Y (2009) Observer-based fuzzy adaptive control for strict-feedback nonlinear systems. Fuzzy Sets Syst 160(12):1749–1764

    Article  MathSciNet  MATH  Google Scholar 

  13. The san diego supercomputer center (sdsc) sp2 log. http://www.cs.huji.ac.il/labs/parallel/workload/l_sdsc_sp2/index.html. Accessed 6 Feb 2015

  14. Logs of real parallel workloads from production systems. http://www.cs.huji.ac.il/labs/parallel/workload/logs.html. Accessed 6 Feb 2015

  15. The anl intrepid log. http://www.cs.huji.ac.il/labs/parallel/workload/l_anl_int/index.html. Accessed 6 Feb 2015

  16. The intel netbatch logs. http://www.cs.huji.ac.il/labs/parallel/workload/l_intel_netbatch/index.html. Accessed 6 Feb 2015

  17. Jackson D, Snell Q, Clement M (2001) Core algorithms of the maui scheduler. In: Job Scheduling Strategies for Parallel Processing. Springer, pp 87–102

  18. Soner S, Ozturan C (2013) An auction based slurm scheduler for heterogeneous supercomputers and its comparative performance study. Technical report, PRACE. http://www.prace-project.eu/IMG/pdf/wp59_an_auction_based_slurm_scheduler_heterogeneous_supercomputers_and_its_comparative_study.pdf. Accessed 6 Dec 2013

  19. Schroeder B, Harchol-Balter M (2004) Evaluation of task assignment policies for supercomputing servers: the case for load unbalancing and fairness. Clust Comput 7(2):151–161

    Article  Google Scholar 

  20. Chandio AA, Bilal K, Tziritas N, Yu Z, Jiang Q, Khan SU, Xu C-Z (2014) A comparative study on resource allocation and energy efficient job scheduling strategies in large-scale parallel computing systems. Clust Comput 17(4):1349–1367

    Article  Google Scholar 

  21. Mämmelä O, Majanen M, Basmadjian R, De Meer H, Giesler A, Homberg W (2012) Energy-aware job scheduler for high-performance computing. Comput Sci Res Dev 27(4):265–275

    Article  Google Scholar 

  22. Liu C, Qin X, Kulkarni S, Wang C, Li S, Manzanares A, Baskiyar S (2008) Distributed energy-efficient scheduling for data-intensive applications with deadline constraints on data grids. In: Performance, Computing and Communications Conference, 2008. IPCCC 2008. IEEE International. IEEE, pp 26–33

  23. Zhang Y, Duan L, Li B, Peng L, Sadagopan S (2014) Energy efficient job scheduling in single-isa heterogeneous chip-multiprocessors. In: 2014 15th International Symposium on Quality Electronic Design (ISQED). IEEE, pp 660–666

  24. Chen J-J, Kuo T-W (2005) Energy-efficient scheduling of periodic real-time tasks over homogeneous multiprocessors. PARC 1:30–35

    Google Scholar 

  25. Fox K, Im S, Moseley B (2013) Energy efficient scheduling of parallelizable jobs. In: Proceedings of the Twenty-Fourth Annual ACM-SIAM Symposium on Discrete Algorithms. SIAM, pp 948–957

  26. Khuller S, Li J, Saha B (2010) Energy efficient scheduling via partial shutdown. In: Proceedings of the Twenty-First Annual ACM-SIAM Symposium on Discrete Algorithms. Society for Industrial and Applied Mathematics, pp 1360–1372

  27. Zadeh LA (2008) Is there a need for fuzzy logic? Inf Sci 178(13):2751–2779

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding authors

Correspondence to Jun Wang or Dezhi Han.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, J., Han, D. & Wang, R. A new rule-based power-aware job scheduler for supercomputers. J Supercomput 74, 2508–2527 (2018). https://doi.org/10.1007/s11227-018-2281-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-018-2281-1

Keywords

Navigation