Abstract
In High Performance Computing, it is common to deal with substantial computing resources, and the use of a Resource Management System (RMS) becomes fundamental. The job scheduling algorithm is a key part of a RMS, and the selection of the best job scheduling that meets the user needs is of most relevance. In this work, we use a real-world scientific application to evaluate the performance of 4 different job scheduling algorithms: First in, first out (FIFO), Shortest Job First (SJF), EASY-backfilling and Fattened-backfilling. These algorithms worked with RMS SLURM workload manager, considering a scientific application that predicts the earth’s ionosphere dynamics. In the results we highlight each algorithm’s strength and weakness for different scenarios that change the possibility of advancing smaller jobs. To deepen our analysis, we also compared the job scheduling algorithms using 4 jobs of Numerical Aerodynamic Sampling (NAS) Parallel Benchmarks in a controlled scenario.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Arndt, O., Freisleben, B., Kielmann, T., Thilo, F.: A comparative study of online scheduling algorithms for networks of workstations. Cluster Comput. 3(2), 95–112 (2000)
Bailey, D.H.: NAS parallel benchmarks. In: Padua, D. (ed.) Encyclopedia of Parallel Computing, pp. 1254–1259. Springer, Boston (2011). https://doi.org/10.1007/978-0-387-09766-4
Bailey, D.H., et al.: The NAS parallel benchmarks. Int. J. Supercomput. Appl. 5(3), 63–73 (1991)
Natural Resources Canada: Monthly averages of solar 10.7 cm flux. http://www.spaceweather.gc.ca/solarflux/sx-5-mavg-en. Acessado em 20 de julho de 2018
Casalicchio, E., Perciballi, V.: Measuring docker performance: what a mess!!! In: Proceedings of the 8th ACM/SPEC on International Conference on Performance Engineering Companion, ICPE 2017 Companion, pp. 11–16. ACM, New York (2017)
Ciliendo, E., Kunimasa, T., Braswell, B.: Linux performance and tuning guidelines. IBM, International Technical Support Organization (2007). https://lenovopress.com/redp4285.pdf
Feitelson, D.G., Rudolph, L., Schwiegelshohn, U., Sevcik, K.C., Wong, P.: Theory and practice in parallel job scheduling. In: Feitelson, D.G., Rudolph, L. (eds.) JSSPP 1997. LNCS, vol. 1291, pp. 1–34. Springer, Heidelberg (1997). https://doi.org/10.1007/3-540-63574-2_14
Georgiou, Y.. et al.: A scheduler-level incentive mechanism for energy efficiency in HPC. In: 2015 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), pp. 617–626. IEEE (2015). https://ieeexplore.ieee.org/abstract/document/7152527
Georgiou, Y., Glesser, D., Trystram, D.: Adaptive resource and job management for limited power consumption. In: 2015 IEEE International Parallel and Distributed Processing Symposium Workshop (IPDPSW), pp. 863–870. IEEE (2015). https://ieeexplore.ieee.org/abstract/document/7284402
Gibbons, R.: A historical application profiler for use by parallel schedulers. In: Feitelson, D.G., Rudolph, L. (eds.) JSSPP 1997. LNCS, vol. 1291, pp. 58–77. Springer, Heidelberg (1997). https://doi.org/10.1007/3-540-63574-2_16
Gómez-Martín, C., Vega-Rodríguez, M.A., González-Sánchez, J.-L.: Fattened backfilling: an improved strategy for job scheduling in parallel systems. J. Parallel Distrib. Comput. 97, 69–77 (2016)
Mao, W., Kincaid, R.K.: A look-ahead heuristic for scheduling jobs with release dates on a single machine. Comput. Oper. Res. 21(10), 1041–1050 (1994)
Mu’alem, A.W., Feitelson, D.G.: Utilization, predictability, workloads, and user runtime estimates in scheduling the IBM SP2 with backfilling. IEEE Trans. Parallel Distrib. Syst. 12(6), 529–543 (2001)
Petry, A., et al.: First results of operational ionospheric dynamics prediction for the Brazilian space weather program. Adv. Space Res. 54(1), 22–36 (2014)
Skovira, J., Chan, W., Zhou, H., Lifka, D.: The EASY—LoadLeveler API project. In: Feitelson, D.G., Rudolph, L. (eds.) JSSPP 1996. LNCS, vol. 1162, pp. 41–47. Springer, Heidelberg (1996). https://doi.org/10.1007/BFb0022286
Subbulakshmi, T., Manjaly, J.S.: A comparison study and performance evaluation of schedulers in hadoop yarn. In: 2017 2nd International Conference on Communication and Electronics Systems (ICCES), pp. 78–83, October 2017
Talby, D., Feitelson, D.G.: Supporting priorities and improving utilization of the IBM SP scheduler using slack-based backfilling. In: 13th International and 10th Symposium on Parallel and Distributed Processing Parallel Processing, 1999. 1999 IPPS/SPDP. Proceedings, pp. 513–517. IEEE (1999)
Tsafrir, D., Etsion, Y., Feitelson, D.G.: Backfilling using system-generated predictions rather than user runtime estimates. IEEE Trans. Parallel Distrib. Syst. 18(6), 789–803 (2007)
Vasupongayya, S., Chiang, S.-H.: On job fairness in non-preemptive parallel job scheduling. In: IASTED PDCS, pp. 100–105. Citeseer (2005)
Wong, A.K., Goscinski, A.M.: The impact of under-estimated length of jobs on easy-backfill scheduling. In: 16th Euromicro Conference on Parallel, Distributed and Network-Based Processing, pp. 343–350. IEEE (2008). https://ieeexplore.ieee.org/abstract/document/4457142
Yoo, A.B., Jette, M.A., Grondona, M.: SLURM: simple Linux utility for resource management. In: Feitelson, D., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2003. LNCS, vol. 2862, pp. 44–60. Springer, Heidelberg (2003). https://doi.org/10.1007/10968987_3
Zhou, X., Chen, H., Wang, K., Lang, M., Raicu, I.: Exploring distributed resource allocation techniques in the SLURM job management system. Technical report Illinois Institute of Technology, Department of Computer Science (2013)
Acknowledgments
The author Fernando Emilio Puntel thanks Coordenaçño de Aperfeiçoamento de Pessoal de Nível Superior (CAPES) for the financial support in this research.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Puntel, F.E., Charão, A.S., Petry, A. (2020). Comparative Performance Analysis of Job Scheduling Algorithms in a Real-World Scientific Application. In: Gervasi, O., et al. Computational Science and Its Applications – ICCSA 2020. ICCSA 2020. Lecture Notes in Computer Science(), vol 12249. Springer, Cham. https://doi.org/10.1007/978-3-030-58799-4_33
Download citation
DOI: https://doi.org/10.1007/978-3-030-58799-4_33
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-58798-7
Online ISBN: 978-3-030-58799-4
eBook Packages: Computer ScienceComputer Science (R0)