Abstract
Collaborative resource sharing in distributed computing requires scalable mechanisms for allocation and control of user quotas. Decentralized fairshare prioritization is a technique for enforcement of user quotas that can be realized without centralized control. The technique is based on influencing the job scheduling order of local resource management systems using an algorithm that establishes a semantic for prioritization of jobs based on the individual distances between user’s quota allocations and user’s historical resource usage (i.e. intended and current system state). This work addresses the design and evaluation of priority operators, mathematical functions to quantify fairshare distances, and identify a set of desirable characteristics for fairshare priority operators. In addition, this work also proposes a set of operators for fairshare prioritization, establishes a methodology for verification and evaluation of operator characteristics, and evaluates the proposed operator set based on this mathematical framework. Limitations in the numerical representation of scheduling factor values are identified as a key challenge in priority operator formulation, and it is demonstrated that the contributed priority operators (the Sigmoid operator family) behave robustly even in the presence of severe resolution limitations.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Raman, R., Livny, M., Solomon, M.: Matchmaking: distributed resource management for high throughput computing. In: Proceedings of the Seventh International Symposium on High Performance Distributed Computing, pp. 140–146. IEEE (1998)
Foster, I., Kesselman, C.: The Grid: Blueprint for a New Computing Infrastructure. Morgan Kaufmann, San Francisco (2004)
Foster, I., Kesselman, C., Tuecke, S.: The anatomy of the grid: enabling scalable virtual organizations. Int. J. Supercomput. Appl. 15(3), 200–222 (2001)
Kay, J., Lauder, P.: A fair share scheduler. Commun. ACM 31(1), 44–55 (1988)
Yoo, A.B., Jette, M.A., Grondona, M.: SLURM: simple linux utility for resource management. In: Feitelson, D., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2003. LNCS, vol. 2862, pp. 44–60. Springer, Heidelberg (2003)
Maui Cluster Scheduler, January 2014. http://www.adaptivecomputing.com/products/open-source/maui/
Elmroth, E., Gardfjäll, P.: Design and evaluation of a decentralized system for Grid-wide fairshare scheduling. In: Stockinger, H., et al. (eds.) Proceedings of e-Science 2005, pp. 221–229. IEEE CS Press (2005)
Östberg, P.-O., Espling, D., Elmroth, E.: Decentralized scalable fairshare scheduling. Future Gener. Comput. Syst. 29(1), 130–143 (2013)
Östberg, P.-O., Elmroth, E.: Decentralized prioritization-based management systems for distributed computing. In: 2013 IEEE 9th International Conference on eScience (eScience), pp. 228–237. IEEE (2013)
Espling, D., Östberg, P.-O., Elmroth, E.: Integration and evaluation of decentralized fairshare prioritization (aequus). In: Proceedings of PDSEC 2014 - The 15th IEEE International Workshop on Parallel and Distributed Scientific and Engineering Computing (PDSEC 2014), pp. 1198–1207. IEEE (2014)
SLURM: Multifactor priority plugin - simplified fair-share formula, January 2014. https://computing.llnl.gov/linux/slurm/priority_multifactor.html
Rodrigo, G.P.: Proof of compliance for the relative operator on the proportional distribution of unused share in an ordering fairshare system, January 2014. http://urn.kb.se/resolve?urn=urn:nbn:se:umu:diva-89298
Rodrigo, G.P.: Establishing the equivalence between operators, January 2014. http://urn.kb.se/resolve?urn=urn:nbn:se:umu:diva-89297
Swegrid: Swegrid organization, January 2014. http://snicdocs.nsc.liu.se/wiki/SweGrid
Jackson, D., Snell, Q., Clement, M.: Core algorithms of the Maui scheduler. In: Feitelson, D.G., Rudolph, L. (eds.) JSSPP 2001. LNCS, vol. 2221, pp. 87–102. Springer, Heidelberg (2001)
CERN: It services - batch service, January 2014. http://information-technology.web.cern.ch/services/batch
LSF: Fairshare scheduling, January 2014. http://www.ccs.miami.edu/hpc/lsf/7.0.6/admin/fairshare.html
Lehoczky, J., Sha, L., Ding, Y.: The rate monotonic scheduling algorithm: exact characterization and average case behavior. In: Proceedings of the Real Time Systems Symposium, pp. 166–171. IEEE (1989)
Sha, L., Lehoczky, J.P., Rajkumar, R.: Task scheduling in distributed real-time systems. In: Robotics and IECON 1987 Conferences, International Society for Optics and Photonics, pp. 909–917 (1987)
Lehoczky, J.P., Sha, L.: Performance of real-time bus scheduling algorithms. ACM SIGMETRICS Perform. Eval. Rev. 14(1), 44–53 (1986)
Acknowledgments
The authors extend their gratitude to Daniel Espling for prior work and technical support, Cristian Klein for feedback, and Tomas Forsman for technical assistance. Financial support for the project is provided by the Swedish Government’s strategic research effort eSSENCE and the Swedish Research Council (VR) under contract number C0590801 for the project Cloud Control.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Rodrigo, G.P., Östberg, PO., Elmroth, E. (2015). Priority Operators for Fairshare Scheduling . In: Cirne, W., Desai, N. (eds) Job Scheduling Strategies for Parallel Processing. JSSPP 2014. Lecture Notes in Computer Science(), vol 8828. Springer, Cham. https://doi.org/10.1007/978-3-319-15789-4_5
Download citation
DOI: https://doi.org/10.1007/978-3-319-15789-4_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-15788-7
Online ISBN: 978-3-319-15789-4
eBook Packages: Computer ScienceComputer Science (R0)