Skip to main content
SpringerLink
Log in

Job Allocation Strategies with User Run Time Estimates for Online Scheduling in Hierarchical Grids

  • Published:
Journal of Grid Computing Aims and scope Submit manuscript

Abstract

We address non-preemptive non-clairvoyant online scheduling of parallel jobs on a Grid. We consider a Grid scheduling model with two stages. At the first stage, jobs are allocated to a suitable Grid site, while at the second stage, local scheduling is independently applied to each site. We analyze allocation strategies depending on the type and amount of information they require. We conduct a comprehensive performance evaluation study using simulation and demonstrate that our strategies perform well with respect to several metrics that reflect both user- and system-centric goals. Unfortunately, user run time estimates and information on local schedules does not help to significantly improve the outcome of the allocation strategies. When examining the overall Grid performance based on real data, we determined that an appropriate distribution of job processor requirements over the Grid has a higher performance than an allocation of jobs based on user run time estimates and information on local schedules. In general, our experiments showed that rather simple schedulers with minimal information requirements can provide a good performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Krauter, K., Buyya, R., Maheswaran, M.: A taxonomy and survey of Grid resource management systems for distributed computing. International Journal of Software: Practice and Experience (SPE) 32, 135–164 (2002)

    Article  MATH  Google Scholar 

  2. Rodero, I., Corbalan, J., Badía, R.M., Labarta, J.: eNANOS Grid resource broker. In: Advances in Grid Computing. European Grid Conference (EGC 2005), pp. 111–121. Springer, Amsterdam (2005)

    Google Scholar 

  3. Rodero, I., Guim, F., Corbalan, J., Goyeneche, A.: The Grid backfilling: a multi-site scheduling architecture with data mining prediction techniques. In: Talia, D., Yahyapour, R., Ziegler, W. (eds.) Grid Middleware and Services. Challenges and Solutions, vol. 8, pp. 137–152. Springer, New York (2008)

    Chapter  Google Scholar 

  4. Elmroth, E., Tordsson, J.: An interoperable, standards-based Grid resource broker and job submission service. In: First International Conference on e-Science and Grid Computing, 2005, pp. 212–220. IEEE Computer Society, Melbourne, Vic. (2005)

  5. Avellino, G., Beco, S., Cantalupo, B., Maraschini, A., Pacini, F., Terracina, A., Barale, S., Guarise, A., Werbrouck, A., Sezione Di Torino, Colling, D., Giacomini, F., Ronchieri, E., Gianelle, A., Peluso, R., Sgaravatto, M., Mezzadri, M., Prelz, F., Salconi, L.: The EU datagrid workload management system: towards the second major release. In: 2003 Conference for Computing in High Energy and Nuclear Physics. University of California, La Jolla, California, USA (2003)

  6. Ranganathan, K., Foster, I.: Simulation studies of computation and data scheduling algorithms for data Grids. Journal Grid Computing 1, 53–62 (2003)

    Article  Google Scholar 

  7. Derbal, Y.: Entropic Grid scheduling. Journal Grid Computing 4, 373–394 (2006)

    Article  MATH  Google Scholar 

  8. de Lucchese, O.F., Huerta Yero, E., Sambatti, F., Henriques, M.: An adaptive scheduler for Grids. Journal Grid Computing 4, 1–17 (2006)

    Article  Google Scholar 

  9. Ernemann, C., Yahyapour, R.: Applying economic scheduling methods to Grid environments. In: Grid Resource Management: State of the Art and Future Trends, pp. 491–506. Kluwer, Dordrecht (2004)

    Google Scholar 

  10. Ernemann, C., Hamscher, V., Schwiegelshohn, U., Yahyapour, R., Streit, A.: On advantages of Grid computing for parallel job scheduling. In: 2nd IEEE/ACM International Symposium on Cluster Computing and the Grid, pp. 39. IEEE Computer Society (2002)

  11. Ernemann, C., Hamscher, V., Yahyapour, R.: Benefits of global Grid computing for job scheduling. In: Fifth IEEE/ACM International Workshop on Grid Computing (Grid ’04), in Conjunction with SuperComputing 2004, pp. 374–379. IEEE Computer Society, Pittsburgh (2004)

  12. Vázquez-Poletti, J.L., Huedo, E., Montero, R.S., Llorente, I.M.: A comparison between two Grid scheduling philosophies: EGEE WMS and Grid way. Multiagent and Grid System. Grid Computing, High Performance and Distributed Applications 3, 429–439 (2007)

    MATH  Google Scholar 

  13. Schwiegelshohn, U., Yahyapour, R.: Attributes for communication between Grid scheduling instances. In: Nabrzyski, J., Schopf, J.M., Weglarz, J. (eds.) Grid Resource Management: State of the Art and Future Trends, pp. 41–52. Kluwer, Norwell (2004)

    Google Scholar 

  14. Kurowski, K., Nabrzyski, J., Oleksiak, A., Weglarz, J.: A multicriteria approach to two-level hierarchy scheduling in Grids. J. Sched. 11, 371–379 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  15. Zikos, S., Karatza, H.D.: Resource allocation strategies in a 2-level hierarchical Grid system. In: Simulation Symposium, 2008. ANSS 2008. 41st Annual, pp. 157–164. Ottawa, Ont. (2008)

  16. Chunlin, L., Layuan, L.: Multi-level scheduling for global optimization in Grid computing. Comput. Electr. Eng. 34, 202–221 (2008)

    Article  MATH  Google Scholar 

  17. Wäldrich, O., Wieder, P., Ziegler, W.: A meta-scheduling service for co-allocating arbitrary types of resources. In: Wyrzykowski, R., Dongarra, J., Meyer, N., Wasniewski, J. (eds.) Parallel Processing and Applied Mathematics, vol. 3911, pp. 782–791. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  18. Tchernykh, A., Ramírez, J., Avetisyan, A., Kuzjurin, N., Grushin, D., Zhuk, S.: Two level job-scheduling strategies for a computational Grid. In: Wyrzykowski, R., Dongarra, J., Meyer, N., Wasniewski, J. (eds.) 6th International Conference on Parallel Processing and Applied Mathematics PPAM 2005, LNCS, vol. 3911, pp. 774–781. Springer, Heidelberg (2006)

    Google Scholar 

  19. Zhuk, S., Chernykh, A., Avetisyan, A., Gaissaryan, S., Grushin, D., Kuzjurin, N., Pospelov, A., Shokurov, A.: Comparison of scheduling heuristics for Grid resource broker. In: Third International IEEE Conference on Parallel Computing Systems (PCS 2004), pp. 388–392. IEEE, Colima, Colima, México (2004)

  20. Pugliese, A., Talia, D., Yahyapour, R.: Modeling and supporting Grid scheduling. Journal of Grid Computing 6, 195–213 (2008)

    Article  Google Scholar 

  21. Schwiegelshohn, U.: An owner-centric metric for the evaluation of online job schedules. In: Multidisciplinary International Conference on Scheduling. Theory and Applications (MISTA 2009), pp. 557–569. Dublin, Ireland (2009)

    Google Scholar 

  22. Graham, R.L., Lawler, E.L., Lenstra, J.K., Rinnooy Kan, A.H.G.: Optimization and approximation in deterministic sequencing and scheduling: a survey. In: Hammer, P.L., Johnson, E.L., Korte, B.H. (eds.) Annals of Discrete Mathematics 5. Discrete Optimization II, pp. 287–326. North-Holland, Amsterdam (1979)

    Google Scholar 

  23. Naroska, E., Schwiegelshohn, U.: On an on-line scheduling problem for parallel jobs. Inf. Process. Lett. 81, 297–304 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  24. Schwiegelshohn, U., Tchernykh, A., Yahyapour, R.: Online scheduling in Grids. In: IEEE International Symposium on Parallel and Distributed Processing 2008 (IPDPS 2008), pp. 1–10. Miami, FL, USA (2008)

  25. Garey, M.R., Graham, R.L.: Bounds for multiprocessor scheduling with resource constraints. SIAM J. Comput. 4, 187–200 (1975)

    Article  MathSciNet  MATH  Google Scholar 

  26. Tchernykh, A., Schwiegelshohn, U., Yahyapour, R., Kuzjurin, N.: Online hierarchical job scheduling on Grids. In: Priol, T., Vanneschi, M. (eds.) From Grids to Service and Pervasive Computing, pp. 77–91. Springer, New York (2008)

    Chapter  Google Scholar 

  27. Tchernykh, A., Schwiegelshohn, U., Yahyapour, R., Kuzjurin, N.: Online hierarchical job scheduling on Grids with admissible allocation. J. Sched. 13, 545–552 (2010). doi:10.1007/s10951-010-0169-x

    Article  MathSciNet  MATH  Google Scholar 

  28. Bar-Noy, A., Freund, A.: On-line load balancing in a hierarchical server topology. SIAM J. Comput. 31, 527–549 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  29. Pascual, F., Rzadca, K., Trystram, D.: Cooperation in multi-organization scheduling. Concurr. Comput.: Practice and Experience 21, 905–921 (2009)

    Article  Google Scholar 

  30. Zhuk, S.: Approximate algorithms to pack rectangles into several strips. Discrete Math. Appl. 16, 73–85 (2007)

    Article  MathSciNet  Google Scholar 

  31. Bougeret, M., Dutot, P.-F., Jansen, K., Otte, C., Trystram, D.: A fast 5/2 approximation algorithm for hierarchical scheduling. In: 16th International European Conference on Parallel and Distributed Computing, Euro-Par 2010. Ischia, Italy (2010)

  32. Tsafrir, D., Etsion, Y., Feitelson, D.G.: Modeling user runtime estimates. In: Feitelson, D.G., Frachtenberg, E., Rudolph, L., Schwiegelshohn, U. (eds.) 11th Workshop on Job Scheduling Strategies for Parallel Processing (JSSPP 2005). LNCS, vol. 3834, pp. 1–35. Springer, Cambridge (2006)

    Chapter  Google Scholar 

  33. Tsafrir, D., Etsion, Y., Feitelson, D.G.: Backfilling using system-generated predictions rather than user runtime estimates. IEEE Trans. Parallel Distrib. Syst. 18, 789–803 (2007)

    Article  Google Scholar 

  34. Chiang, S.-H., Arpaci-Dusseau, A.C., Vernon, M.K.: The impact of more accurate requested runtimes on production job scheduling performance. In: 8th International Workshop on Job Scheduling Strategies for Parallel Processing, pp. 103–127. Springer Verlang (2002)

  35. Bailey Lee, C., Schwartzman, Y., Hardy, J., Snavely, A.: Are user runtime estimates inherently inaccurate? In: Feitelson, D.G., Frachtenberg, E., Rudolph, L., Schwiegelshohn, U. (eds.) Job Scheduling Strategies for Parallel Processing. Springer, New York (2004)

    Google Scholar 

  36. Mu’alem, A.W., Feitelson, D.G.: Utilization, predictability, workloads, and user runtime estimates in scheduling the IBM SP2 with backfilling. IEEE Trans. Parallel Distrib. Syst. 12, 529–543 (2001)

    Article  Google Scholar 

  37. Guim, F., Corbalan, J., Labarta, J.: Prediction of based models for evaluating backfilling scheduling policies. In: Eighth International Conference on Parallel and Distributed Computing, Applications and Technologies, pp. 9–17. IEEE Computer Society (2007)

  38. Goyeneche, A., Guim, F., Rodero, I., Terstyanszky, G., Corbalan, J.: Extracting performance hints for Grid Users using data mining techniques: a case study in the NGS. The Mediterranean Journal of Computers and Networks (MEDJCN). SPECIAL ISSUE on Data Mining Applications on Supercomputing and Grid Environments 3(2), 52–61 (2007)

    Google Scholar 

  39. Smith, W.: Improving resource selection and scheduling using predictions. In: Nabrzyski, J., Schopf, J.M., Weglarz, J. (eds.) Grid resource management: state of the art and future trends, pp. 237–253. Kluwer, Dordrecht (2004)

    Google Scholar 

  40. Tsafrir, D., Feitelson, D.G.: The dynamics of backfilling: solving the mystery of why increased inaccuracy may help. In: IEEE International Symposium on Workload Characterization (IISWC 2006), pp. 131–141. IEEE, San Jose, California (2006)

  41. Zotkin, D., Keleher, P.J.: Job-length estimation and performance in backfilling schedulers. In: Eighth IEEE International Symposium on High Performance Distributed Computing (HPDC-8 ’99), pp. 39–46. IEEE Computer Society (1999)

  42. Talby, D., Tsafrir, D., Goldberg, Z., Feitelson, D.G.: Session-based, estimation-less, and information-less runtime prediction algorithms for parallel and Grid job scheduling. Technical report, School of Computer Science and Engineering, Hebrew University of Jerusalem (2006)

  43. Parallel Workloads Archive. http://www.cs.huji.ac.il/labs/parallel/workload/

  44. Grid Workloads Archive, TU Delft. http://gwa.ewi.tudelft.nl

Download references

Author information

Authors and Affiliations

  1. Computer Science Department, CICESE Research Center, Ensenada, BC, México

    Juan Manuel Ramírez-Alcaraz, Andrei Tchernykh, Ariel Quezada-Pina, José Luis González-García & Adán Hirales-Carbajal

  2. Telematics Faculty, Colima University, 28040, Colima, Col., México

    Juan Manuel Ramírez-Alcaraz

  3. IT and Media Center, Technische Universität Dortmund, 44221, Dortmund, Germany

    Ramin Yahyapour

  4. Robotics Research Institute, Technische Universität Dortmund, 44221, Dortmund, Germany

    Uwe Schwiegelshohn

  5. Science Faculty, Autonomous University of Baja California, Ensenada, 22860, BC, México

    Adán Hirales-Carbajal

Authors
  1. Juan Manuel Ramírez-Alcaraz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrei Tchernykh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ramin Yahyapour
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Uwe Schwiegelshohn
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ariel Quezada-Pina
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. José Luis González-García
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Adán Hirales-Carbajal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Andrei Tchernykh.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ramírez-Alcaraz, J.M., Tchernykh, A., Yahyapour, R. et al. Job Allocation Strategies with User Run Time Estimates for Online Scheduling in Hierarchical Grids. J Grid Computing 9, 95–116 (2011). https://doi.org/10.1007/s10723-011-9179-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10723-011-9179-y

Keywords

Search

Navigation