Skip to main content

Advertisement

Springer Nature Link
Log in

A Comparative Analysis of Adaptive Solutions for Grid Environments

  • Published:
International Journal of Parallel Programming Aims and scope Submit manuscript

Abstract

Grid computing environments are distributed systems composed by heterogeneous and geographically distributed resources. This type of systems mainly emerged to satisfy the increasing computing power demand within the scientific community. Despite the advantages of such paradigm, there are still several challenges related to the discovery, monitoring and selection of grid resources. Moreover, the dynamic nature and changing characteristics of such environments worsen the applications performance. Thus, improving their efficiency is a fundamental issue. The present contribution analyses two self-adaptive solutions focused on enhancing the grid resource selection process by using resources in an efficient way. On the one hand, the Efficient Resources Selection model which is defined from the user’s point of view (it avoids controlling or modifying the infrastructure) and it is based on the Scatter Search method for achieving a suitable selection of resources. On the other hand, Montera2, a framework designed for addressing an efficient execution of distributed applications on the grid; it defines and employs a dynamic scheduling algorithm to determine the size and number of tasks to be executed. Both approaches have been tested on a real European infrastructure belonging to the well-known European Grid Infrastructure (EGI) project. The study also compares both solutions with the standard scheduling technique that governs this infrastructure, the gLite WMS scheduler, showing a much better performance by reducing the final makespan by a factor of 20 if compared to the gLite WMS scheduler. An analysis of task and time overheads for both approaches is also included. Furthermore, comparisons with many other solutions proposed in the literature are presented, showing the advantages of our approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. The Computing Element, CE, is a set of services located in every grid site that provides access for Grid jobs to a local resource management system.

  2. When the performance of the infrastructure gets worse there are very few actions that users can carry out to mitigate it.

  3. http://www.egi.eu/.

  4. http://www.globus.org/.

  5. The Information System, IS, records information about both the status of resources and tasks.

  6. http://fusion.bifi.unizar.es/?page_id=118.

  7. http://www.egi.eu.

References

  1. Foster, I.: What is the grid? A three point checklist. GRIDtoday 1(6), 22–25 (2002)

    Google Scholar 

  2. Berman, F., et al.: Adaptive computing on the grid using AppLeS. IEEE Trans. Parallel Distrib. Syst. 14(4), 369–382 (2003)

    Article  Google Scholar 

  3. Vadhiyar, S.S., Dongarra, J.J.: Self adaptivity in grid computing. Concurr. Comput. Pract. Exp. 17(2–4), 235–257 (2005)

    Article  Google Scholar 

  4. Huedo, E., Montero, R.S., Llorente, I.M.: A framework for adaptive execution in grids. Softw. Pract. Exp. 34(7), 631–651 (2004)

    Article  Google Scholar 

  5. Keung, H.N.L.C., Dyson, J.R.D., Jarvis, S.A., Nudd, G.R.: Self-adaptive and self-optimising resource monitoring for dynamic grid environments. In: DEXA’04 Proceedings of the Database and Expert Systems Applications, 15th International Workshop, IEEE Computer Society, Zaragoza, Spain, pp. 689–693 (2004)

  6. Gao, Y., Rong, H., Huang, J.Z.: Adaptive grid job scheduling with genetic algorithms. Future Gener. Comput. Syst. 21(1), 151–161 (2005)

    Article  Google Scholar 

  7. Xhafa, F., Abraham, A.: Metaheuristics for scheduling in distributed computing environments. Stud. Comput. Intell. 146, 1–37 (2008)

    Article  Google Scholar 

  8. Xhafa, F., Abraham, A.: Computational models and heuristic methods for grid scheduling problems. Future Gener. Comput. Syst. 26(4), 608–621 (2010)

    Article  Google Scholar 

  9. Yu, J., Buyya, R., Ramamohanarao, K.: Workflow scheduling algorithms for grid computing. Metaheuristics Sched. Distrib. Comput. Environ. 146, 173–214 (2008)

    Article  Google Scholar 

  10. Yu, J., Buyya, R.: A taxonomy of workflow management systems for grid computing. J. Grid Comput. 3(3–4), 171–200 (2006)

    Google Scholar 

  11. Rahman, M., Ranjan, R., Buyya, R., Benatallah, B.: A taxonomy and survey on autonomic management of applications in grid computing environments. Concurr. Comput. Pract. Exp. 23(16), 1990–2019 (2011)

    Article  Google Scholar 

  12. Deelman, E., Singh, G., Su, M., Blythe, J.: Pegasus: a framework for mapping complex scientific workflows onto distributed systems. Sci. Program. 13(3), 219–237 (2005)

    Google Scholar 

  13. Cameron, D., Gholami, A., Karpenko, D., Konstantinov, A.: Adaptive data management in the ARC grid middleware. J. Phys. Conf. Ser. 331, 1990–2018 (2011)

  14. Batista, D.M., Da Fonseca, L.S.: A survey of self-adaptive grids. IEEE Commun. Mag. 48(7), 94–100 (2010)

    Article  Google Scholar 

  15. Laguna, M., Martí, R.: Scatter Search. Metaheuristic Procedures for Training Neural Networks. Springer, Berlin (2006)

    Google Scholar 

  16. Resende, M., Ribeiro, C., Glover, F., Martí, R.: Scatter Search and Path Relinking: Fundamentals, Advances and Applications. Handbook of Metaheuristics. Springer, Berlin (2009)

    Google Scholar 

  17. Botón-Fernández, M., Vega-Rodríguez, M.A., Prieto Castrillo, F.: A self-adaptive selection model based on the scatter search for grid applications, computer aided systems theory. IUCTC. Universidad de Las Palmas de Gran Canaria, Las Palmas de Gran Canaria, Spain, pp. 233–235 (2013)

  18. Rodríguez-Pascual, M., Martín Llorente, I., Mayo-García, R.: Montera: a framework for the efficient executions of Monte Carlo codes on the Grid. Comput. Inform. 32, 113–144 (2013)

    Google Scholar 

  19. Vazquez-Poletti, J.L., Huedo, E., Montero, R.S., Martín Llorente, I.: CD-HIT workflow execution on grids using replication heuristics. In: 8th IEEE International Symposium on Cluster Computing and the Grid, pp. 735–740 (2008)

  20. Herrera Sanz, J., et al.: Loosely-coupled loop scheduling in computational grids. In: 20th Internationa Parallel and Distributed Processing Symposium, 6 (2006)

  21. Groen, D., Harftst, S., Zwart, S.P.: On the origin of grid species: the living application. In: Proceedings of the 9th International Conference on Computational Science: Part I, LNCS, vol. 5544, Springer, Berlin, Heidelberg, pp. 205–212 (2009)

  22. Murugavel, S.S., Vadhiyar, S.S., Nanjundiah, R.S.: Adaptive executions of multi-physics coupled applications on batch grids. J. Grid Comput. 9(4), 455–478 (2011)

    Article  Google Scholar 

  23. Shin, W.C., Yang, C.-T., Tseng, S.-S.: Using a performance-based skeleton to implement divisible load applications on grid computing environments. J. Inf. Sci. Eng. 25(1), 59–81 (2009)

    Google Scholar 

  24. González-Vélez, H., Cole, M.: Adaptive structured parallelism for distributed heterogeneous architectures: a methodological approach with pipelines and farms. Concurr. Comput. Pract. Exp. 22, 2073–2094 (2010)

    Google Scholar 

  25. Kretsis, A., Kokkinos, P., Varvarigos, E.A.: Implementing and evaluating scheduling policies in gLite middleware. Concurr. Comput. Pract. Exp. 25(3), 349366 (2013)

    Article  Google Scholar 

  26. Hirales-Carbajal, A., et al.: Multiple workflow scheduling strategies with user run time estimates on a grid. J. Grid Comput. 10(2), 325–346 (2012)

    Article  Google Scholar 

  27. Ludascher, B et al.: Kepler: An extensible system for design and execution of scientific workflows. In: Proceedings of the 16th International Conference on Scientific and Statistical Database Management 2004, pp. 423–424 (2004)

  28. Quezada-Pina, A., et al.: Adaptive parallel job scheduling with resource admissible allocation on two-level hierarchical grids. Future Gener. Comput. Syst. 28(7), 1–12 (2012)

  29. Leal, K., Huedo, E.: A decentralized model for scheduling independent tasks in federated grids. Future Gener. Comput. Syst. 25(8), 840–852 (2009)

    Article  Google Scholar 

  30. Li, Y., Yang, Y., Ma, M., Zhou, L.: A hybrid load balancing strategy of sequential tasks for grid computing environments. Future Gener. Comput. Syst. 25(8), 819–828 (2009)

    Article  Google Scholar 

  31. Li, Y., Mascagni, M.: Grid-based Monte Carlo application. In: Grid, pp. 13–24 (2002)

  32. Díaz, J., Reyes, S., Niño, A., Muñoz-Caro, C.: Derivation of self-scheduling algorithms for heterogeneous distributed computer systems: application to internet-based grids of computers. Future Gener. Comput. Syst. 25(6), 617–626 (2009)

    Article  Google Scholar 

  33. Herrera Sanz, J.: Modelo de Programación Para Infraestructuras Grid Computacionales. Ph.D. Thesis. Universidad Complutense de Madrid, Madrid (2009)

  34. Netto, M.A.S., Buyya, R.: Coordinated rescheduling of bag-of-tasks for executions on multiple resource providers. Concurr. Comput. Pract. Exp. 24(12), 1362–1376 (2011)

    Article  Google Scholar 

  35. Camarasu-Pop, S., et al.: Monte Carlo simulation on heterogeneous distributed systems: a computing framework with parallel merging and checkpointing strategies. Future Gener. Comput. Syst. 29(3), 728–738 (2013)

    Article  Google Scholar 

  36. Mościcki, J.T., Lamanna, M., Bubak, M., Sloot, P.M.A.: Processing moldable tasks on the grid: late job binding with lightweight user-level overlay. Future Gener. Comput. Syst. 27(6), 725–736 (2011)

    Article  Google Scholar 

  37. Sim, J., Garrochinho, T., Veiga, L.: A checkpointing-enabled and resource-aware Java Virtual Machine for efficient and robust e-Science applications in grid environments. Concurr. Comput. Pract. Exp. 24(13), 1421–1442 (2012)

    Article  Google Scholar 

  38. Hsu, C.-C., Huang, K.-C., Wang, F.-J.: Online scheduling of workflow applications in grid environments. Future Gener. Comput. Syst. 27(6), 860–870 (2011). doi:10.1016/j.future.2010.10.015

    Article  Google Scholar 

  39. Trinder, P.W., et al.: Resource analyses for parallel and distributed coordination. Concurr. Comput. Pract. Exp. 25(3), 309–348 (2011)

    Article  Google Scholar 

  40. Tao, Y., et al.: Dependable grid workflow scheduling based on resource availability. J. Grid Comput. 11(1), 47–61 (2012)

    Article  Google Scholar 

  41. Pinel, F., Pecero, J.E., Bouvry, P., Khan, S.U.: A review on task performance prediction in multi-core based systems, pp. 615–620. Presented at the Proceedings of the 2011 IEEE 11th International Conference on Computer and Information Technology, Washington, DC, USA: IEEE Computer Society (2011)

  42. Alfaro, M.E., Huelsenbeck, J.P.: Comparative performance of Bayesian and AIC-based measures of phylogenetic model uncertainty. Syst. Biol. 55(1), 89–96 (2006)

    Article  Google Scholar 

  43. Andreo, P.: Monte Carlo techniques in medical radiation physics. Phys. Med. Biol. 36, 861 (1991)

    Article  Google Scholar 

  44. Lemarinier, P., Bouteiller, A., Herault, T., Krawezik, G., Cappello, F.: Improved message logging versus improved coordinated checkpointing for fault tolerant MPI, pp. 115–124. Presented at the CLUSTER ’04: Proceedings of the 2004 IEEE International Conference on Cluster Computing, IEEE Computer Society (2004)

  45. Coti, C., et al.: Blocking vs. non-blocking coordinated checkpointing for large-scale fault tolerant MPI, p. 18. Presented at the SC ’06: Proceedings of the 2006 ACM/IEEE conference on Supercomputing, ACM (2006)

  46. Bouteiller, A., Lemarinier, P., Krawezik, K., Capello, F.: Coordinated checkpoint versus message log for fault tolerant MPI, pp. 242–250. Presented at the Cluster Computing, 2003. Proceedings. 2003 IEEE International Conference on (2003)

  47. Caron, E., Garonne, V., Tsaregorodtsev, A.: Evaluation of meta-scheduler architectures and task assignment policies for high throughput computing. Technical Report 2005–27, Ecole Normale Suprieure de Lyon, France (2005)

  48. Banino, C., et al.: Scheduling strategies for master-slave tasking on heterogeneous processor platforms. IEEE Trans. Parallel Distrib. Syst. 15(4), 319–330 (2004)

    Article  Google Scholar 

  49. Vazquez-Poletti, J.L., Huedo, E., Montero, R.S.: A comparison between two grid scheduling philosophies: EGEE WMS and Grid Way. Multiagent Grid Syst. 3(4), 429–439 (2007)

    MATH  Google Scholar 

  50. Bosa, K., Schreiner, W.: Austrian Grid: Report on Experiments with Globus 4 and gLite, Technical Report, Johannes Kepler University, Linz, Austria (2008)

  51. Xhafa, F., Abraham, A.: Meta-heuristics for grid scheduling problems. In: Metaheuristics for Scheduling in Distributed Computing Environments, pp. 1–37 (2008)

  52. Dong, F., Akl, S.G.: Scheduling Algorithms for Grid Computing: State of the Art and Open Problems. Technical Report 2006–504, Queen’s University, Kingston, Ontario, USA (2006)

  53. Jha, S., Cole, M., Katz, D.S., Parashar, M., Rana, O., Weissman, J.: Distributed computing practice for large-scale science and engineering applications. Concurr. Comput. Pract. Exp. 25(11), 1559–1585 (2013)

    Article  Google Scholar 

  54. Chtepen, M., Dhoedt, B., Cleays, F., Vanrolleghem, P.: Evaluation of replication and rescheduling heuristics for grid systems with varying resource availability, pp. 622–627. Presented at the Proceedings of the 18th IASTED international conference on parallel and distributed computing and systems (2006)

  55. Yu, C., Marinescu, D.C.: Algorithms for divisible load scheduling of data-intensive applications. J. Grid Comput. 8(1), 133–155 (2010). doi:10.1007/s10723-009-9129-0

    Article  MATH  Google Scholar 

  56. Olivier, S., Porterfield, A., Wheeler, K., Spiegel, M.: OpenMP task scheduling strategies for multicore NUMA systems. Int. J. High Perform. Comput. Appl. 26(2), 110–124 (2012)

    Article  Google Scholar 

  57. Ramírez-Alcaraz, A., et al.: Job allocation strategies with user run time estimates for online scheduling in hierarchical grids. J. Grid Comput. 9(1), 95–116 (2011)

    Article  Google Scholar 

  58. Rodríguez-Pascual, M., Guasp, J., Castejón Magaña, F., Rubio-Montero, A.J., Martín Llorente, I., Mayo-García, R.: Improvements on the Fusion Code FAFNER2. IEEE Trans. Plasma Sci. 38, 21022110 (2010). doi:10.1109/TPS.2010.2057450

    Google Scholar 

  59. Juve, G., et al.: Comparing futuregrid, amazon ec2, and open science grid for scientific workflows. Comput. Sci. Eng. 15(4), 20–29 (2013)

    Article  Google Scholar 

  60. Wrzesinska, G., Maasen, J., Bal, H.E.: Self-adaptive applications on the grid. In: 12th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, San Jose, CA, USA, pp. 121–129 (2007)

  61. Curnow, H.J., Wichmann, B.A.: A syntetic Benchmark. Comput. J. 19(1), 43–49 (1976)

    Article  Google Scholar 

  62. Hockney, R.W., Jesshope, C.R.: Parallel Computers Two: Architecture, Programming and Algorithms. Taylor & Francis, New York (1988)

    MATH  Google Scholar 

  63. Montero, R.S., Huedo, E., Martín Llorente, I.: Benchmarking of high throughput computing applications on grids. Parallel Comput. 32(4), 267–279 (2006)

    Article  Google Scholar 

  64. Trer, P., Domagalski, P.: Standardization of an API for distributed resource management systems. In: Proceedings of the Seventh IEEE International Symposium on Cluster Computing and the Grid (CCGrid 2007), pp. 619–626 (2007)

  65. Vázquez, C., Huedo, E., Montero, R.S.: Federation of TeraGrid, EGEE and OSG infrastructures through a metascheduler. Future Gener. Comput. Syst. 26(7), 979–985 (2010)

    Article  Google Scholar 

  66. Peris, A.D., Hernndez, J., Huedo, E., Martín Llorente, I.: Data location-aware job scheduling in the grid. Application to the GridWay metascheduler. J. Phys. Conf. Ser. 219, 62043 (2010)

    Article  Google Scholar 

Download references

Acknowledgments

Antonio Juan Rubio-Montero, from CIEMAT, was a great support on the creation and debugging of Montera2 and the administration of the local resources employed. Results obtained in this paper were computed on the European Grid Infrastructure (http://www.egi.eu) and its European Commission co-funded project EGI-InSPIRE (RI-261323). The authors thank the European Grid Infrastructure and supporting National Grid Initiatives for providing the technical support, computing and storage facilities. Authors have also count on the support from COST Action BETTY (IC1201). Last but not least, the authors would also like to acknowledge the support of the European Funds for Regional Development.

Author information

Authors and Affiliations

  1. CETA-CIEMAT, C. Sola 1, 10200, Trujillo, Spain

    María Botón-Fernández & Francisco Prieto-Castrillo

  2. CIEMAT, Avda Complutense 22, 28040, Madrid, Spain

    Manuel Rodríguez-Pascual & Rafael Mayo-García

  3. Escuela Politécnica, University of Extremadura, Avda Universidad s/n, 10003, Cáceres, Spain

    Miguel A. Vega-Rodríguez

Authors
  1. María Botón-Fernández
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Manuel Rodríguez-Pascual
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Miguel A. Vega-Rodríguez
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Francisco Prieto-Castrillo
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Rafael Mayo-García
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Manuel Rodríguez-Pascual.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Botón-Fernández, M., Rodríguez-Pascual, M., Vega-Rodríguez, M.A. et al. A Comparative Analysis of Adaptive Solutions for Grid Environments. Int J Parallel Prog 43, 786–811 (2015). https://doi.org/10.1007/s10766-014-0342-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10766-014-0342-5

Keywords