Skip to main content
SpringerLink
Log in

Bi-level fuzzy based advanced reservation of Cloud workflow applications on distributed Grid resources

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

The increasing demand on execution of large-scale Cloud workflow applications which need a robust and elastic computing infrastructure usually lead to the use of high-performance Grid computing clusters. As the owners of Cloud applications expect to fulfill the requested Quality of Services (QoS) by the Grid environment, an adaptive scheduling mechanism is needed which enables to distribute a large number of related tasks with different computational and communication demands on multi-cluster Grid computing environments. Addressing the problem of scheduling large-scale Cloud workflow applications onto multi-cluster Grid environment regarding the QoS constraints declared by application’s owner is the main contribution of this paper. Heterogeneity of resource types (service type) is one of the most important issues which significantly affect workflow scheduling in Grid environment. On the other hand, a Cloud application workflow is usually consisting of different tasks with the need for different resource types to complete which we call it heterogeneity in workflow. The main idea which forms the soul of all the algorithms and techniques introduced in this paper is to match the heterogeneity in Cloud application’s workflow to the heterogeneity in Grid clusters. To obtain this objective a new bi-level advanced reservation strategy is introduced, which is based upon the idea of first performing global scheduling and then conducting local scheduling. Global-scheduling is responsible to dynamically partition the received DAG into multiple sub-workflows that is realized by two collaborating algorithms: (1) The Critical Path Extraction algorithm (CPE) which proposes a new dynamic task overall critically value strategy based on DAG’s specification and requested resource type QoS status to determine the criticality of each task; and (2) The DAG Partitioning algorithm (DAGP) which introduces a novel dynamic score-based approach to extract sub-workflows based on critical paths by using a new Fuzzy Qualitative Value Calculation System to evaluate the environment. Local-scheduling is responsible for scheduling tasks on suitable resources by utilizing a new Multi-Criteria Advance Reservation algorithm (MCAR) which simultaneously meets high reliability and QoS expectations for scheduling distributed Cloud-base applications. We used the simulation to evaluate the performance of the proposed mechanism in comparison with four well-known approaches. The results show that the proposed algorithm outperforms other approaches in different QoS related terms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Algorithm 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Algorithm 2
Fig. 7
Algorithm 3
Algorithm 4
Algorithm 5
Algorithm 6
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Castillo C, Rouskas GN, Harfoush K (2007) On the design of online scheduling algorithms for advance reservations and QoS in grids. In: IEEE international parallel and distributed processing symposium, IPDPS

    Google Scholar 

  2. Wieczorek M, Prodan R, Fahringer T (2005) Scheduling of scientific workflows in the ASKALON grid environment. ACM SIGMOD Rec 34(3):56–62

    Article  Google Scholar 

  3. Ramakrishnan A, Singh G, Zhao H, Deelman E, Sakellariou R, Vahi K, Blackburn K, Meyers D, Samidi M (2007) Scheduling data intensive workflows onto storage-constrained distributed resources. In: Proceedings of the 7th IEEE symposium on cluster computing and the grid (CCGrid’07)

    Google Scholar 

  4. Yu J, Buyya R (2005) A taxonomy of scientific workflow systems for grid computing. SIGMOD Rec 34(3)

  5. Sih GC, Lee EA (1993) A compile-time scheduling heuristic for interconnection-constrained heterogeneous processor architectures. IEEE Trans Parallel Distrib Syst 4(2):75–87

    Article  Google Scholar 

  6. Kwok W et al (1996) Dynamic critical-path scheduling: an effective technique for allocating task graphs to multiprocessors. IEEE Trans Parallel Distrib Syst 7(5):506–521

    Article  Google Scholar 

  7. Topcuoglu H et al (2002) Performance-effective and low-complexity task scheduling for heterogeneous computing. IEEE Trans Parallel Distrib Syst 13(3):260–274

    Article  Google Scholar 

  8. Cheng J, Zeng G (2011) A two-phase approach to process partitioning for execution optimization migrating workflow. J Comput Interdiscip Sci 7:3478–3490

    Google Scholar 

  9. Tan W, Fan YS (2007) Dynamic workflow model fragmentation for distributed execution. Comput Ind 58(5):381–391

    Article  Google Scholar 

  10. Maurino A, Modafferi S (2005) Partitioning rules for orchestrating mobile information systems. Pers Ubiquitous Comput 9(5):291–300

    Article  Google Scholar 

  11. Baresi L, Maurino A, Modafferi S (2005) Workflow partitioning in mobile information systems. Int Fed Inf Process 158:93–106

    Google Scholar 

  12. Liu B, Wang Y, Jia Y, Wu QY (2005) A role-based approach for decentralized dynamic service composition. China J Softw 16(11):1859–1867

    Article  MATH  Google Scholar 

  13. Daoud MI et al (2011) A hybrid heuristic–genetic algorithm for task scheduling in heterogeneous processor networks. J Parallel Distrib Comput 71(11):1518–1531

    Article  Google Scholar 

  14. Omara FA et al (2010) Genetic algorithms for task scheduling problem. J Parallel Distrib Comput 70(1):13–22

    Article  MATH  Google Scholar 

  15. Sinnen O et al (2011) Contention-aware scheduling with task duplication. J Parallel Distrib Comput, 77–86

  16. Dong F (2009) Workflow scheduling algorithm in grid. PhD thesis

  17. El-Rewini H, Lewis T, Ali H (1994) Task scheduling in parallel and distributed systems. PTR Prentice Hall, New York. ISBN:0130992356

    Google Scholar 

  18. Wong K, Ahmad I (1999) Static scheduling algorithms for allocating directed task graphs to multiprocessors. ACM Comput Surv 31(4):406–471

    Article  Google Scholar 

  19. Casanova H et al (2010) On cluster resource allocation for multiple parallel task graphs. J Parallel Distrib Comput 70(12):1193–1203

    Article  MATH  Google Scholar 

  20. Deelman E, Mehta G, Singh G, Su M-H, Vahi K (2007) Pegasus: mapping large-scale workflows to distributed resources. In: Taylor I, Deelman E, Gannon DB, Shields M (eds) Workflows for e-science: scientific workflows for grids. Springer, Berlin

    Google Scholar 

  21. Deelman E, Singh G, Su M-H, Blythe J, Gil Y, Kesselman C, Mehta G, Vahi K, Berriman GB, Good J, Laity A, Jacob JC, Katz DS (2005) Pegasus: a framework for mapping complex scientific workflows onto distributed systems. Sci Program 13:219–237

    Google Scholar 

  22. Pegasus. http://pegasus.isi.edu

  23. Dong F, Akl SG (2007) Distributed double-level workflow scheduling algorithms for grid computing. J Inf Technol Appl 1(4):261–273

    Google Scholar 

  24. Prodan R, Wieczorek M (2010) Bi-criteria scheduling of scientific grid workflows. IEEE Trans Autom Sci Eng 7(2):364–376

    Article  Google Scholar 

  25. Duan R, Prodan R, Fahringer T (2007) Performance and cost optimization for multiple large-scale grid workflow applications. In: Proc of the 2007 ACM/IEEE conference on supercomputing, pp 1–12

    Chapter  Google Scholar 

  26. Yu J, Buyya R (2006) Scheduling scientific workflow applications with deadline and budget constraints using genetic algorithms. Sci Program 14(3, 4):217–230

    Google Scholar 

  27. Chen WN, Zhang J (2009) An ant colony optimization approach to grid workflow scheduling problem with various QoS requirements. IEEE Trans Syst Man Cybern 39(1):29–43

    Google Scholar 

  28. Tao Q, Chang H, Yi Y, Gu C, Yu Y (2009) QoS constrained grid workflow scheduling optimization based on a novel PSO algorithm. In: Eighth international conference on grid and cooperative computing, pp 153–159

    Google Scholar 

  29. Salehi MA, Buyya R (2010) Adapting market-oriented scheduling policies for cloud computing. In: Proceedings of the 10th int’l conference on algorithms and architectures for parallel processing, ICA3PP 2010, pp 351–362

    Google Scholar 

  30. Pandey S, Wu L, Guru S, Buyya R (2010) A particle swarm optimization-based heuristic for scheduling workflow applications in cloud computing environments. In: 24th IEEE international conference on advanced information networking and applications, AINA, pp 400–407

    Chapter  Google Scholar 

  31. Xu M, Cui L, Wang H, Bi Y (2009) A multiple QoS constrained scheduling strategy of multiple workflows for cloud computing. In: IEEE international symposium on parallel and distributed processing with applications, pp 629–634

    Google Scholar 

  32. Ostermann S, Prodan R, Fahringer T (2010) Dynamic cloud provisioning for scientific grid workflows. In: 11th IEEE/ACM international conference on grid computing, GRID, October 2010, pp 97–104

    Google Scholar 

  33. Byun E-K, Kee Y-S, Kim J-S, Deelman E, Maeng S (2011) BTS: resource capacity estimate for time-targeted science workflows. J Parallel Distrib Comput 71(6):848–862

    Article  Google Scholar 

  34. Byun E-K, Kee Y-S, Kim J-S, Maeng S (2011) Cost optimized provisioning of elastic resources for application workflows. Future Gener Comput Syst 27(8):1011–1026. [Online]. Available http://www.sciencedirect.com/science/article/pii/S0167739X11000744

    Article  Google Scholar 

  35. Chen WN et al (2009) An ant colony optimization approach to grid workflow scheduling problem with various QoS requirements. IEEE Trans Syst Man Cybern 39(1):29–43

    Google Scholar 

  36. Klir GJ (1995) Fuzzy set and fuzzy logic: theory and application. Prentice-Hall, Englewood Cliffs

    Google Scholar 

  37. Ross TJ (1995) Fuzzy logic with engineering applications. McGraw-Hill, New York

    MATH  Google Scholar 

  38. Kruatrachue B (1987) Static Task Scheduling and Grain Packing in Parallel Processing Systems. PhD thesis, Oregon State University

  39. Castillo C et al (2011) Online algorithms for advance resource reservations. J Parallel Distrib Comput, 963–973

  40. Tang X et al (2010) List scheduling with duplication for heterogeneous computing systems. J Parallel Distrib Comput 70(4):323–329

    Article  MATH  Google Scholar 

  41. Zhao L, Ren Y, Li M, Sakurai K (2012) Flexible service selection with user-specific QoS support in service-oriented architecture. J Netw Comput Appl 35(3):962–973

    Article  Google Scholar 

  42. Chunlin L, Xiu ZJ, Layuan L (2009) Resource scheduling with conflicting objectives in grid environments: model and evaluation. J Netw Comput Appl 32(3):760–769

    Article  Google Scholar 

  43. Abawajy JH (2009) Adaptive hierarchical scheduling policy for enterprise grid computing systems. J Netw Comput Appl 32(3):770–779

    Article  Google Scholar 

  44. Kangas J, Kangas A, Leskinen P, Pykalainen J (2001) MCDM methods in strategic planning of forestry on state-owned lands in Finland: applications and experiences. J Multi-Criteria Decision Anal, 257–271

  45. Saaty TL (1994) Fundamentals of decision making and priority theory with the analytic hierarch process. The analytic hierarch process series, vol VI. RWS, Pittsburgh

    Google Scholar 

  46. Daoud MI et al (2008) A high performance algorithm for static task scheduling in heterogeneous distributed computing systems. J Parallel Distrib Comput 68(4):399–409

    Article  MATH  Google Scholar 

  47. Taylor I, Deelman E, Gannon D, Shields M (2006) Workflows in e-science. Springer, Berlin

    Google Scholar 

  48. Afgan E, Bangalore P, Skala T (2012) Scheduling and planning job execution of loosely coupled applications. J Supercomput 59(3):1431–1454

    Article  Google Scholar 

  49. Li C, Li LY (2012) Optimal resource provisioning for cloud computing environment. J Supercomput 62(2):989–1022

    Article  Google Scholar 

  50. Falzon G, Li M (2012) Enhancing genetic algorithms for dependent job scheduling in grid computing environments. J Supercomput 62(1):290–314

    Article  Google Scholar 

  51. Luo J, Wu Z, Cao J, Tian T (2012) Dynamic multi-resource advance reservation in grid environment. J Supercomput 60(3):420–436

    Article  Google Scholar 

  52. Bradley A, Curran K, Parr G (2006) Discovering resources in computational grid environments. J Supercomput 35(1):27–49

    Article  Google Scholar 

  53. Cao J, Spooner DP, Jarvis SA, Nudd GR (2005) Grid load balancing using intelligent agents. Future Gener Comput Syst 21(1):135–149. Special issue on intelligent grid environment: principles and applications

    Article  Google Scholar 

Download references

Acknowledgements

This research is supported by Iran Telecommunication Research Center (ITRC). Our thanks go to Dr. Ali Rezaee who has contributed in this research.

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran

    Sahar Adabi & Amir Masoud Rahmani

  2. Department of Computer Engineering, Sharif University of Technology, Tehran, Iran

    Ali Movaghar

Authors
  1. Sahar Adabi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ali Movaghar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Amir Masoud Rahmani
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sahar Adabi.

Appendix

Appendix

For the benefit of readers, the authors summarize in Table 4 the key symbols and their definitions used in this paper.

Table 4 Notation and basic terms used in the paper
Full size table

Rights and permissions

Reprints and permissions

About this article

Cite this article

Adabi, S., Movaghar, A. & Rahmani, A.M. Bi-level fuzzy based advanced reservation of Cloud workflow applications on distributed Grid resources. J Supercomput 67, 175–218 (2014). https://doi.org/10.1007/s11227-013-0994-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-013-0994-8

Keywords

Search

Navigation