Skip to main content
SpringerLink
Log in

A grid workflow Quality-of-Service estimation based on resource availability prediction

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Accurate estimation of workflow Quality of Service (QoS) enhances the efficiency of scheduling algorithms. The availability and performance variations of Grid computing resources have made this estimation a great challenge. Most workflow QoS estimation algorithms are based on static performance of resources. In this paper, based on resources availability prediction, we propose an algorithm called WQE for estimating the QoS of a Grid workflow. WQE consists of two phases: resource monitoring and analysis and workflow QoS computation. In the first phase, two prediction algorithms are proposed to stochastically predict the availability state of resources. In the second phase, the QoS of each activity is estimated based on the host availability prediction result. The QoS of basic structures is computed by aggregating the QoS of their operands. Using a tree structure corresponding to the workflow, the QoS of basic structures is used to compute the total QoS of the workflow. The simulation results on Notre Dame University trace showed that the proposed method has higher estimation accuracy in comparison with HEFT.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24

Similar content being viewed by others

Notes

  1. The order of activities executions is also required when some parallel activities map to the same resource.

  2. Generally, the CPU threshold value can be defined by resource owners. The Condor uses 30 % as CPU threshold for an available state [6].

  3. In the case that |SI|=1, all observations refer to one availability state. In the case that |SI|=L, each observation refers to a different availability state in comparison with its previous and next observation.

  4. There is no difference between complexity of PW and PE. PW scans subintervals twice while PE scans once.

  5. As the number of availability levels is bounded, it can be ignored in complexity computation.

  6. Since we are assuming approximately the same amount of tasks, the workflow gets closer to the shape of a sequence structure.

References

  1. EGEE homepage (2008). http://egee.cesnet.cz/en/info/

  2. Teragrid homepage. http://www.teragrid.org

  3. PlanetLab (2008) P.L.A. open platform for developing debugging and accessing planetary scale services. http://www.planet-lab.org

  4. Frey J, Tannenbaum T, Livny M, Foster I, Tuecke S (2001) Condor-g: a computation management agent for multi-institutional grids. In: International conference on high performance distributed computing, pp 55–63

    Google Scholar 

  5. Litzkow M, Livny M, Mutka M (1988) Condor—a hunter of idle workstations. In: International conference on distributed computing systems, pp 104–111

    Google Scholar 

  6. Rood B, Lewis MJ (2009) Grid resource availability prediction-based scheduling and task replication. J Grid Comput 7:479–500

    Article  Google Scholar 

  7. Kiran M, Hashim A-HA, Kuan LM, Jiun YY (2009) Execution time prediction of imperative paradigm tasks for grid scheduling optimization. Int J Comput Sci Netw Secur 9:155–163

    Google Scholar 

  8. Smith W (2007) Prediction services for distributed computing. In: International symposium on parallel and distributed processing, pp 1–10

    Google Scholar 

  9. Tao M, Dong S, Zhang L (2010) A multi-strategy collaborative prediction model for the runtime of online tasks in computing cluster/grid. Clust Comput 14:199–210

    Google Scholar 

  10. Glasner C, Volkert J (2011) Adaps—a three-phase adaptive prediction system for the runtime of jobs based on user behaviour. J Comput Syst Sci 77:244–261

    Article  MathSciNet  Google Scholar 

  11. Byun E, Choi S, Baik M, Gil J, Park C, Hwang C (2007) MJSA: Markov job scheduler based on availability in desktop grid computing environment. Future Gener Comput Syst 23:616–622

    Article  Google Scholar 

  12. Ramakrishnan L, Reed D (2009) Predictable quality of service atop degradable distributed systems. Clust Comput. doi:10.1007/s10586-009-0078-y

  13. Wang H-C, Lee C-S, Ho T-H (2007) Combining subjective and objective QoS factors for personalized web service selection. Expert Syst Appl 32:571–584

    Article  Google Scholar 

  14. Hwang S-Y, Wang H, Tang J, Srivastava J (2007) A probabilistic approach to modeling and estimating the QoS of web-services-based workflows. Inf Sci 177:5484–5503

    Article  MATH  Google Scholar 

  15. Jaeger MC, Rojec-Goldmann G, Muehl G (2004) QoS aggregation for web service composition using workflow patterns. In: International conference on enterprise distributed object computing, pp 149–159

    Google Scholar 

  16. Zheng H, Yang J, Zhao W (2010) QoS probability distribution estimation for web services and service compositions. In: International conference on service-oriented computing and applications, pp 1–8

    Google Scholar 

  17. Maheswaran M, Ali S, Siegel HJ, Hensgen D, Freund RF (1999) Dynamic mapping of a class of independent tasks onto heterogeneous computing systems. J Parallel Distrib Comput 59:107–131

    Article  Google Scholar 

  18. Topcuoglu H, Hariri S, Wu MY (2002) Performance-effective and low-complexity task scheduling for heterogeneous computing. IEEE Trans Parallel Distrib Syst 13:260–274

    Article  Google Scholar 

  19. Cao H, Jin H, Wu X, Wu S, Shi X (2010) DAGMap: efficient and dependable scheduling of DAG workflow job in grid. J Supercomput 51:201–223

    Article  Google Scholar 

  20. Chen WN, Zhang J (2009) An ant colony optimization approach to a grid workflow scheduling problem with various QoS requirements. IEEE Trans Syst Man Cybern 39:29–43

    Article  Google Scholar 

  21. http://fta.scem.uws.edu.au/

  22. Wolski R, Spring N, Hayes J (1999) The network weather service: a distributed resource performance forecasting service for metacomputing. Future Gener Comput Syst 15:757–768

    Article  Google Scholar 

  23. Dinda P, O’Hallaron D (1999) An extensive toolkit for resource prediction in distributed systems. Technical report CMU-CS-99-138, Carnegie Mellon University

  24. Hu L, Che X-L, Zheng S-Q (2012) Online system for grid resource monitoring and machine learning-based prediction. IEEE Trans Parallel Distrib Syst 23:134–145

    Article  Google Scholar 

  25. Jiong Y, Guo-Zhong T, Ling C (2008) Allocating resource in grid workflow based on state prediction. In: IEEE/IFIP international conference on embedded and ubiquitous computing, pp 417–422

    Google Scholar 

  26. Lili S, Shoubao Y (2009) A Markov chain based resource prediction in computational grid. In: International conference on frontier of computer science and technology, pp 119–124

    Google Scholar 

  27. Ren X, Lee S, Eigenmann R, Bagchi S (2007) Prediction of resource availability in fine-grained cycle sharing systems empirical evaluation. J Grid Comput 5:173–195

    Article  Google Scholar 

  28. Wu AS, Yu H, Jin S, Lin K-C, Schiavone G (2004) An incremental genetic algorithm approach to multiprocessor scheduling. IEEE Trans Parallel Distrib Syst 15:824–834

    Article  Google Scholar 

  29. Tao F, Zhao D, Hu Y, Zhou Z (2008) Resource service composition and its optimal-selection based on particle swarm optimization in manufacturing grid system. IEEE Trans Ind Inform 4:315–327

    Article  Google Scholar 

  30. Young L, McGough S, Newhouse S, Darlington J (2003) Scheduling architecture and algorithms within the ICENI grid middleware. In: UK e-science all hands meeting, pp 5–12

    Google Scholar 

  31. Cardellini V, Casalicchio E, Grassi V, Iannucci S, Presti FL, Mirandola RR (2012) MOSES: a framework for QoS driven runtime adaptation of service-oriented systems. IEEE Trans Softw Eng 38:1138–1159

    Article  Google Scholar 

  32. Ramakrishnan L, Plale B (2010) A multi-dimensional classification model for scientific workflow characteristics. In: International workshop on workflow approaches to new data-centric science

    Google Scholar 

  33. Pegasus Workflow Generator. http://confluence.pegasus.isi.edu/display/pegasus/WorkflowGenerator

  34. Chong A, Sourin A, Levinski K (2006) Grid-based computer animation rendering. In: International conference on computer graphics and interactive techniques, pp 39–47

    Google Scholar 

Download references

Acknowledgements

This work is being supported by Iran Telecommunication Research Center (ITRC), Tehran, Iran, Contract No. 12200/500.

Author information

Authors and Affiliations

  1. Faculty of Electrical and Computer Engineering, Tarbiat Modares University, Tehran, Iran

    Somayeh Kianpisheh & Nasrolah Moghadam Charkari

Authors
  1. Somayeh Kianpisheh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nasrolah Moghadam Charkari
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nasrolah Moghadam Charkari.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kianpisheh, S., Moghadam Charkari, N. A grid workflow Quality-of-Service estimation based on resource availability prediction. J Supercomput 67, 496–527 (2014). https://doi.org/10.1007/s11227-013-1014-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-013-1014-8

Keywords

Search

Navigation