Skip to main content
SpringerLink
Log in

Data-Oriented Scheduling with Dynamic-Clustering Fault-Tolerant Technique for Scientific Workflows in Clouds

  • Published:
Programming and Computer Software Aims and scope Submit manuscript

Abstract

Cloud computing is one of the most prominent parallel and distributed computing paradigm. It is used for providing solution to a huge number of scientific and business applications. Large scale scientific applications which are structured as scientific workflows are evaluated through cloud computing. Scientific workflows are data-intensive applications, as a single scientific workflow may consist of hundred thousands of tasks. Task failures, deadline constraints, budget constraints and improper management of tasks can also instigate inconvenience. Therefore, provision of fault-tolerant techniques with data-oriented scheduling is an important approach for execution of scientific workflows in Cloud computing. Accordingly, we have presented enhanced data-oriented scheduling with Dynamic-clustering fault-tolerant technique (EDS-DC) for execution of scientific workflows in cloud computing. We have presented data-oriented scheduling as a proposed scheduling technique. We have also equipped EDS-DC with Dynamic-clustering fault-tolerant technique. To know the effectiveness of EDS-DC, we compared its results with three well-known enhanced heuristic scheduling policies referred to as: (a) MCT-DC, (b) Max-min-DC, and (c) Min-min-DC. We considered scientific workflow of CyberShake as a case study, because it contains most of the characteristics of scientific workflows such as integration, disintegration, parallelism, and pipelining. The results show that EDS-DC reduced make-span of 10.9% as compared to MCT-DC, 13.7% as compared to Max-min-DC, and 6.4% as compared to Min-min-DC scheduling policies. Similarly, EDS-DC reduced the cost of 4% as compared to MCT-DC, 5.6% as compared to Max-min-DC, and 1.5% as compared to Min-min-DC scheduling policies. These results in respect of make-span and cost are highly significant for EDS-DC as compared with above referred three scheduling policies. The SLA is not violated for EDS-DC in respect of time and cost constraints, while it is violated number of times for MCT-DC, Max-min-DC, and Min-min-DC scheduling techniques.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.

Similar content being viewed by others

REFERENCES

  1. Shi, J., Luo, J., Dong, F., Zhang, J., and Zhang, J., Elastic resource provisioning for scientific workflow scheduling in cloud under budget and deadline constraints, Cluster Comput., 2016, vol. 19, no. 1, pp. 167–182.

    Article  Google Scholar 

  2. Sun, D., Chang, G., Miao, C., and Wang, X., “Analyzing, modeling and evaluating dynamic adaptive fault tolerance strategies in cloud computing environments, J. Supercomput., 2013, vol. 66, no. 1, pp. 193–228.

    Article  Google Scholar 

  3. Lifka, D., et al., XSEDE Cloud Survey Report, Urbana, IL: Natl. Center Supercomput. Appl., 2013.

    Google Scholar 

  4. Li, X., Song, J., and Huang, B., A scientific workflow management system architecture and its scheduling based on cloud service platform for manufacturing big data analytics, Int. J. Adv. Manuf. Technol., 2016, vol. 84, nos. 1–4, pp. 119–131.

    Article  Google Scholar 

  5. Abbott, B.P., et al., LIGO: the Laser Interferometer Gravitational-Wave Observatory, Rep. Prog. Phys., 2009 vol. 72, no. 7, p. 76901.

    Article  Google Scholar 

  6. Bharathi, S., Deelman, E., Mehta, G., Vahi, K., Chervenak, A., and Su, M., Characterization of scientific workflows, Proc. 3rd Workshop on Workflows in Support of Large Scale Science, Austin, TX, 2008.

  7. Callaghan, S., et al., Metrics for heterogeneous scientific workflows : a case study of an earthquake science application, Int. J. High Perform. Comput. Appl., 2011, vol. 25, no. 3, pp. 274-285.

    Article  Google Scholar 

  8. Callaghan, S., et al., Reducing time-to-solution using distributed high-throughput mega-workflows—experiences from SCEC CyberShake, Proc. 4th Int. Conf. on e-Science, e-Science, December 7–12, 2008, Indianapolis, 2008, pp. 151–158.

  9. Abrishami, S., Naghibzadeh, M., and Epema, D.H.J., Deadline-constrained workflow scheduling algorithms for Infrastructure as a Service Clouds, Future Gener. Comput. Syst., 2013, vol. 29, no. 1, pp. 158–169.

    Article  Google Scholar 

  10. Chakraborty, D., Mankar, V.V., and Nanavati, A.A., Enabling runtime adaptation of workflows to external events in enterprise environments, Proc. IEEE Int. Conf. on Web Service ICWS 2007, Salt Lake City, 2007, pp. 1112–1119.

  11. Deelman, E., Singh, G., Livny, M., Berriman, B., and Good, J., “The cost of doing science on the cloud: the montage example, Proc. Int. Conf. High Performance Computing Networking, Storage and Analysis SC 2008, Austin, TX, 2008.

  12. Serrano, D., et al., “SLA guarantees for cloud services, Future Gener. Comput. Syst., 2016, vol. 54, pp. 233–246.

    Article  Google Scholar 

  13. Choi, J., Adufu, T., and Kim, Y., Data-locality aware scientific workflow scheduling methods in HPC cloud environments, Int. J. Parallel Program., 2017, vol. 45, no. 5, pp. 1128–1141.

    Article  Google Scholar 

  14. Tang, W., et al., Data-aware resource scheduling for multicloud workflows: a fine-grained simulation approach, Proc. Int. Conf. on Cloud Computer Technologies and Science CloudCom, Marrakesh, 2015, pp. 887–892.

  15. Chen, W. and Deelman, E., “Fault tolerant clustering in scientific workflows, Proc. 8th IEEE World Congr. on Service, Honolulu, 2012, pp. 9–16.

  16. Harshitha, S.B., Kaneria, P., and Manjaiah, D.H., Comparative study of workflow scheduling algorithms in cloud computing, Int. J. Innovation Res. Comput. Commun. Eng., 2014, special issue 2, pp. 31–37.

    Google Scholar 

  17. Mathew, T., et al., Study and analysis of various task scheduling algorithms in the cloud computing environment, Proc. Int. Conf. on Advances in Computing, Communications and Informatics (ICACCI), September 24–27, 2014, New Delhi, 2014, pp. 658–664.

  18. Chen, W. and Deelman, E., WorkflowSim: A toolkit for simulating scientific workflows in distributed environments, Proc. 8th IEEE Int. Conf. E-Science, Chicago, 2012.

  19. Kosar, T. and Balman, M., A new paradigm: data-aware scheduling in grid computing, Future Gener. Comput. Syst., 2009, vol. 25, no. 4, pp. 406–413.

    Article  Google Scholar 

  20. Zeng, L., Veeravalli, B., and Zomaya, A.Y., An integrated task computation and data management scheduling strategy for workflow applications in cloud environments, J. Network Comput. Appl., 2015, vol. 50, pp. 39–48.

    Article  Google Scholar 

  21. Poola, D., Ramamohanarao, K., and Buyya, R., Fault-tolerant workflow scheduling using spot instances on clouds, Procedia Comput. Sci., 2014, vol. 29, pp. 523–533.

    Article  Google Scholar 

  22. Kumar, D., Baranwal, G., Raza, Z., and Vidyarthi, D.P., A systematic study of double auction mechanisms in cloud computing, J. Syst. Software, 2017, vol. 125, pp. 234–255.

    Article  Google Scholar 

  23. Malawski, M., Juve, G., Deelman, E., and Nabrzyski, J., Algorithms for cost-and deadline-constrained provisioning for scientific workflow ensembles in IaaS clouds, Future Gener. Comput. Syst., 2015, vol. 48, pp. 1–18.

    Article  Google Scholar 

  24. He, X., Sun, X., and Laszewski, G., A QoS guided scheduling algorithm for grid computing, Office, 2002, vol. 18, no. 4, pp. 1–15.

    Google Scholar 

  25. Madureira, A.M. and Definitions, A.B., Ordered minimum completion time heuristic for unrelated parallel-machines problems, Proc. 9th Iberian Conf. on Information Systems and Technologies (CISTI), Barcelona, 2014.

  26. Priyadarsini, R.J., Performance evaluation of min-min and max-min algorithms for job scheduling in federated cloud, Int. J. Comput. Appl., 2014, vol. 99, no. 18, pp. 47–54.

    Google Scholar 

  27. Qureshi, K., Khan, F.G., Manuel, P., and Nazir, B., A hybrid fault tolerance technique in grid computing system, J. Supercomput., 2011, vol. 56, no. 1, pp. 106–128.

    Article  Google Scholar 

  28. Bala, A. and Chana, I., Fault tolerance-challenges, techniques and implementation in cloud computing, Int. J. Comput. Sci., 2012, vol. 9, no. 1, pp. 288–293.

    Google Scholar 

  29. Deelman, E., et al., “Pegasus, a workflow management system for science automation, Future Gener. Comput. Syst., 2015, vol. 46, pp. 17–35.

    Article  Google Scholar 

  30. Tolosana-Calasanz, R., Bañares, J.Á., Pham, C., and Rana, O.F., “Enforcing QoS in scientific workflow systems enacted over Cloud infrastructures, J. Comput. Syst. Sci., 2012, vol. 78, no. 5, pp. 1300–1315.

    Article  Google Scholar 

  31. Chen, W., Ferreira, R., Deelman, E., and Sakellariou, R., Balanced task clustering in scientific workflows, Proc. 9th IEEE Int. Conf. on e-Science, Beijing, 2013, pp. 1–8.

  32. Antonescu, A.F. and Braun, T., Simulation of SLA-based VM-scaling algorithms for cloud-distributed applications, Future Gener. Comput. Syst., 2016, vol. 54, pp. 260–273.

    Article  Google Scholar 

  33. Mustafa, S., Nazir, B., Hayat, A., ur Rehman Khan, A., and Madani, S.A., “Resource management in cloud computing: taxonomy, prospects, and challenges, Comput. Electron. Eng., 2015, vol. 47, pp. 186–203.

    Article  Google Scholar 

  34. Barladian, B.Kh., et al., An efficient mulithreading algorithm for the simulation of global illumination, Program. Comput. Software, 2017, vol. 43, no. 4, pp. 217–223.

    Article  Google Scholar 

  35. Fursova, N.I., et al., A lightweight method for virtual machine introspection, Program. Comput. Software, 2017, vol. 43, no. 5, pp. 307–313.

    Article  Google Scholar 

  36. Gusev, A.D., Nasonov, A.V., and Krylov, A.S., Fast parallel grid warping-based image sharpening method, Program. Comput. Software, 2017, vol. 43, no. 4, pp. 230–233.

    Article  MathSciNet  MATH  Google Scholar 

  37. Kuplyakov, D., Shalnov, E., and Konushin, A., Markov chain Monte Carlo based video tracking algorithm, Program. Comput. Software, 2017, vol. 43, no. 4, pp. 224–229.

    Article  MathSciNet  MATH  Google Scholar 

  38. Massobrio, R., et al., Towards a cloud computing paradigm for big data analysis in smart cities, Program. Comput. Software, 2018, vol. 44, no. 3, pp. 181–189.

    Article  Google Scholar 

  39. Muruganantham, R. and Ganeshkumar, P., Quality of service enhancement in wireless sensor network using flower pollination algorithm, Program. Comput. Software, 2018, vol. 44, no. 6, pp. 398–406.

    Article  Google Scholar 

  40. Raja, R. and Ganeshkumar, P., QoSTRP: a trusted clustering based routing protocol for mobile ad-hoc networks, Program. Comput. Software, 2018, vol. 44, no. 6, pp. 407–416.

    Article  Google Scholar 

  41. Pashchenko, N.F., Zipa, K.S., and Ignatenko, A.V., An algorithm for the visualization of stereo images simultaneously captured with different exposures, Program. Comput. Software, 2017, vol. 43, no. 4, pp. 250–257.

    Article  Google Scholar 

  42. Varnovskiy, N.P., et al., Secure cloud computing based on threshold homomorphic encryption, Program. Comput. Software, 2015, vol. 41, no. 4, pp. 215–218.

    Article  MathSciNet  MATH  Google Scholar 

  43. Zelenova, S.A., and Zelenov, S.V., Schedulability analysis for strictly periodic tasks in RTOS, Program. Comput. Software, 2018, vol. 44, no. 3, pp. 159–169.

    Article  MathSciNet  MATH  Google Scholar 

  44. Zipa, K.S. and Ignatenko, A.V., Algorithms for the analysis and visualization of high dynamic range images based on human perception, Program. Comput. Software, 2016, vol. 42, no. 6, pp. 367–374.

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Information Technology, Hazara University, Front of Multipurpose Hall, 21120, Dhodial, Mansehra, Khyber Pakhtunkhwa, Pakistan

    Z. Ahmad, A. I. Jehangiri, M. Iftikhar, A. I. Umer & I. Afzal

Authors
  1. Z. Ahmad
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. A. I. Jehangiri
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. M. Iftikhar
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. A. I. Umer
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. I. Afzal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Z. Ahmad, A. I. Jehangiri, M. Iftikhar, A. I. Umer or I. Afzal.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ahmad, Z., Jehangiri, A.I., Iftikhar, M. et al. Data-Oriented Scheduling with Dynamic-Clustering Fault-Tolerant Technique for Scientific Workflows in Clouds. Program Comput Soft 45, 506–516 (2019). https://doi.org/10.1134/S0361768819080097

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S0361768819080097

Search

Navigation