Skip to main content
SpringerLink
Log in

A hybrid bi-objective scheduling algorithm for execution of scientific workflows on cloud platforms with execution time and reliability approach

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Heterogeneous cloud datacenters are well-suited and cost-efficient platforms for execution of scientific workflows requested from academics. Workflow scheduling algorithms have drastic impacts on the objectives that stakeholders in the system expect. This paper models the scientific workflow scheduling issue to a bi-objective optimization problem with makespan and reliability optimization approach because the users not only expect to have quick response, but also they need reliable executions. To address the issue, a new system framework and different concepts are introduced. A centralized log as a repository module is embedded in the system framework to register all kinds of system failures. In addition to, the new scheduling failure factor (SFF), which has reciprocal relation with system reliability, is defined. Therefore, the broker module quantifies the failure proneness of all resources and the most reliable ones are incorporated in scheduling model. The aforementioned scheduling model is then formulated to a bi-objective optimization problem with makespan and SFF minimization viewpoint which is an NP-Hard problem. To solve this combinatorial problem, a hybrid bi-objective discrete cuckoo search algorithm (HDCSA) is proposed. The proposed hybrid algorithm utilizes different novel Levy flight operators commensurate with discrete search space that makes good balance between exploration and exploitation in optimization process. The proposed HDCSA was validated in 12 extensive scenarios that were conducted on both symmetric and asymmetric scientific workflows in different conditions. The final results prove that the proposed bi-objective HDCSA scheduler has the amount of 22.11%, 12.97%, 11.81%, 12.18%, and 12.42% on average improvement against other state-of-the-arts in terms of makespan, SFF, speedup, efficiency, and SLR, respectively, which are prominent performance evaluation metrics is this scheduling domain.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Data availability

The data will be available upon reasonable request.

References

  1. Hosseini Shirvani M, Rahmani AM, Sahafi A (2018) An iterative mathematical decision model for cloud migration: a cost and security risk approach. Softw Pract Exp 48(3):449–485. https://doi.org/10.1002/spe.2528

    Article  Google Scholar 

  2. Przybylski B (2021) Parallel-machine scheduling of jobs with mixed job-, machine- and position-dependent processing times. J Comb Optim. https://doi.org/10.1007/s10878-021-00821-2

    Article  MATH  Google Scholar 

  3. Konjaang JK, Xu L (2021) Multi-objective workflow optimization strategy (MOWOS) for cloud computing. J Cloud Comput 10:11. https://doi.org/10.1186/s13677-020-00219-1

    Article  Google Scholar 

  4. Zhou X, Wang H, Ding B, Hu T, Shang S (2018) Balanced connected task allocations for multi-robot systems: an exact ßow-based integer program and an approximate tree-based genetic algorithm. Expert Syst Appl. https://doi.org/10.1016/j.eswa.2018.09.001

    Article  Google Scholar 

  5. Hosseini Shirvani M (2020) A hybrid meta-heuristic algorithm for scientific workflow scheduling in heterogeneous distributed computing systems. Eng Appl Artif Intell 90:103501. https://doi.org/10.1016/j.engappai.2020.103501

    Article  Google Scholar 

  6. Bharathi S, Chervenak A, Deelman E, Mehta G, Su MH, Vahi K (2008) Characterization of scientific workflows. In: 2008 Third Workshop on Workflows in Support of Large-Scale Science. IEEE, pp 1–10. https://doi.org/10.1109/WORKS.2008.4723958

  7. Mohammadzadeh A, Masdari M, Gharehchopogh FS et al (2021) A hybrid multi-objective metaheuristic optimization algorithm for scientific workflow scheduling. Cluster Comput 24:1479–1503. https://doi.org/10.1007/s10586-020-03205-z

    Article  Google Scholar 

  8. Topcuoglu H, Hariri S, Wu MY (2002) Performance-effective and low-complexity task scheduling for heterogeneous computing. IEEE Trans Parallel Distrib Syst 13(3):260–274. https://doi.org/10.1109/71.993206

    Article  Google Scholar 

  9. Mohammadzadeh A, Masdari M, Gharehchopogh FS et al (2021) Improved chaotic binary grey wolf optimization algorithm for workflow scheduling in green cloud computing. Evol Intell 14:1997–2025. https://doi.org/10.1007/s12065-020-00479-5

    Article  Google Scholar 

  10. Arabnejad H, Barbosa JG (2014) List scheduling algorithm for heterogeneous systems by an optimistic cost table. IEEE Trans Parallel Distrib Syst 25(3):682–694. https://doi.org/10.1109/TPDS.2013.57

    Article  Google Scholar 

  11. Xu Y, Li K, Hu J, Li K (2014) A genetic algorithm for task scheduling on heterogeneous computing systems using multiple priority queues. Inform Sci 270:255–287. https://doi.org/10.1016/j.ins.2014.02.122

    Article  MathSciNet  MATH  Google Scholar 

  12. Al Badawi A, Shatnawi A (2013) Static scheduling of directed acyclic data flow graphs onto multiprocessors using particle swarm optimization. Comput Oper Res 40(10):2322–2328. https://doi.org/10.1016/j.cor.2013.03.015

    Article  MathSciNet  MATH  Google Scholar 

  13. Dordaie N, Jafari Navimipour N (2018) A hybrid particle swarm optimization and hill climbing algorithm for task scheduling in the cloud environments. ICT Press. 4(4):199–202. https://doi.org/10.1016/j.icte.2017.08.001

    Article  Google Scholar 

  14. Keshanchi B, Jafari NN (2016) Priority-based task scheduling algorithm in cloud systems using a memetic algorithm. J Circuits Syst Comput 25(10):1650119. https://doi.org/10.1142/S021812661650119X

    Article  Google Scholar 

  15. Hosseini Shirvani M, Noorian Talouki R (2021) Bi-objective scheduling algorithm for scientific workflows on cloud computing platform with makespan and monetary cost minimization approach. Complex Intell Syst. https://doi.org/10.1007/s40747-021-00528-1

    Article  Google Scholar 

  16. Oukfif K, Oulebsir FB, Bouzefrane S, Banerjee S (2020) Workflow scheduling with data transfer optimization and enhancement of reliability in cloud data centers. Int J Commun Netw Distrib Syst. https://doi.org/10.1504/IJCNDS.2020.10021223

    Article  Google Scholar 

  17. Boeres C, Sardiña IM, Drummond LMA (2011) An efficient weighted bi-objective scheduling algorithm for heterogeneous systems. Parallel Comput 37(8):349–364. https://doi.org/10.1016/j.parco.2010.10.003

    Article  Google Scholar 

  18. Zhang L, Li K, Li C, Li K (2017) Bi-objective workflow scheduling of the energy consumption and reliability in heterogeneous computing systems. Inf Sci 379(10):241–256. https://doi.org/10.1016/j.ins.2016.08.003

    Article  Google Scholar 

  19. Wang X, Yeo CS, Buyya R, Su J (2011) Optimizing makespan and reliability for workflow applications with reputation and look-ahead genetic algorithm. Fut Gener Comput Syst 27(8):1124–1134. https://doi.org/10.1016/j.future.2011.03.008

    Article  Google Scholar 

  20. Amandeep V, Sakshi K (2017) A hybrid multi-objective particle swarm optimization for scientific workflow scheduling. Parallel Comput 62:1–19. https://doi.org/10.1016/j.parco.2017.01.002

    Article  MathSciNet  Google Scholar 

  21. Mohammadzadeh A, Masdari M, Gharehchopogh FS (2021) Energy and cost-aware workflow scheduling in cloud computing data centers using a multi-objective optimization algorithm. J Netw Syst Manag 29(31):2021. https://doi.org/10.1007/s10922-021-09599-4

    Article  Google Scholar 

  22. Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Proceedings of IEEE International Conference on Neural Networks. IV, pp 1942–1948. https://doi.org/10.1109/ICNN.1995.488968

  23. Mirjalili S, Mirjalili SM, Lewis A (2014) Grey Wolf optimizer. Adv Eng Softw 69(46–61):2014. https://doi.org/10.1016/j.advengsoft.2013.12.007

    Article  Google Scholar 

  24. Mirjalili S, Lewis A (2016) The Whale optimization algorithm. Adv Eng Softw 95:51–67. https://doi.org/10.1016/j.advengsoft.2016.01.008

    Article  Google Scholar 

  25. Sathya Sofia A, GaneshKumar P (2018) Multi-objective task scheduling to minimize energy consumption and makespan of cloud computing using NSGA-II. J Netw Syst Manag 26:463–485. https://doi.org/10.1007/s10922-017-9425-0

    Article  Google Scholar 

  26. Bahnasawy NA, Fatma O, Magdy AK, Mervat M (2011) Optimization procedure for algorithms of task scheduling in high performance heterogeneous distributed computing systems. Egypt Inform J 12(3):219–229. https://doi.org/10.1016/j.eij.2011.10.001

    Article  Google Scholar 

  27. Zhou J, Zhang M, Sun J, Wang T, Zhou X, Hu S (2022) DRHEFT: deadline-constrained reliability-aware HEFT algorithm for real-time heterogeneous MPSoC systems. IEEE Trans Reliab 71:178–189. https://doi.org/10.1109/TR.2020.2981419

    Article  Google Scholar 

  28. Zhou J, Sun J, Zhou X, Wei T, Hu XS (2018) Resource management for improving soft error and lifetime reliability of real-time MPSoCs. IEEE Trans Comput-Aided Des Integr Circuits Syst 38(12):2215–2228. https://doi.org/10.1109/TCAD.2018.2883993

    Article  Google Scholar 

  29. Hosseini Shirvani MS, Noorian TR (2021) A novel hybrid heuristic-based list scheduling algorithm in heterogeneous cloud computing environment for makespan optimization. Parallel Comput 108:102828. https://doi.org/10.1016/j.parco.2021.102828

    Article  MathSciNet  Google Scholar 

  30. Gulbaz R, Siddiqui AB, Anjum N, Alotaibi AA, Althobaiti T, Ramzan N (2021) Balancer genetic algorithm—a novel task scheduling optimization approach in cloud computing. Appl Sci 11:6244. https://doi.org/10.3390/app11146244

    Article  Google Scholar 

  31. Alsaidy SA, Abbood AD, Sahib MA (2020) Heuristic initialization of PSO task scheduling algorithm in cloud computing. J King Saud Univ Comput Inf Sci. https://doi.org/10.1016/j.jksuci.2020.11.002

    Article  Google Scholar 

  32. Natesan G, Chokkalingam A (2019) Task scheduling in heterogeneous cloud environment using mean grey wolf optimization algorithm. ICT Press 5(2):110–114. https://doi.org/10.1016/j.icte.2018.07.002

    Article  Google Scholar 

  33. Chen X et al (2020) A WOA-based optimization approach for task scheduling in cloud computing systems. IEEE Syst J 14(3):3117–3128. https://doi.org/10.1109/JSYST.2019.2960088

    Article  Google Scholar 

  34. Thennarasu SR, Selvam M, Srihari K (2020) A new whale optimizer for workflow scheduling in cloud computing environment. J Ambient Intell Human Comput 12(3):3807–3814. https://doi.org/10.1007/s12652-020-01678-9

    Article  Google Scholar 

  35. Zhou J, Wang T, Cong P, Lu P, Wei T, Chen M (2019) Cost and makespan-aware workflow scheduling in hybrid clouds. J Syst Archit 100:101631. https://doi.org/10.1016/j.sysarc.2019.08.004

    Article  Google Scholar 

  36. Natesan G, Chokkalingam A (2020) An improved Grey Wolf optimization algorithm based task scheduling in cloud computing environment. Int Arab J Inf Technol 17(1):73–81. https://doi.org/10.34028/iajit/17/1/9

    Article  Google Scholar 

  37. Abdel-Basset M, Shahat DE, Deb K, Abouhawwash M (2020) Energy-aware whale optimization algorithm for real-time task scheduling in multiprocessor systems. Appl Soft Comput. https://doi.org/10.1016/j.asoc.2020.106349

    Article  Google Scholar 

  38. Zhou X, Zhang G, Wang T, Zhang M, Wang X, Zhang W (2020) Makespan–cost–reliability-optimized workflow scheduling using evolutionary techniques in clouds. J Circuits Syst Comput 29(10):1–21. https://doi.org/10.1142/S0218126620501674

    Article  Google Scholar 

  39. Tanha M, Hosseini Shirvani MS, Rahmani AM (2021) A hybrid meta-heuristic task scheduling algorithm based on genetic and thermodynamic simulated annealing algorithms in cloud computing environments. Neural Comput Appl 33:16951–16984. https://doi.org/10.1007/s00521-021-06289-9

    Article  Google Scholar 

  40. Noorian TR, Hosseini Shirvani MS, Motameni H (2021) A heuristic-based task scheduling algorithm for scientific workflows in heterogeneous cloud computing platforms. J King Saud Univ-Comput Inf Sci. https://doi.org/10.1016/j.jksuci.2021.05.011

    Article  Google Scholar 

  41. Durillo JJ, NaeV PR (2014) Multi-objective energy-efficient workflow scheduling using list-based heuristics. Futur Gener Comput Syst 36:221–236. https://doi.org/10.1016/j.future.2013.07.005

    Article  Google Scholar 

  42. Javadi B, Abawajy J, Buyya R (2012) Failure-aware resource provisioning for hybrid cloud infrastructure. J Parallel Distrib Comput 72(10):1318–1331. https://doi.org/10.1016/j.jpdc.2012.06.012

    Article  Google Scholar 

  43. Choudhary A, Govil MC, Singh G, Awasthi LK, Pilli ES (2018) Task clustering-based energy-aware workflow scheduling in cloud environment. In: 2018 IEEE 20th International Conference on High Performance Computing and Communications; IEEE 16th International Conference on Smart City; IEEE 4th International Conference on Data Science and Systems (HPCC/SmartCity/DSS), 2018, pp 968–973. https://doi.org/10.1109/HPCC/SmartCity/DSS.2018.00160.

  44. Lin CS, Lin CS, Lin YS, Hsiung PA, Shih C (2013) Multi-objective exploitation of pipeline parallelism using clustering, replication and duplication in embedded multi-core systems. J Syst Archit 59(10):1083–1094. https://doi.org/10.1016/j.sysarc.2013.05.024

    Article  Google Scholar 

  45. Akbari M, Rashidi H (2016) A multi-objectives scheduling algorithm based on cuckoo optimization for task allocation problem at compile time in heterogeneous systems. Expert Syst Appl 60(30):234–248. https://doi.org/10.1016/j.eswa.2016.05.014

    Article  Google Scholar 

  46. Jin S, Schiavone G, Turgut D (2008) A performance study of multiprocessor task scheduling algorithms. J Supercomput 43(1):77–97. https://doi.org/10.1007/s11227-007-0139-z

    Article  Google Scholar 

  47. Yang XS, Deb S (2009) Cuckoo search via Levy flights. In: Proceedings of World Congress on Nature and Biologically Inspired Computing, pp 210–214. https://doi.org/10.1109/NABIC.2009.5393690.

  48. Saeedi P, Hosseini Shirvani MS (2021) An improved thermodynamic simulated annealing-based approach for resource-skewness-aware and power-efficient virtual machine consolidation in cloud datacenters. Soft Comput 25:5233–5260. https://doi.org/10.1007/s00500-020-05523-1

    Article  Google Scholar 

  49. Mirmohseni SM, Javadpour A, Tang C (2021) LBPSGORA: create load balancing with particle swarm genetic optimization algorithm to improve resource allocation and energy consumption in clouds networks. Math Problems Eng 2021:1–15. https://doi.org/10.1155/2021/5575129

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Sari Branch, Islamic Azad University, Sari, Iran

    Yeganeh Asghari Alaie & Mirsaeid Hosseini Shirvani

  2. Future Technology Research Center, National Yunlin University of Science and Technology, Yunlin, Taiwan

    Amir Masoud Rahmani

Authors
  1. Yeganeh Asghari Alaie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mirsaeid Hosseini Shirvani
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Amir Masoud Rahmani
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mirsaeid Hosseini Shirvani.

Ethics declarations

Conflict of interest

There is not any conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Asghari Alaie, Y., Hosseini Shirvani, M. & Rahmani, A.M. A hybrid bi-objective scheduling algorithm for execution of scientific workflows on cloud platforms with execution time and reliability approach. J Supercomput 79, 1451–1503 (2023). https://doi.org/10.1007/s11227-022-04703-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-022-04703-0

Keywords

Search

Navigation