Skip to main content
Springer Nature Link
Log in

Cost-effective approaches for deadline-constrained workflow scheduling in clouds

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Nowadays, heterogeneous cloud resources are charged by cloud providers according to the pay-as-you-go pricing model. To execute workflow applications in clouds under deadline constraints, cloud resources have to be utilized appropriately and judiciously, challenging traditional workflow scheduling algorithms, which are either inapplicable to the cloud environment or fail to fully exploit the features of scheduling problem for cost optimization. In this paper, we propose a heuristic algorithm CSDW and a meta-heuristic algorithm N-WOA to minimize the execution cost of the given workflow subject to the deadline constraint in clouds. CSDW first assigns the sub-deadline to each task based on the modified probabilistic upward rank, and then tasks are sorted and mapped to appropriate instances, finally instance-type upgrading and downgrading method is adopted to further accelerate workflow execution and reduce the total cost, respectively. N-WOA employs whale optimization algorithm for deadline-constrained cost optimization by refining the task ordering step in CSDW. By simulation experiments on scientific workflows with existing algorithms, the results demonstrate the capability of the proposed algorithms in meeting the deadlines and reducing the execution costs, CSDW is highly competitive and N-WOA achieves the best performance in all cases.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data availability

Data sharing is not applicable as no new datasets were created or analyzed in this study.

References

  1. Gavvala SK, Jatoth C, Gangadharan GR, Buyya R (2019) QoS-aware cloud service composition using eagle strategy. Futur Gener Comput Syst 90:273–290

    Article  Google Scholar 

  2. Van Der Aalst W, Van Hee KM, Van Hee K (2004) Workflow Management: models, methods, and systems. MIT Press

  3. Pinedo M, Hadavi K, Gaul W, Bachem A, Habenicht W, Runge W, Stahl WW (eds). Scheduling: theory, algorithms and systems development. In: Gaul W, Bachem A, Habenicht W, Runge W, Stahl WW (eds). Springer, pp 35–42

  4. Adhikari M, Amgoth T, Srirama SN (2019) A survey on scheduling strategies for workflows in cloud environment and emerging trends. ACM Comput Surv 52(4):68:1-68:36

    Google Scholar 

  5. Abrishami S, Naghibzadeh M, Epema DHJ (2013) Deadline-constrained workflow scheduling algorithms for infrastructure as a service clouds. Future Gener Comput Syst 29(1):158–169

    Article  Google Scholar 

  6. Wu Q, Ishikawa F, Zhu Q, Xia Y, Wen J (2017) Deadline-constrained cost optimization approaches for workflow scheduling in clouds. IEEE Trans Parallel Distrib Syst 28(12):3401–3412

    Article  Google Scholar 

  7. Rodriguez MA, Buyya R (2014) Deadline based resource provisioningand scheduling algorithm for scientific workflows on clouds. IEEE Trans Cloud Comput 2(2):222–235

    Article  Google Scholar 

  8. Arabnejad V, Bubendorfer K, Ng B (2017) Scheduling deadline constrained scientific workflows on dynamically provisioned cloud resources. Future Gener Comput Syst 75:348–364

    Article  Google Scholar 

  9. Zhu Z, Tang X (2019) Deadline-constrained workflow scheduling in IaaS clouds with multi-resource packing. Future Gener Comput Syst 101:880–893

    Article  Google Scholar 

  10. Juve G et al (2013) Characterizing and profiling scientific workflows. Future Gener Comput Syst 29(3):682–692

    Article  Google Scholar 

  11. Rodriguez MA, Buyya RA (2017) A Taxonomy and survey on scheduling algorithms for scientific workflows in IaaS cloud computing environments. Concurr Comput Pract Exp 29(8):e4041

    Article  Google Scholar 

  12. Faragardi HR, Sedghpour MRS, Fazliahmadi S, Fahringer T, Rasouli N (2020) GRP-HEFT: a budget-constrained resource provisioning scheme for workflow scheduling in IaaS clouds. IEEE Trans Parallel Distrib Syst 31(6):1239–1254

    Article  Google Scholar 

  13. Abrishami S, Naghibzadeh M, Epema DHJ (2012) Cost-driven scheduling of grid workflows using partial critical paths. IEEE Trans Parallel Distrib Syst 23(8):1400–1414

    Article  Google Scholar 

  14. Genez TAL, Bittencourt LF, Madeira ERM (2020) Time-discretization for speeding-up scheduling of deadline-constrained workflows in clouds. Future Gener Comput Syst 107:1116–1129

    Article  Google Scholar 

  15. Tang X et al (2022) Cost-efficient workflow scheduling algorithm for applications with deadline constraint on heterogeneous clouds. IEEE Trans Parallel Distrib. Syst 33(9):2079–2092

    Article  Google Scholar 

  16. Li H, Wang Y, Huang J, Fan Y (2022) Mutation and dynamic objective-based farmland fertility algorithm for workflow scheduling in the cloud. J Parallel Distrib. Comput 164:69–82

    Article  Google Scholar 

  17. Hemasian-Etefagh F, Safi-Esfahani F (2019) Dynamic scheduling applying new population grouping of whales meta-heuristic in cloud computing. J Supercomput 75(10):6386–6450

    Article  Google Scholar 

  18. Iranmanesh A, Naji HR (2021) DCHG-TS: a deadline-constrained and cost-effective hybrid genetic algorithm for scientific workflow scheduling in cloud computing. Clust Comput 24(2):667–681

    Article  Google Scholar 

  19. Nasr AA, El-Bahnasawy NA, Attiya G, El-Sayed A (2019) Cost-effective algorithm for workflow scheduling in cloud computing under deadline constraint. Arab J Sci Eng 44(4):3765–3780

    Article  Google Scholar 

  20. Chakravarthi KK, Shyamala L, Vaidehi V (2021) Cost-effective workflow scheduling approach on cloud under deadline constraint using firefly algorithm. Appl Intell 51(3):1629–1644

    Article  Google Scholar 

  21. Zheng W, Sakellariou R (2013) Budget-deadline constrained workflow planning for admission control. J Grid Comput 11(4):633–651

    Article  Google Scholar 

  22. Topcuoglu H, Hariri S, Wu M (2002) Performance-effective and low-complexity task scheduling for heterogeneous computing. IEEE Trans Parallel Distributed Syst 13(3):260–274

    Article  Google Scholar 

  23. Arabnejad H, Barbosa JG (2014) A budget constrained scheduling algorithm for workflow applications. J Grid Comput 12(4):665–679

    Article  Google Scholar 

  24. Wu F, Wu Q, Tan Y, Li R, Wang W (2016) PCP-B\({}^{\text{2 }}\): partial critical path budget balanced scheduling algorithms for scientific workflow applications. Future Gener Comput Syst 60:22–34

    Article  Google Scholar 

  25. Durillo JJ, Prodan R (2014) Multi-objective workflow scheduling in amazon EC2. Clust Comput 17(2):169–189

    Article  Google Scholar 

  26. Chen Z et al (2019) Multiobjective cloud workflow scheduling: a multiple populations ant colony system approach. IEEE Trans Cybern 49(8):2912–2926

    Article  Google Scholar 

  27. Zhu Z, Zhang G, Li M, Liu X (2016) Evolutionary multi-objective workflow scheduling in cloud. IEEE Trans Parallel Distrib Syst 27(5):1344–1357

    Article  Google Scholar 

  28. Paknejad P, Khorsand R, Ramezanpour M (2021) Chaotic improved PICEA-g-based multi-objective optimization for workflow scheduling in cloud environment. Future Gener Comput Syst 117:12–28

    Article  Google Scholar 

  29. Choudhary A, Gupta I, Singh V, Jana PK (2018) A GSA based hybrid algorithm for bi-objective workflow scheduling in cloud computing. Future Gener Comput Syst 83:14–26

    Article  Google Scholar 

  30. Xia X, Qiu H, Xu X, Zhang Y (2022) Multi-objective workflow scheduling based on genetic algorithm in cloud environment. Inf Sci 606:38–59

    Article  Google Scholar 

  31. Rodriguez MA, Buyya R (2018) Scheduling dynamic workloads in multi-tenant scientific workflow as a service platforms. Future Gener Comput Syst 79:739–750

    Article  Google Scholar 

  32. Saeedizade E, Ashtiani M (2021) DDBWS: a dynamic deadline and budget-aware workflow scheduling algorithm in workflow-as-a-service environments. J Supercomput 77(12):14525–14564

    Article  Google Scholar 

  33. Liu J et al (2021) Online multi-workflow scheduling under uncertain task execution time in IaaS clouds. IEEE Trans Cloud Comput 9(3):1180–1194

    Article  Google Scholar 

  34. Mirjalili S, Lewis A (2016) The whale optimization algorithm. Adv Eng Softw 95:51–67

    Article  Google Scholar 

  35. Cormen TH, Leiserson CE, Rivest RL, Stein C (2009) Introduction to Algorithms, 3rd edn. MIT Press

  36. Magistrale H, Day S, Clayton RW, Graves R (2000) The SCEC southern California reference three-dimensional seismic velocity model version 2. Bull Seismol Soc Am 90(6B):S65–S76

    Article  Google Scholar 

  37. Deelman E et al (2015) Pegasus, a workflow management system for science automation. Future Gener Comput Syst 46:17–35

    Article  Google Scholar 

  38. Brown DA et al. (2007) A case study on the use of workflow technologies for scientific analysis: gravitational wave data analysis. In: Taylor IJ, Deelman E, Gannon DB, Shields MS (eds). Workflows for e-Science, Scientific Workflows for Grids. Springer, pp 39–59

  39. Jain A, Kumari R, Modi N, Verma P,Trivedi B (eds). (2017) A review on comparison of workflow scheduling algorithms with scientific workflows. In: Modi N, Verma P, Trivedi B (eds) Proceedings of International Conference on Communication and Networks. Springer, pp 613–622

  40. Bittencourt LF, Madeira ER (2010) Towards the scheduling of multiple workflows on computational grids. J Grid Comput 8(3):419–441

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (No. 61772200), Natural Science Foundation of Shanghai (No. 21ZR1416300), and Capacity Building Project of Local Universities Science and Technology Commission of Shanghai Municipality (No. 22010504100).

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, East China University of Science and Technology, Shanghai, 200237, China

    Zengpeng Li, Huiqun Yu & Guisheng Fan

  2. Shanghai Key Laboratory of Computer Software Evaluating and Testing, Shanghai, 201112, China

    Huiqun Yu

Authors
  1. Zengpeng Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huiqun Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guisheng Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Conceptualization was done by ZL; methodology was done by ZL; formal analysis and investigation were carried out by ZL; writing—original draft preparation were carried out by ZL; and supervision was done by HY and GF.

Corresponding authors

Correspondence to Huiqun Yu or Guisheng Fan.

Ethics declarations

Conflicts of interest

The authors have no conflict of interest to declare.

Consent to participate

Not applicable.

Consent for publication

All authors have approved to this submission to The Journal of Supercomputing. Its publication is also approved tacitly by the responsible authorities where the work was carried out. We agree that if accepted, the paper will not be published elsewhere in the same form, in English or in any other language, without the written consent of the publisher.

Ethical approval

Not applicable.

Human and animal Rights

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, Z., Yu, H. & Fan, G. Cost-effective approaches for deadline-constrained workflow scheduling in clouds. J Supercomput 79, 7484–7512 (2023). https://doi.org/10.1007/s11227-022-04962-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-022-04962-x

Keywords