Skip to main content
SpringerLink
Log in

Reliable budget aware workflow scheduling strategy on multi-cloud environment

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

The resource provisioning and workflow execution in a multi-cloud environment using a pay-as-you-use framework have recently gained the attention of the cloud computing research community. Scheduling of workflows in the multi-cloud platform is challenging due to the cloud dynamics, particularly, heterogeneous resource types, multiple billing mechanisms, elasticity, on-demand provisioning, and systems reliability. In addition, these workflow applications have a runtime constraint—the most typical being the execution time and the execution cost. Another vital Quality of Service (QoS) metric that is of critical concern is reliability. This paper proposes a Normalization based Reliable Budget constraint Workflow Scheduling (NRBWS) algorithm to improve the workflow execution reliability and reduce the makespan under the budget constraint specified by the user. This scheme undergoes a min–max normalization process that is trailed by the computation of the expect reasonable budget (\(erb\)) to assign the tasks to one of the computational resources. The NRBWS algorithm lowers the makespan by assigning each workflow task to the most reliable computing resource with the earliest finish time under the allocated budget. Simulation results demonstrate that the proposed NRBWS algorithm outperforms existing state-of-the-art heuristics.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

Provided on Request.

Code availability

Provided on Request.

References

  1. Sahni, J., Vidyarthi, D.: A cost-effective deadline-constrained dynamic scheduling algorithm for scientific workflows in a cloud environment. IEEE Trans. Cloud Comp. 6(1), 2–18 (2018)

    Article  Google Scholar 

  2. He, J., Ota, K., Dong, M., Yang, L.T., Fan, M., Wang, G., Yau, S.S.: Customized network security for cloud service. IEEETrans. Serv. Comput. 13(5), 801–814 (2020)

    Article  Google Scholar 

  3. Zhang, Z., Dong, M., Zhu, L., Guan, Z., Chen, R., Xu, R., Ota, K.: Achieving privacy-friendly storage and secure statistics for smart meter data on outsourced clouds. IEEE Trans. Cloud Comput. 7(3), 638–649 (2019)

    Article  Google Scholar 

  4. Farid, M., Latip, R., Hussin, M., Hamid, N.: Scheduling scientific workflow using multi-objective algorithm with fuzzy resource utilization in multi-cloud environment. IEEE Access 8, 24309–24322 (2020)

    Article  Google Scholar 

  5. Zhou, X., Zhang, G., Sun, J., Zhou, J., Wei, T., Hu, S.: ”Minimizing cost and makespan for workflow scheduling in cloud using fuzzy dominance sort based HEFT. Future Gen. Comput. Syst. 93, 278–289 (2019)

    Article  Google Scholar 

  6. Ramesh, K., Renjith, P.N., Sasikumar, S.: Optimizing the role of orchestrator for integration Aneka PaaS with AWS cloud. In: The 2020 International Conference on Inventive Computation Technologies (ICICT), pp. 613–614, (2020)

  7. Kang, S., Veeravalli, B., Aung, K.: Dynamic scheduling strategy with efficient node availability prediction for handling divisible loads in multi-cloud systems. J. Parallel Distrib. Comput. 113, 1–16 (2018)

    Article  Google Scholar 

  8. Ardagna, D., Ciavotta, M., Passacantando, M.: Generalized nash equilibria for the service provisioning problem in multi-cloud systems. IEEE Trans. Serv. Comput. 10(3), 381–395 (2017)

    Article  Google Scholar 

  9. Li, R., Zheng, Q., Li, X., Yan, Z.: Multi-objective optimization for rebalancing virtual machine placement. Future Gen. Comput. Syst. 105, 824–842 (2020)

    Article  Google Scholar 

  10. Han, L., Canon, L., Casanova, H., Robert, Y., Vivien, F.: Checkpointing workflows for fail-stop errors. IEEE Trans. Comput. 67(8), 1105–1120 (2018)

    MathSciNet  MATH  Google Scholar 

  11. Wen, Z., Cala, J., Watson, P., Romanovsky, A.: Cost effective, reliable and secureworkflow deployment over federated clouds. IEEE Trans Serv. Comput. 10(6), 929–941 (2017)

    Article  Google Scholar 

  12. Di, S., Robert, Y., Vivien, F., Cappello, F.: Toward an optimal online checkpoint solution under a two-level HPC checkpoint model. IEEE Trans. Parallel Distrib. Syst. 28(1), 244–259 (2017)

    Article  Google Scholar 

  13. Zhou, A., Wang, S., Cheng, B., Zheng, Z., Yang, F., Chang, R.N., Lyu, M.R., Buyya, R.: Cloud service reliability enhancement via virtual machine placement optimization. IEEE Trans. Serv. Comput. 10(6), 902–913 (2017)

    Article  Google Scholar 

  14. Han, H., Bao, W., Zhu, X., Feng, X., Zhou, W.: Fault-tolerant scheduling for hybrid real-time tasks based on CPB model in cloud. IEEE Access 6, 18616–18629 (2018)

    Article  Google Scholar 

  15. Tang, X.: Reliability-aware cost-efficient scientific workflows scheduling strategy on multi-cloud systems. IEEE Trans. Cloud Comput. doi: https://doi.org/10.1109/TCC.2021.3057422.

  16. Lin, b., Guo, W., Chen, G., Xiong, N., Li, R.: Cost-driven scheduling for deadline-constrained work_ow on multi-clouds. In: IEEE International Parallel and Distributed Processing Symposium Workshops, pp. 1191–1198 (2015)

  17. I. Gupta, M. S. Kumar, and P. K. Jana, ``Compute-intensive workflow scheduling in multi-cloud environment,'' in Proc. Int. Conf. Adv. Comput.,Commun. Inform. (ICACCI), 2016, pp. 315_321.

  18. Ullman, J.: Np-complete scheduling problems. J. Comput. Syst. Sci. 10(3), 384–393 (1975)

    Article  MathSciNet  Google Scholar 

  19. Rodriguez, M.A., Buyya, R.: Deadline based resource provisioning and scheduling algorithm for scientific workflows on clouds. IEEE Trans. Cloud Comput. 2(2), 222235 (2014)

    Article  Google Scholar 

  20. Li, Z., Ge, J., Hu, H., Song, W., Hu, H., Luo, B.: Cost and energy aware scheduling algorithm for scientific workflows with deadline constraint in clouds. IEEE Trans. Serv. Comput. 11(4), 713–726 (2018)

    Article  Google Scholar 

  21. Zhu, Z., Zhang, G., Li, M., Liu, X.: Evolutionary multi-objective Workflow scheduling in cloud. IEEE Trans. Parallel Distrib. Syst. 27(5), 13441357 (2016)

    Article  Google Scholar 

  22. Jeannot, E., Saule, E., Trystram, D.: Optimizing performance and reliability on heterogeneous parallel systems: approximation algorithms and heuristics. J. Parallel Distrib. Comput. 72(2), 268–280 (2012)

    Article  Google Scholar 

  23. Dogan, A., Ozuguner, F.: Matching and scheduling algorithms for minimizing execution time and failure probability of applications in heterogeneous computing. IEEE Trans. Parallel Distrib. Syst. 13(3), 308–323 (2002)

    Article  Google Scholar 

  24. Tang, X., Li, K., Qiu, M., Sha, E.: A hierarchical reliability-driven scheduling algorithm in grid systems. J. Parallel Distrib. Comput. 72(4), 525–535 (2012)

    Article  Google Scholar 

  25. Al-Maytami, B.A., Fan, P., Hussain, A., Baker, T., Liatsis, P.: A task scheduling algorithm with improved makespan based on prediction of tasks computation time algorithm for cloud computing. IEEE Access 7, 160916–160926 (2019)

    Article  Google Scholar 

  26. Hwang, S., Kesselman, C.: Grid workflow: a flexible failure handling framework for the grid. In: IEEE international symposium on high-performance parallel distributed computing, pp. 126–137 (2004)

  27. Poola, D., Ramamohanarao, K., Buyya, R.: Enhancing reliability of workflow execution using task replication and spot instances. ACMTrans. Auton. Adapt. Syst. 10(4), 1–21 (2016)

    Article  Google Scholar 

  28. Singh, A., Chatterjee, K.: Cloud security issues and challenges: a survey. J. Netw. Comput. Appl. 79, 88–115 (2017)

    Article  Google Scholar 

  29. Chen, W., Deelman, E.: Workflow overhead analysis and optimizations. In: Proceedings of the 6th workshop on workflows in support of large-scale science-WORKS 11. (2011). doi:https://doi.org/10.1145/2110497.2110500

  30. Li, X., Ma, H., Yao, W., Gui, X.: Data-driven and feedback-enhanced trust computing pattern for large-scale multi-cloud collaborative services. IEEE Trans. Serv. Comput. 11(4), 671–684 (2018)

    Article  Google Scholar 

  31. Trihinas, D., Pallis, G., Dikaiakos, M.D.: Monitoring elastically adaptive multi-cloud services. IEEE Trans. Cloud Comput. 6(3), 800–814 (2018)

    Article  Google Scholar 

  32. Diaz-Montes, J., Diaz-Granados, M., Zou, M., Tao, S., Parashar, M.: Supporting s. IEEE Trans. Cloud Computing 6(1), 250–263 (2018)

    Article  Google Scholar 

  33. Wang, L., Yang, Z., Song, X.: SHAMC: a secure and highly available database system in multi-cloud environment. Future Gen. Comput. Syst. 105, 873–883 (2020)

    Article  Google Scholar 

  34. Kazim, M., Liu, L., Zhu, S.: A framework for orchestrating secure and dynamic access of IoT services in multi-cloud environments. IEEE Access 6, 58619–58633 (2018)

    Article  Google Scholar 

  35. Arabnejad, H., Barbosa, J.G.: Maximizing the completion rate of concurrent scientific applications under time and budget constraints. J. Comput. Sci. 23, 120–129 (2017). https://doi.org/10.1016/j.jocs.2016.10.013

    Article  MathSciNet  Google Scholar 

  36. Arabnejad, H., Barbosa, J.G.: Multi-workflow QoS-constrained scheduling for utility computing. In: IEEE 18th international conference on computational science and engineering. (2015) doi:https://doi.org/10.1109/cse.2015.29

  37. Topcuoglu, H., Hariri, S., Wu, M.: Performance-effective and low-complexity task scheduling for heterogeneous computing. IEEE Trans. Parallel Distrib. Syst. 13(3), 260–274 (2002). https://doi.org/10.1109/71.993206

    Article  Google Scholar 

  38. Arabnejad, H., Barbosa, J.G.: Multi-QoS constrained and Profit-aware scheduling approach for concurrent workflows on heterogeneous systems. Futur. Gener. Comput. Syst. 68, 211–221 (2017). https://doi.org/10.1016/j.future.2016.10.003

    Article  Google Scholar 

  39. Xie, G., Liu, L., Yang, L., Li, R.: Scheduling trade-off of dynamic multiple parallel workflows on heterogeneous distributed computing systems. Concurr. Comput. (2016). https://doi.org/10.1002/cpe.3782

    Article  Google Scholar 

  40. Rimal, B.P., Maier, M.: Workflow scheduling in multi-tenant cloud computing environments. IEEE Trans. Parallel Distrib. Syst. 28(1), 290–304 (2017). https://doi.org/10.1109/tpds.2016.2556668

    Article  Google Scholar 

  41. Ghasemzadeh, M., Arabnejad, H., Barbosa, J.G.: Deadline-budget constrained scheduling algorithm for scientific workflows in a cloud environment. In: Proceedings of the 20th international conference on principles of distributed systems, Vol. 70. 19:1–19. (2017) doi: https://doi.org/10.4230/LIPIcs.OPODIS.2016.19

  42. Zhou, J., Wang, T., Cong, P., Lu, P., Wei, T., Chen, M.: Cost and makespan-aware workflow scheduling in hybrid clouds. J. Syst. Architect. 100, 101631 (2019). https://doi.org/10.1016/j.sysarc.2019.08.004

    Article  Google Scholar 

  43. Zhou, N., Li, F., Xu, K., Qi, D.: Concurrent workflow budget- and deadline-constrained scheduling in heterogeneous distributed environments. Soft. Comput. 22(23), 7705–7718 (2018). https://doi.org/10.1007/s00500-018-3229-3

    Article  Google Scholar 

  44. Wylie, A., Shi, W., Corriveau, J., Wang, Y.: A scheduling algorithm for hadoop mapreduce workflows with budget constraints in the heterogeneous cloud. In: IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW). (2016) doi:https://doi.org/10.1109/ipdpsw.2016.30

  45. Wu, C.Q., Cao, H.: Optimizing the performance of big data workflows in multi-cloud environments under budget constraint. In: IEEE International Conference on Services Computing (SCC). (2016) doi:https://doi.org/10.1109/scc.2016.25

  46. Wu, C.Q., Lin, X., Yu, D., Xu, W., Li, L.: End-to-end delay minimization for scientific workflows in clouds under budget constraint. IEEE Trans. Cloud Comput. 3(2), 169–181 (2015). https://doi.org/10.1109/tcc.2014.2358220

    Article  Google Scholar 

  47. Zheng, W., Sakellariou, R.: Budget-deadline constrained workflow planning for admission control. J. Grid Comput. 1(4), 633–651 (2013). https://doi.org/10.1007/s10723-013-9257-4

    Article  Google Scholar 

  48. Verma, A., Kaushal, S.: Cost-time efficient scheduling plan for executing workflows in the cloud. J. Grid Comput. 13(4), 495–506 (2015). https://doi.org/10.1007/s10723-015-9344-9

    Article  MathSciNet  Google Scholar 

  49. Li, J., Qiu, M., Ming, Z., Quan, G., Qin, X., Gu, Z.: Online optimization for scheduling preemptable tasks on IaaS cloud systems. J. Parallel Distrib. Comput. 72(5), 666–677 (2012). https://doi.org/10.1016/j.jpdc.2012.02.002

    Article  Google Scholar 

  50. Panda, S.K., Jana, P.K.: Efficient task scheduling algorithms for heterogeneous multi-cloud environment. J. Supercomput. 71(4), 1505–1533 (2015). https://doi.org/10.1007/s11227-014-1376-6

    Article  Google Scholar 

  51. Zhou, X., Zhang, G., Sun, J., Zhou, J., Wei, T., Hu, S.: Minimizing cost and makespan for workflow scheduling in cloud using fuzzy dominance sort based HEFT. Futur. Gener. Comput. Syst. 93, 278–289 (2019). https://doi.org/10.1016/j.future.2018.10.046

    Article  Google Scholar 

  52. Chakravarthi, K.K., Shyamala, L., Vaidehi, V.: Budget aware scheduling algorithm for workflow applications in IaaS clouds. Clust. Comput. (2020). https://doi.org/10.1007/s10586-020-03095-1

    Article  Google Scholar 

  53. Tang, X., Li, K., Liao, G.: An effective reliability-driven technique of allocating tasks on heterogeneous cluster systems. Clust. Comput. 17(4), 1413–1425 (2014)

    Article  Google Scholar 

  54. Kianpisheh, S., Charkari, N.M., Kargahi, M.: Reliabilitydriven scheduling of time/cost-constrained grid workflows. Future Gen. Comput. Syst. 55, 1–16 (2016)

    Article  Google Scholar 

  55. Huang, J., Li, R., Jiao, X., Jiang, Yu., Chang, W.: ”Dynamic DAG scheduling on multiprocessor systems: reliability, energy, and makespan. IEEE Trans. Comput.-Aided Design Integr. Circuits Syst. 38(11), 3336–3347 (2020)

    Article  Google Scholar 

  56. Tian, Y., Tian, J., Li, N.: Cloud reliability and efficiency improvement via failure risk based proactive actions. J. Syst. Softw. 163, Article 110524, (2020)

  57. Zhu, X., Wang, J., Guo, H., Zhu, D., Yang, L.T., Liu, L.: Fault-Tolerant Scheduling for Real-Time Scientific Workflows with Elastic Resource Provisioning in Virtualized Clouds. IEEE Trans. Parallel Distribut Syst 27(12), 3501–3517 (2016)

    Article  Google Scholar 

  58. Abrishami, S., Naghibzadeh, M., Epema, D.H.: Deadline-constrained workflow scheduling algorithms for Infrastructure as a Service Clouds. Futur. Gener. Comput. Syst. 29(1), 158–169 (2013). https://doi.org/10.1016/j.future.2012.05.004

    Article  Google Scholar 

  59. Amazon EC2, http://aws.amazon.com/ec2/. Accessed March 30, 2021.

  60. Microsoft Azure, https://azure.microsoft.com. Accessed March 25, 2021.

  61. Zhang, Y., Chakrabarty, K.: Energy-aware adaptive checkpointing in embedded real-time systems. In: Proceedings of the design, automation & Test in Europe conference, pp. 918–923 (2003)

  62. Zhang, L., Li, K., Li, C., Li, K.: Bi-objective workow scheduling of the energy consumption and reliability in heterogeneous computing systems. Inf. Sci. 379, 241256 (2017)

    Google Scholar 

  63. Fard, H.M., Prodan, R., Fahringer, T.: Multi-objective list scheduling of workow applications in distributed computing infrastructures. J. Parallel Distrib. Comput. 74(3), 21522165 (2014)

    Article  Google Scholar 

  64. Han, J., Kamber, M., Pei, J.: Data mining concepts and techniques. Morgan Kaufmann (2011)

  65. Calheiros, R.N., Ranjan, R., Beloglazov, A., Rose, C.A., Buyya, R.: CloudSim: A toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms. Software 41(1), 23–50 (2010). https://doi.org/10.1002/spe.995

    Article  Google Scholar 

  66. Panda, S.K., Jana, P.K.: Normalization-based task scheduling algorithms for heterogeneous multi-cloud environment. Inf. Syst. Front. 20(2), 373–399 (2016). https://doi.org/10.1007/s10796-016-9683-5

    Article  Google Scholar 

  67. Farid, M., Latip, R., Hussin, Hamid, M., N.A.W. A.: Scheduling scientific workflow using multi-objective algorithm with fuzzy resource utilization in multi-cloud Environment. In: IEEE Access, vol. 8, pp. 24309–24322, 2020, doi: https://doi.org/10.1109/ACCESS.2020.2970475.

  68. Ostermann, S., Iosup, A., Yigitbasi, N., Prodan, R., Fahringer, T., & Epema, D. (2010). A performance analysis of EC2 cloud computing services for scientific computing. cloud computing lecture notes of the institute for computer sciences, social-informatics and telecommunications engineering, pp. 115-131. doi:https://doi.org/10.1007/978-3-642-12636-9_9

  69. Mao, M., Humphrey, M.: A performance study on the vm startup time in the cloud. In: IEEE fifth international conference on cloud computing. (2012) doi:https://doi.org/10.1109/cloud.2012.103

Download references

Funding

Not Applicable.

Author information

Authors and Affiliations

  1. Caterpillar India Engineering Solutions Pvt Limited, Chennai, India

    K. Kalyana Chakravarthi

  2. VNR Vignana Jyothi Institute of Engineering &Technology, Hyderabad, India

    P. Neelakantan

  3. VIT, Chennai, India

    L. Shyamala

  4. Mother Theresa Women’s University, Kodaikanal, India

    V. Vaidehi

Authors
  1. K. Kalyana Chakravarthi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. P. Neelakantan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. L. Shyamala
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. V. Vaidehi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to K. Kalyana Chakravarthi.

Ethics declarations

Conflict of interest

The authors of the paper do have any conflict of interest with any companies or institutions.

Human and animal rights statement

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chakravarthi, K.K., Neelakantan, P., Shyamala, L. et al. Reliable budget aware workflow scheduling strategy on multi-cloud environment. Cluster Comput 25, 1189–1205 (2022). https://doi.org/10.1007/s10586-021-03464-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-021-03464-4

Keywords

Search

Navigation