Skip to main content
SpringerLink
Log in

Entropy based swarm intelligent searching for scheduling deadline constrained workflows in hybrid cloud

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

In this paper, we consider the problem of scheduling tasks of workflows to heterogeneous VMs (virtual machine) in a hybrid cloud. Workflow scheduling in a hybrid cloud is complex due to factors such as varying electricity prices, uncertain resource requirements, and workflow-specific budget and deadline constraints. To address these challenges, we propose a novel intelligent searching algorithm framework, SEPSO (Swarm Entropy based Particle Swarm Optimization). SEPSO enhances the scheduling process by introducing a swarm entropy, a measure that takes into account the diversification of each iteration. Additionally, we develop rules for sorting workflows and tasks that consider the budget and deadline constraints of each workflow. An iteration-varying flight parameter mechanism is also introduced to balance intensification and diversification during the search process. The components and parameters of our proposed algorithm were statistically calibrated using a comprehensive set of random instances. We then compared SEPSO to adapted existing algorithms for similar problems. Our experimental results demonstrate that SEPSO is effective for the considered problem, offering a solution to workflow scheduling that is more time efficient than traditional particle swarm optimization (PSO) algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Data availability

The data that support the findings of this study are openly available in https://pan.seu.edu.cn:443/link/9952035B8F62087E31E955EDB2F0CD7D.

Notes

  1. Code and Data available at https://cse.seu.edu.cn/2023/0708/c30019a451020/page.htm.

References

  1. Zuo X, Zhang G, Tan W (2014) Self-adaptive learning PSO-based deadline constrained task scheduling for hybrid IAAS cloud. IEEE Trans Autom Sci Eng 11(2):564–573

    Article  Google Scholar 

  2. Yuan H, Bi J, Tan W (2017) Temporal task scheduling with constrained service delay for profit maximization in hybrid clouds. IEEE Trans Autom Sci Eng 14(1):337–348

    Article  Google Scholar 

  3. Liu Q, Zeng L, Bilal M, Song H, Liu X, Zhang Y, Cao X (2023) A multi-swarm PSO approach to large-scale task scheduling in a sustainable supply chain datacenter. IEEE Trans Green Commun Netw. https://doi.org/10.1109/TGCN.2023.3283509

    Article  Google Scholar 

  4. Chen X, Zhang J, Lin B, Chen Z, Wolter K, Min G (2022) Energy-efficient offloading for DNN-based smart IoT systems in cloud-edge environments. IEEE Trans Parallel Distrib Syst 33(3):683–697

    Article  Google Scholar 

  5. Zhao B, Liu X, Song A, Chen W-N, Lai K-K, Zhang J, Deng RH (2022) Primpso: a privacy-preserving multiagent particle swarm optimization algorithm. IEEE Trans Cybern. https://doi.org/10.1109/TCYB.2022.3224169

    Article  PubMed  Google Scholar 

  6. Yang L, Xia Y, Ye L, Gao R, Zhan Y (2023) A fully hybrid algorithm for deadline constrained workflow scheduling in clouds. IEEE Trans Cloud Comput. https://doi.org/10.1109/TCC.2023.3269144

    Article  Google Scholar 

  7. Wang Z-J, Zhan Z-H, Yu W-J, Lin Y, Zhang J, Gu T-L, Zhang J (2020) Dynamic group learning distributed particle swarm optimization for large-scale optimization and its application in cloud workflow scheduling. IEEE Trans Cybern 50(6):2715–2729

    Article  PubMed  Google Scholar 

  8. Chopra N, Singh S (2013) Heft based workflow scheduling algorithm for cost optimization within deadline in hybrid clouds. In: 2013 Fourth International Conference on Computing, Communications and Networking Technologies (ICCCNT), pp 1–6

  9. Bittencourt LF (2011) Hcoc: a cost optimization algorithm for workflow scheduling in hybrid clouds. J Internet Serv Appl 2(3):207–227

    Article  MathSciNet  Google Scholar 

  10. Li C, Tang J, Luo Y (2018) Towards operational cost minimization for cloud bursting with deadline constraints in hybrid clouds. Clust Comput 21(4):2013–2029

    Article  Google Scholar 

  11. Li C, Tang J, Luo Y (2019) Cost-aware scheduling for ensuring software performance and reliability under heterogeneous workloads of hybrid cloud. Autom Softw Eng 26(1):125–159

    Article  Google Scholar 

  12. Bosmans S, Maricaux G, Van Der Schueren F, Hellinckx P (2019) Cost-aware hybrid cloud scheduling of parameter sweep calculations using predictive algorithms. Int J Grid Util Comput 10(1):63–75

    Article  Google Scholar 

  13. Zhu J, Li X, Ruiz R, Xu X (2018) Scheduling stochastic multi-stage jobs to elastic hybrid cloud resources. IEEE Trans Parallel Distrib Syst 29(6):1401–1415

    Article  Google Scholar 

  14. Leena V, Ajeena Beegom AS, Rajasree M (2016) Genetic algorithm based bi-objective task scheduling in hybrid cloud platform. Int J Comput Theory Eng 8(1):7–13

    Article  Google Scholar 

  15. Jing M, Li K, Ouyang A, Li K (2015) A profit maximization scheme with guaranteed quality of service in cloud computing. IEEE Trans Comput 64(11):3064–3078

    Article  MathSciNet  Google Scholar 

  16. Ghamkhari M, Mohsenian-Rad H (2013) Energy and performance management of green data centers: a profit maximization approach. IEEE Trans Smart Grid 4(2):1017–1025

    Article  Google Scholar 

  17. Lin M, Wierman A, Andrew LLH, Thereska E (2013) Dynamic right-sizing for power-proportional data centers. IEEE/ACM Trans Netw 21(5):1378–1391

    Article  Google Scholar 

  18. Sharma NK, Reddy GRM (2016) Multi-objective energy efficient virtual machines allocation at the cloud data center. IEEE Trans Serv Comput 12(1):158–171

    Article  Google Scholar 

  19. Cheng D, Zhou X, Lama P, Ji M, Jiang C (2018) Energy efficiency aware task assignment with dvfs in heterogeneous hadoop clusters. IEEE Trans Parallel Distrib Syst 29(1):70–82

    Article  Google Scholar 

  20. Stavrinides GL, Karatza HD (2018) Energy-aware scheduling of real-time workflow applications in clouds utilizing dvfs and approximate computations. In: 2018 IEEE 6th International Conference on Future Internet of Things and Cloud (FiCloud). IEEE, pp 33–40

  21. Sajid M, Raza Z (2016) Energy-aware stochastic scheduling model with precedence constraints on dvfs-enabled processors. Turk J Electr Eng Comput Sci 24(5):4117–4128

    Article  Google Scholar 

  22. Xu Z, Liang W, Xia Q (2015) Electricity cost minimization in distributed clouds by exploring heterogeneity of cloud resources and user demands. In: 2015 IEEE 21st International Conference on Parallel and Distributed Systems (ICPADS), pp 388–395

  23. Wei Z, Wen Y, Lai LL, Fang L, Rui F, Wei Z, Wen Y, Lai LL, Fang L, Rui F (2017) Electricity cost minimization for interruptible workload in datacenter servers. IEEE Trans Serv Comput 13(99):1–1

    CAS  Google Scholar 

  24. Guo Y, Fang Y (2013) Electricity cost saving strategy in data centers by using energy storage. IEEE Trans Parallel Distrib Syst 24(6):1149–1160

    Article  Google Scholar 

  25. Huang J, Liu Y, Li R, Li K, An J, Bai Y, Yang F, Xie G (2019) Optimal power allocation and load balancing for non-dedicated heterogeneous distributed embedded computing systems. J Parallel Distrib Comput 130:24–36

    Article  Google Scholar 

  26. Wu F, Wu Q, Tan Y (2015) Workflow scheduling in cloud: a survey. J Supercomput 71(9):3373–3418

    Article  Google Scholar 

  27. Chen L, Guo Y, Li X, Ruiz R (2016) Hybrid resource provisioning for workflow scheduling in cloud computing. In: International Conference on Human Centered Computing, pp. 34–46

  28. Abrishami S, Naghibzadeh M, Epema HJD (2013) Deadline-constrained workflow scheduling algorithms for infrastructure; as a service clouds. Future Gener Comput Syst 29(1):158–169

    Article  Google Scholar 

  29. Xie G, Jiang J, Liu Y, Li R, Li K (2017) Minimizing energy consumption of real-time parallel applications using downward and upward approaches on heterogeneous systems. IEEE Trans Industr Inf 13(3):1068–1078

    Article  Google Scholar 

  30. Topcuoglu H, Hariri S, Wu MY (2002) Performance-effective and low-complexity task scheduling for heterogeneous computing. IEEE Trans Parallel Distrib Syst 13(3):260–274

    Article  Google Scholar 

  31. Zhan Z-H, Xiao J, Zhang J, Chen W-n (2007) Adaptive control of acceleration coefficients for particle swarm optimization based on clustering analysis. In: 2007 IEEE Congress on Evolutionary Computation, pp. 3276–3282

  32. Shi Y, et al. (2001) Particle swarm optimization: developments, applications and resources. In: Proceedings of the 2001 congress on evolutionary computation (IEEE Cat. No. 01TH8546), vol. 1, pp 81–86

  33. Zhan Z, Zhang J (2009) Adaptive particle swarm optimization. IEEE Trans Syst Man Cybern Part B Cybern 39(6):1362–1381

    Article  Google Scholar 

  34. Chen W, Deelman E (2012) Workflowsim: a toolkit for simulating scientific workflows in distributed environments. In: 2012 IEEE 8th International Conference on E-Science, pp 1–8

  35. Calheiros RN, Ranjan R, Beloglazov A, De Rose CA, Buyya R (2011) Cloudsim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms. Softw Pract Exp 41(1):23–50

    Article  Google Scholar 

  36. Deelman E, Singh G, Su M-H, Blythe J, Gil Y, Kesselman C, Mehta G, Vahi K, Berriman GB, Good J et al (2005) Pegasus: a framework for mapping complex scientific workflows onto distributed systems. Sci Program 13(3):219–237

    Google Scholar 

  37. Doctor S, Venayagamoorthy GK, Gudise VG (2004) Optimal pso for collective robotic search applications. In: Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No. 04TH8753), vol. 2, pp 1390–1395

Download references

Acknowledgements

This work is supported by the National Key Research and Development Program of China (No. 2022YFB3305500), the National Natural Science Foundation of China (Nos. 62273089, 61832004) and Collaborative Innovation Center of Wireless Communications Technology.

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Key Laboratory of Computer Network and Information Integration, Ministry of Education, Southeast University, Nanjing, 211189, China

    He Li, Xiaoping Li, Jingwen Xu & Long Chen

Authors
  1. He Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaoping Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jingwen Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Long Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaoping Li.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, H., Li, X., Xu, J. et al. Entropy based swarm intelligent searching for scheduling deadline constrained workflows in hybrid cloud. Int. J. Mach. Learn. & Cyber. 15, 1183–1199 (2024). https://doi.org/10.1007/s13042-023-01962-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-023-01962-y

Keywords

Search

Navigation