Skip to main content
SpringerLink
Log in

DSM: a dynamic scheduling method for concurrent workflows in cloud environment

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

Cloud computing, emerged as a commercial service model, has been widely concerned in both industry and academia. Massive workflow applications could be performed simultaneously on the cloud platforms, which significantly benefits from the elasticity and convenience of cloud computing. However, it is still a challenge to schedule virtualized resources for the concurrent workflows in cloud environment, with limited high-performance resources in a timesaving and efficient manner. In view of this challenge, a dynamic scheduling method for concurrent workflows, named as DSM, in cloud environment is proposed to satisfy the various resource requirements of the workflows. Technically, a time overhead model for the workflows and a resource utilization model for cloud datacenter are presented. Then a relevant dynamic scheduling method is designed based on critical path lookup, which aims at minimizing the makespan of workflows, and maximizing the resource utilization of the datacenter during the execution of the workflows. Extensive experimental evaluations demonstrate the efficiency and effectiveness of our proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Netjinda, N., Sirinaovakul, B., Achalakul, T.: Cost optimal scheduling in IaaS for dependent workload with particle swarm optimization. J. Supercomput. 68(3), 1579–1603 (2014)

    Article  Google Scholar 

  2. Rao, J., Zhou, X.: Towards fair and efficient SMP virtual machine scheduling. In: ACM Sigplan Symposium on Principles and Practice of Parallel Programming. vol. 49(8), pp. 273–286 (2014)

  3. Wang, P., Huang, Y., Li, K., Guo, Y.: Load balancing degree first algorithm on phase space for cloud computing cluster. J. Comput. Res. Dev. 51(5), 1095–1107 (2014)

    Google Scholar 

  4. Armbrust, M., Fox, A., Griffith, R., et al.: Above the Clouds: A Berkeley View of Cloud Computing. Eecs Department University of California Berkeley, vol. 53(4), pp. 50–58 (2009)

  5. Zhang, S., Qian, Z., Wu, J., Lu, S., Epstein, L.: Virtual network embedding with opportunistic resource sharing. IEEE Trans. Parallel Distrib. Syst. 25(3), 816–827 (2014)

    Article  Google Scholar 

  6. Li, W., Zhang, Q., Wu, J., Li, J., Hao, H.: Trust-driven and QoS demand clustering analysis based cloud workflow scheduling strategies. Clust. Comput. 17(3), 1–18 (2014)

    Article  Google Scholar 

  7. Dou, W., Xu, X., Meng, S., Zhang, X., Hu, C., Yu, S., Yang, J.: An energy-aware virtual machine scheduling method for service QoS enhancement in clouds over big data. Concurr. Comput. Pract. Exp. 29(14), 1–20 (2017)

  8. Shen, H., Li, X.: Algorithm for the cloud service workflow scheduling with setup time and deadline constraints. J. Commun. 36(6), 183–192 (2015)

    Google Scholar 

  9. Guo, H., Chen, Z., Yu, Y., Wang, Y., Chen, X.: A communication aware DAG workflow cost optimization model and algorithm. J. Comput. Res. Dev. 52(6), 1400–1408 (2015)

    Google Scholar 

  10. Chen, W., Lee, Y.C., Fekete, A., Zomaya, A.Y.: Adaptive multiple-workflow scheduling with task rearrangement. J. Supercomput. 71(4), 1297–1317 (2015)

    Article  Google Scholar 

  11. Dong, J., WANG, H., Cheng, S.: Energy-performance tradeoffs in laaS cloud with virtual machine scheduling. China Commun. 12(2), 155–166 (2015)

    Article  Google Scholar 

  12. Durao, F., Carvalho, J.F.S., Fonseka, A., Garcia, V.C.: A systematic review on cloud computing. J. Supercomput. 68(3), 1321–1346 (2014)

    Article  Google Scholar 

  13. Ahmad, S.G., Liew, C.S., Rafique, M.M., Munir, E.U., Khan, S.U.: Data-intensive workflow optimization based on application task graph partitioning in heterogeneous computing systems. In: IEEE Fourth International Conference on Big Data and Cloud Computing pp. 129–136 (2015)

  14. Chen, C.: Workflow task scheduling in cloud computing based on hybrid improved CS algorithm and decision tree. J. Univ. Electron. Sci. Technol. China 45(6), 974–980 (2016)

    MATH  Google Scholar 

  15. Liu, J.X., Yang, X.F., X. Y.: Cloud workflow scheduling method based on batch processing strategy. Comput. Integr. Manufac. Syst. 21(2), 336–343 (2015)

  16. Luo, Z., Wang, P., You, B., Jie, S.U.: Optimization scheduling of workflow’s accuracy based on reverse reduction under constraint time. J. Beijing Univ. Posts Telecommun. 40(1), 99–104 (2017)

    Google Scholar 

  17. Doulamis, N.D., Kokkino, P., Varvarigos, E.: Resource selection for tasks with time requirements using spectral clustering. IEEE Trans. Comput. 63(2), 461–474 (2014)

    Article  Google Scholar 

  18. Kong, Y., Zhang, M., Ye, D.: A belief propagation-based method for task allocation in open and dynamic cloud environments. Knowl. Based Syst. 115, 123–132 (2016)

    Article  Google Scholar 

  19. Xing, G., Xu, X., Xiang, H., Xue, S., Ji, S., Yang, J.: Fair energy-efficient virtual machine scheduling for internet of things applications in cloud environment. Int. J. Distrib. Sensor Netw. 13(2), 1–11 (2017)

    Article  Google Scholar 

  20. Hao, L., Cui, G., Qu, M., Ke, W.: Resource scheduling optimization algorithm of energy consumption for cloud computing based on task tolerance. J. Softw. 9(4), 895–901 (2014)

    Google Scholar 

  21. Cao, F., Zhu, M.M., Wu, C.Q.: Energy-efficient resource management for scientific workflows in clouds. In: 2014 IEEE World Congress on Services (SERVICES), pp. 402–409 (2014)

  22. Jrad, F., Tao, J., Brandic, I., Streit, A.: SLA enactment for large-scale healthcare workflows on multi-Cloud. Future Gener. Comput. Syst. 43(4), 135–148 (2015)

    Article  Google Scholar 

  23. Wang, Y., Wang, J., Han, Y.: A two-stage resource scheduling method for workflow cloud computing system. J. Sourth China Univ. Technol. 45(1), 80–87 (2017)

    Google Scholar 

  24. Rodriguez, M.A., Buyya, R.: Deadline based resource provisioning and scheduling algorithm for scientific workflows on clouds. IEEE Trans. Cloud Comput. 2(2), 222–235 (2014)

    Article  Google Scholar 

  25. Luo, Z.Y., Wang, P., You, B., Zhu, X.S.: Serial reduction optimization research of complex product workflow’s accuracy under the tine constraint. Adv. Mech. Eng. 8(10), 1–9 (2016)

    Google Scholar 

  26. Ahmed, W., Wu, Y.: Estimation of cloud node acquisition. Tsinghua Sci. Technol. 19(1), 1–12 (2014)

    Article  Google Scholar 

  27. Cao, B., Wang, X., Xiong, L., Fan, J.: Searching method for partical swarm optimization of cloud workflow scheduling with time constraint. Comput. Integr. Manufac. Syst. 22(2), 372–380 (2016)

    Google Scholar 

  28. Xu, Y., Li, K., Hu, J., Li, K.: A genetic algorithm for task scheduling on heterogeneous computing systems using multiple priority queues. Inf. Sci. 270(6), 255–287 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  29. Chen, H., Zhu, J., Manho, M.A., Zhu, X.: Scheduling for stochastic tasks and resources in virtualized clouds. Syst. Eng. Electron. 39(2), 348–354 (2017)

    Google Scholar 

  30. Wu, C.M., Chang, R.S., Chan, H.Y.: A green energy-efficient scheduling algorithm using the DVFS technique for cloud datacenters. Future Gener. Comput. Syst. 37(7), 141–147 (2014)

    Article  Google Scholar 

  31. Calheiros, R.N., Buyya, R.: Meeting deadlines of scientific work-flows in public clouds with tasks replication. IEEE Trans. Parallel Distrib. Syst. 25(7), 1787–1796 (2014)

    Article  Google Scholar 

  32. Guo, Y.Q., Song, J.X.: Optimal task-level scheduling based on multimedia applications in cloud. Comput. Sci. 42(11), 413–416 (2015)

    MathSciNet  Google Scholar 

  33. Mandal, Vasundhara, D., Kar, R., Ghoshal, S.P.: Digital FIR filter design using fitness based hybrid adaptive differential evolution with particle swarm optimization. Nat. Comput. 13(1), 55–64 (2014)

    Article  MathSciNet  Google Scholar 

  34. Prasad, A.S., Rao, S.: A mechanism design approach to resource procurement in cloud computing. IEEE Trans. Comput. 63(1), 17–30 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  35. Chen, H.K., Zhu, J.H., Zhu, X.M., Ma, M.H., Zhang, Z.S.: Resource-delay-aware scheduling for real-time tasks in clouds. J. Comput. Res. Dev. 54(2), 446–456 (2017)

    Google Scholar 

  36. Xu, X., Dou, W., Zhang, X., Chen, J.: EnReal: an energy-aware resource allocation method for scientific workflow executions in cloud environment. IEEE Trans. Cloud Comput. 4, 166–179 (2016)

    Article  Google Scholar 

  37. Abrishami, S., Naghibzadeh, M., Epema, D.H.J.: Deadline-constrained workflow scheduling algorithms for infrastructure as a service clouds. Future Gener. Comput. Syst. 29(1), 158–169 (2013)

    Article  Google Scholar 

  38. Yang, G., Stolyar, A.L., Walid, A.: Shadow-routing based dynamic algorithms for virtual machine placement in a network cloud. IEEE Infocom 12(11), 620–628 (2013)

    Google Scholar 

  39. Li, X., Wu, J., Tang, S., Lu, S.: Let’s stay together: towards traffic aware virtual machine placement in data centers. In: 2014 IEEE Conference on Computer Communications (INFOCOM), pp. 1842–1850 (2014)

  40. Xiaohu, W., Loiseau, P.: Algorithms for scheduling deadline-sensitive malleable tasks. In: 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton), pp. 530–537 (2015)

Download references

Acknowledgements

This research is supported by the National Science Foundation of China under Grant Nos. 61702277, 61672276, 61402167 and 61672290. Besides, this work is also supported by The Startup Foundation for Introducing Talent of NUIST, the open project from State Key Laboratory for Novel Software Technology, Nanjing University under grant no. KFKT2017B04, the Priority Academic Program Development of Jiangsu Higher Education Institutions (PAPD) fund, Jiangsu Collaborative Innovation Center on Atmospheric Environment and Equipment Technology (CICAEET), the project “Six Talent Peaks Project in Jiangsu Province” under grant no. XYDXXJS-040, and Innovation Platform Open Foundation of Hunan Provincial Education Department (No. 17K033).

Author information

Authors and Affiliations

  1. Jiangsu Engineering Center of Network Monitoring, Nanjing University of Information Science and Technology, Nanjing, 210044, China

    Shengjun Xue, Yue Peng, Xiaolong Xu & Chao Shen

  2. School of Computer and Software, Nanjing University of Information Science and Technology, Nanjing, 210044, China

    Shengjun Xue, Yue Peng & Chao Shen

  3. State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, 210023, China

    Xiaolong Xu & Jie Zhang

  4. School of Information and Control, Nanjing University of Information Science and Technology, Nanjing, 210044, China

    Feng Ruan

Authors
  1. Shengjun Xue
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yue Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaolong Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jie Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Chao Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Feng Ruan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaolong Xu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xue, S., Peng, Y., Xu, X. et al. DSM: a dynamic scheduling method for concurrent workflows in cloud environment. Cluster Comput 22 (Suppl 1), 693–706 (2019). https://doi.org/10.1007/s10586-017-1189-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-017-1189-5

Keywords

Search

Navigation