Skip to main content
Log in

A Novel Approach to Scheduling Workflows Upon Cloud Resources with Fluctuating Performance

Mobile Networks and Applications Aims and scope Submit manuscript

Abstract

Cloud computing is recently getting increasingly popular for supporting scientific applications and complex business processes. Clouds are highly potent for executing workflow-based tasks due to the fact that they provide elastic resource provisioning styles through which computational-intensive workflows can obtain requested resources according to their elastic demand and establish execution environment over virtual machines (VMs). However, it remains a challenge to guarantee cost-effectiveness and quality of service of workflow deployed upon clouds due to the fact that real-world cloud infrastructures are usually with fluctuating and time-varying performance. Existing researches mainly consider that cloud infrastructures are with fixed, random, or bounded quality of service (QoS). In this work, however, we consider that scientific computing processes to be supported by decentralized cloud infrastructures with fluctuating QoS and aim at managing the monetary cost of workflows with the completion-time constraint to be satisfied. We address the performance-variation-aware workflow scheduling problem by leveraging a time-series-based prediction model and a Critical-Path-Duration-Estimation-based (CPDE for short) VM Selection strategy. The proposed method is capable of exploiting real-time trends of performance changes of cloud infrastructures and generating dynamic workflow scheduling plans. To prove the effectiveness of our proposed method, we perform extensive experimental case analysis over real-world third-party commercial clouds and show that our method clearly beats existing approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

References

  1. Xia Y, Zhou M, Luo X, Pang S, Zhu Q (2015) Stochastic modeling and performance analysis of migration-enabled and error-prone clouds. IEEE Transactions on Industrial Informatics 11(2):495–504

    Article  Google Scholar 

  2. He Q, Han J, Yang Y, Jin H, Schneider J, Versteeg S (2014) Formulating cost-effective monitoring strategies for service-based systems. IEEE Trans Softw Eng 40(5):461–482

    Article  Google Scholar 

  3. Yao Y, Cao J, Jiang Y, Wang J (2016) An optimal engine component placement strategy for cloud workflow service. In: IEEE international conference on web services, pp 380–387

  4. He Q, Xie X, Chen F et al (2015) Spectrum-based runtime anomaly localisation in service-based systems. In: IEEE international conference on services computing, pp 90–97

  5. Wu Q, Ishikawa F, Zhu Q, Xia Y, Wen J (2017) Deadline-constrained cost optimization approaches for workflow scheduling in clouds. IEEE Transactions on Parallel and Distributed Systems 28(12):3401–3412

    Article  Google Scholar 

  6. He Q, Xie X, Wang Y et al (2017) Localizing runtime anomalies in service-oriented systems. IEEE Transactions on Services Computing 10(1):94–106

    Article  Google Scholar 

  7. Xia Y, Zhou M, Luo X, Pang S, Zhu Q (2015) A stochastic approach to analysis of energy-aware DVS-enabled cloud datacenters. Systems Man and Cybernetics 45(1):73–83

    Google Scholar 

  8. Hwang S, Hsu C, Lee C (2015) Service selection for web services with probabilistic QoS. IEEE Trans Serv Comput 8(3):467–480

    Article  Google Scholar 

  9. Xia Y, Zhou M, Luo X, Zhu Q, Li J, Huang Y (2015) Stochastic modeling and quality evaluation of infrastructure-as-a-service clouds. IEEE Trans Autom Sci Eng 12(1):162–170

    Article  Google Scholar 

  10. Hadad JE, Manouvrier M, Rukoz M (2010) TQOs: transactional and QoS-aware selection algorithm for automatic web service composition. IEEE Trans Serv Comput 3(1):73–85

    Article  Google Scholar 

  11. Wang Y, Liu H, Zheng W et al (2019) Multi-objective workflow scheduling with Deep-Q-Network-Based multi-agent reinforcement learning. IEEE Access, pp 39974–39982

  12. Li W, Liao K, He Q, Xia Y (2019) Performance-aware cost-effective resource provisioning for future grid IoT-cloud system. Journal of Energy Engineering-asce 145(5)

  13. Yin Y, Xia J, Li Y, Xu Y, Xu W, Yu L (2019) Group-wise itinerary planning in temporary mobile social network. IEEE Access, pp 83682–83693

  14. Yin Y, Chen L, Xu Y, Wan J, Zhang H, Mai Z (2019) QoS prediction for service recommendation with deep feature learning in edge computing environment. Mobile Netw Appl. https://doi.org/10.1007/s11036-019-01241-7

  15. Yin Y, Zhang W, Xu Y, Zhang H, Mai Z, Yu L (2019) Qos prediction for mobile edge service recommendation with auto-encoder. IEEE Access, pp 62312–62324

  16. Gao H, Huang W, Duan Y, Yang X, Zou Q (2019) Research on cost-driven services composition in an uncertain environment. J Internet Technol 20(3):755–769

    Google Scholar 

  17. Gao H, Huang W, Yang X (2019) Applying probabilistic model checking to path planning in an intelligent transportation system using mobility trajectories and their statistical data. Intell Autom Soft Comput 25(3):547–559

    Google Scholar 

  18. Gao H, Miao H, Liu L, Kai J, Zhao K (2018) Automated quantitative verification for service-based system design: a visualization transform tool perspective. Int J Softw Eng Knowl Eng 28(10):1369–1397

    Article  Google Scholar 

  19. Schad J, Dittrich J, Quianeruiz J (2010) Runtime measurements in the cloud: observing, analyzing, and reducing variance. Very Large Data Bases 3(1):460–471

    Google Scholar 

  20. Jackson K, Ramakrishnan L, Muriki K et al (2010) Performance analysis of high performance computing applications on the amazon web services cloud. In: IEEE international conference on cloud computing technology and science, pp 159–168

  21. Wu Q, Zhu Q, Jian X, Ishikawa F (2014) Broker-based SLA-aware composite service provisioning. J Syst Softw, pp 194–201

  22. Ibarra OH, Kim CE (1977) Heuristic algorithms for scheduling independent tasks on nonidentical processors. J ACM 24(2):280–289

    Article  MathSciNet  Google Scholar 

  23. Casas I, Taheri J, Ranjan R, Wang L, Zomaya AY (2016) GA-ETI: an enhanced genetic algorithm for the scheduling of scientific workflows in cloud environments. J of Comput Sci, pp 318–331

  24. Verma A, Kaushal S (2017) A hybrid multi-objective Particle Swarm Optimization for scientific workflow scheduling. Parall Comput, pp 1–19

  25. Zhou X, Zhang G, Sun J, Zhou J, Wei T, Hu S (2019) Minimizing cost and makespan for workflow scheduling in cloud using fuzzy dominance sort based HEFT. Futur Gener Comput Syst, pp 278–289

  26. Wang Y, Jiang J, Xia Y, Wu Q, Luo X, Zhu Q (2018) A multi-stage dynamic game-theoretic approach for multi-workflow scheduling on heterogeneous virtual machines from multiple infrastructure-as-a-service clouds. In: International conference on services computing, pp 137–152

  27. Sahni J, Vidyarthi DP (2018) A cost-effective deadline-constrained dynamic scheduling algorithm for scientific workflows in a cloud environment. IEEE Trans Cloud Comput 6(1):2–18

    Article  Google Scholar 

  28. Mao M, Humphrey M (2011) Auto-scaling to minimize cost and meet application deadlines in cloud workflows. In: IEEE international conference on high performance computing data and analytics

  29. Calheiros RN, Buyya R (2014) Meeting deadlines of scientific workflows in public clouds with tasks replication. IEEE Trans Parall Distr Sys 25(7):1787–1796

    Article  Google Scholar 

  30. Poola D, Garg SK, Buyya R, Yang Y, Ramamohanarao K (2014) Robust scheduling of scientific workflows with deadline and budget constraints in clouds. In: Advanced information networking and applications, pp 858–865

  31. Ghosh R, Longo F, Frattini F, Russo S, Trivedi KS (2014) Scalable analytics for IaaS cloud availability. IEEE Int Conf Cloud Comput Technol Sci 2(1):57–70

    Google Scholar 

  32. Yin X, Ma X, Trivedi KS (2013) An interacting stochastic models approach for the performance evaluation of DSRC vehicular safety communication. IEEE Trans Comput 62(5):873–885

    Article  MathSciNet  Google Scholar 

  33. Zheng W, Zhou M, Wu L, et al. (2017) Percentile performance estimation of unreliable IaaS clouds and their cost-optimal capacity decision. IEEE Access, pp 2808–2818

  34. Li W, Xia Y, Zhou M, Sun X, Zhu Q (2018) Fluctuation-aware and predictive workflow scheduling in cost-effective infrastructure-as-a-service clouds. IEEE Access, pp 61488–61502

  35. Ma H, Zhu H, Hu Z, Tang W, Dong P (2017) Multi-valued collaborative QoS prediction for cloud service via time series analysis. Futur Gener Comput Syst 68(68):275–288

    Article  Google Scholar 

  36. Coello CA, Pulido GT, Lechuga MS (2004) Handling multiple objectives with particle swarm optimization. IEEE Trans Evol Comput 8(3):256–279

    Article  Google Scholar 

  37. Deb K, Pratap A, Agarwal S, Meyarivan T (2002) A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans Evol Comput 6(2):182–197

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Lei Wu.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pan, Y., Wang, S., Wu, L. et al. A Novel Approach to Scheduling Workflows Upon Cloud Resources with Fluctuating Performance. Mobile Netw Appl 25, 690–700 (2020). https://doi.org/10.1007/s11036-019-01450-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11036-019-01450-0

Keywords

Navigation