Abstract
Cloud computing is recently getting increasingly popular for supporting scientific applications and complex business processes. Clouds are highly potent for executing workflow-based tasks due to the fact that they provide elastic resource provisioning styles through which computational-intensive workflows can obtain requested resources according to their elastic demand and establish execution environment over virtual machines (VMs). However, it remains a challenge to guarantee cost-effectiveness and quality of service of workflow deployed upon clouds due to the fact that real-world cloud infrastructures are usually with fluctuating and time-varying performance. Existing researches mainly consider that cloud infrastructures are with fixed, random, or bounded quality of service (QoS). In this work, however, we consider that scientific computing processes to be supported by decentralized cloud infrastructures with fluctuating QoS and aim at managing the monetary cost of workflows with the completion-time constraint to be satisfied. We address the performance-variation-aware workflow scheduling problem by leveraging a time-series-based prediction model and a Critical-Path-Duration-Estimation-based (CPDE for short) VM Selection strategy. The proposed method is capable of exploiting real-time trends of performance changes of cloud infrastructures and generating dynamic workflow scheduling plans. To prove the effectiveness of our proposed method, we perform extensive experimental case analysis over real-world third-party commercial clouds and show that our method clearly beats existing approaches.
References
Xia Y, Zhou M, Luo X, Pang S, Zhu Q (2015) Stochastic modeling and performance analysis of migration-enabled and error-prone clouds. IEEE Transactions on Industrial Informatics 11(2):495–504
He Q, Han J, Yang Y, Jin H, Schneider J, Versteeg S (2014) Formulating cost-effective monitoring strategies for service-based systems. IEEE Trans Softw Eng 40(5):461–482
Yao Y, Cao J, Jiang Y, Wang J (2016) An optimal engine component placement strategy for cloud workflow service. In: IEEE international conference on web services, pp 380–387
He Q, Xie X, Chen F et al (2015) Spectrum-based runtime anomaly localisation in service-based systems. In: IEEE international conference on services computing, pp 90–97
Wu Q, Ishikawa F, Zhu Q, Xia Y, Wen J (2017) Deadline-constrained cost optimization approaches for workflow scheduling in clouds. IEEE Transactions on Parallel and Distributed Systems 28(12):3401–3412
He Q, Xie X, Wang Y et al (2017) Localizing runtime anomalies in service-oriented systems. IEEE Transactions on Services Computing 10(1):94–106
Xia Y, Zhou M, Luo X, Pang S, Zhu Q (2015) A stochastic approach to analysis of energy-aware DVS-enabled cloud datacenters. Systems Man and Cybernetics 45(1):73–83
Hwang S, Hsu C, Lee C (2015) Service selection for web services with probabilistic QoS. IEEE Trans Serv Comput 8(3):467–480
Xia Y, Zhou M, Luo X, Zhu Q, Li J, Huang Y (2015) Stochastic modeling and quality evaluation of infrastructure-as-a-service clouds. IEEE Trans Autom Sci Eng 12(1):162–170
Hadad JE, Manouvrier M, Rukoz M (2010) TQOs: transactional and QoS-aware selection algorithm for automatic web service composition. IEEE Trans Serv Comput 3(1):73–85
Wang Y, Liu H, Zheng W et al (2019) Multi-objective workflow scheduling with Deep-Q-Network-Based multi-agent reinforcement learning. IEEE Access, pp 39974–39982
Li W, Liao K, He Q, Xia Y (2019) Performance-aware cost-effective resource provisioning for future grid IoT-cloud system. Journal of Energy Engineering-asce 145(5)
Yin Y, Xia J, Li Y, Xu Y, Xu W, Yu L (2019) Group-wise itinerary planning in temporary mobile social network. IEEE Access, pp 83682–83693
Yin Y, Chen L, Xu Y, Wan J, Zhang H, Mai Z (2019) QoS prediction for service recommendation with deep feature learning in edge computing environment. Mobile Netw Appl. https://doi.org/10.1007/s11036-019-01241-7
Yin Y, Zhang W, Xu Y, Zhang H, Mai Z, Yu L (2019) Qos prediction for mobile edge service recommendation with auto-encoder. IEEE Access, pp 62312–62324
Gao H, Huang W, Duan Y, Yang X, Zou Q (2019) Research on cost-driven services composition in an uncertain environment. J Internet Technol 20(3):755–769
Gao H, Huang W, Yang X (2019) Applying probabilistic model checking to path planning in an intelligent transportation system using mobility trajectories and their statistical data. Intell Autom Soft Comput 25(3):547–559
Gao H, Miao H, Liu L, Kai J, Zhao K (2018) Automated quantitative verification for service-based system design: a visualization transform tool perspective. Int J Softw Eng Knowl Eng 28(10):1369–1397
Schad J, Dittrich J, Quianeruiz J (2010) Runtime measurements in the cloud: observing, analyzing, and reducing variance. Very Large Data Bases 3(1):460–471
Jackson K, Ramakrishnan L, Muriki K et al (2010) Performance analysis of high performance computing applications on the amazon web services cloud. In: IEEE international conference on cloud computing technology and science, pp 159–168
Wu Q, Zhu Q, Jian X, Ishikawa F (2014) Broker-based SLA-aware composite service provisioning. J Syst Softw, pp 194–201
Ibarra OH, Kim CE (1977) Heuristic algorithms for scheduling independent tasks on nonidentical processors. J ACM 24(2):280–289
Casas I, Taheri J, Ranjan R, Wang L, Zomaya AY (2016) GA-ETI: an enhanced genetic algorithm for the scheduling of scientific workflows in cloud environments. J of Comput Sci, pp 318–331
Verma A, Kaushal S (2017) A hybrid multi-objective Particle Swarm Optimization for scientific workflow scheduling. Parall Comput, pp 1–19
Zhou X, Zhang G, Sun J, Zhou J, Wei T, Hu S (2019) Minimizing cost and makespan for workflow scheduling in cloud using fuzzy dominance sort based HEFT. Futur Gener Comput Syst, pp 278–289
Wang Y, Jiang J, Xia Y, Wu Q, Luo X, Zhu Q (2018) A multi-stage dynamic game-theoretic approach for multi-workflow scheduling on heterogeneous virtual machines from multiple infrastructure-as-a-service clouds. In: International conference on services computing, pp 137–152
Sahni J, Vidyarthi DP (2018) A cost-effective deadline-constrained dynamic scheduling algorithm for scientific workflows in a cloud environment. IEEE Trans Cloud Comput 6(1):2–18
Mao M, Humphrey M (2011) Auto-scaling to minimize cost and meet application deadlines in cloud workflows. In: IEEE international conference on high performance computing data and analytics
Calheiros RN, Buyya R (2014) Meeting deadlines of scientific workflows in public clouds with tasks replication. IEEE Trans Parall Distr Sys 25(7):1787–1796
Poola D, Garg SK, Buyya R, Yang Y, Ramamohanarao K (2014) Robust scheduling of scientific workflows with deadline and budget constraints in clouds. In: Advanced information networking and applications, pp 858–865
Ghosh R, Longo F, Frattini F, Russo S, Trivedi KS (2014) Scalable analytics for IaaS cloud availability. IEEE Int Conf Cloud Comput Technol Sci 2(1):57–70
Yin X, Ma X, Trivedi KS (2013) An interacting stochastic models approach for the performance evaluation of DSRC vehicular safety communication. IEEE Trans Comput 62(5):873–885
Zheng W, Zhou M, Wu L, et al. (2017) Percentile performance estimation of unreliable IaaS clouds and their cost-optimal capacity decision. IEEE Access, pp 2808–2818
Li W, Xia Y, Zhou M, Sun X, Zhu Q (2018) Fluctuation-aware and predictive workflow scheduling in cost-effective infrastructure-as-a-service clouds. IEEE Access, pp 61488–61502
Ma H, Zhu H, Hu Z, Tang W, Dong P (2017) Multi-valued collaborative QoS prediction for cloud service via time series analysis. Futur Gener Comput Syst 68(68):275–288
Coello CA, Pulido GT, Lechuga MS (2004) Handling multiple objectives with particle swarm optimization. IEEE Trans Evol Comput 8(3):256–279
Deb K, Pratap A, Agarwal S, Meyarivan T (2002) A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans Evol Comput 6(2):182–197
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Pan, Y., Wang, S., Wu, L. et al. A Novel Approach to Scheduling Workflows Upon Cloud Resources with Fluctuating Performance. Mobile Netw Appl 25, 690–700 (2020). https://doi.org/10.1007/s11036-019-01450-0
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11036-019-01450-0