Abstract
Workflow scheduling in cloud computing environments is nowadays a hot topic as scientific workflows application are gradually taking advantage of commercial cloud assets. Common users’ quality of service (QoS) requirements are the respect of defined budget and deadline when executing their workflow job. Since execution cost minimization and completion time minimization are contradictory objectives, addressing such issue through trade-off function approaches have proved to be an efficient way. This paper presents the Cost-Time Trade-off efficient Workflow Scheduling with Dynamic provisioning (CTTWSDP) algorithm. CTTWSDP relies on dynamic VMs provisioning with a limited number of leased VMs, and a cost-time trade-off function over heterogeneous instances to determine the most viable schedule. CTTWSDP also proposed an improved Implicit Requested Instance Types Range (IRITR) evaluation, which is a scheduling concept introduced in our previous work. The IRITR evaluation aims at determining a range of VMs instance types that best suits the workflow execution, in order to avoid overbidding and underbidding that may lead to budget and deadline violation respectively. The results of simulations prove the effectiveness of the proposal. CTTWSDP achieves a 17.09–76.06% higher success rate when compared to four state-of-the-art algorithms. Furthermore, ANOVA along with Tukey–Kramer post-hoc tests have been conducted, revealing the effectiveness of CTTWSDP over three of the baseline algorithm, while for the fourth one the outperformance of CTTWSDP is not statistically significant. An analysis of the standard deviation of the success rate proves that CTTWSDP is more stable in its performance no matter the type and the workload of the workflow. With a standard deviation of 6.73, smaller than the ones obtained by the other algorithms from 18.66 to 34.10.
Similar content being viewed by others
Data availability
The data that support the findings of this study are available from the corresponding author, upon reasonable request.
Notes
https://www.excel-easy.com/examples/anova.html
http://davidmlane.com/hyperstat/sr_table.html
References
Taylor, I.J., Deelman, E., Gannon, D.B., Shields, M., et al.: Workflows for e-Science: Scientific Workflows for Grids, vol. 1. Springer, New York (2007)
Gil, Y., Deelman, E., Ellisman, M., Fahringer, T., Fox, G., Gannon, D., Goble, C., Livny, M., Moreau, L., Myers, J.: Examining the challenges of scientific workflows. Computer 40(12), 24–32 (2007)
Ullman, J.D.: Np-complete scheduling problems. J. Comput. Syst. Sci. 10(3), 384–393 (1975)
Balis, B., Figiela, K., Jopek, K., Malawski, M., Pawlik, M.: Porting hpc applications to the cloud: a multi-frontal solver case study. J. Comput. Sci. 18, 106–116 (2017)
Madduri, R., Chard, K., Chard, R., Lacinski, L., Rodriguez, A., Sulakhe, D., Kelly, D., Dave, U., Foster, I.: The globus galaxies platform: delivering science gateways as a service. Concurr. Comput. Pract. Expe. 27(16), 4344–4360 (2015)
Vöckler, J.S., Juve, G., Deelman, E., Rynge, M., Berriman, B.: Experiences using cloud computing for a scientific workflow application. In: Proceedings of the 2nd international workshop on Scientific cloud computing, ACM, pp 15–24 (2011)
Jackson, K.R., Ramakrishnan, L., Muriki, K., Canon, S., Cholia, S., Shalf, J., Wasserman, H.J., Wright, N.J.: Performance analysis of high performance computing applications on the amazon web services cloud. In: Proceedings of the 2nd IEEE international conference on cloud computing technology and science, IEEE, pp 159–168 (2010)
Juve, G., Deelman, E., Vahi, K., Mehta, G., Berriman, B., Berman, B.P., Maechling, P.: Scientific workflow applications on amazon ec2. In: Proceedings of the 2009 5th IEEE international conference on e-science workshops, IEEE, pp 59–66 (2009)
Deelman, E., Singh, G., Livny, M., Berriman, B., Good, J.: The cost of doing science on the cloud: the montage example. In: SC’08: Proceedings of the 2008 ACM/IEEE conference on Supercomputing, IEEE, pp 1–12 (2008)
Rodriguez, M.A., Buyya, R.: Scheduling dynamic workloads in multi-tenant scientific workflow as a service platforms. Future Gen. Comput. Syst. 79, 739–750 (2018)
Arabnejad, V., Bubendorfer, K., Ng, B.: Budget and deadline aware e-science workflow scheduling in clouds. IEEE Trans. Parallel Distrib. Syst. 30(1), 29–44 (2019)
Ndamlabin Mboula, J.E., Kamla, V.C., Tayou Djamegni, C.: Cost-time trade-off efficient workflow scheduling in cloud. Simul. Model. Pract. Theory, p. 102107 (2020)
Hilman, M.H., Rodriguez, M.A., Buyya, R.: Multiple workflows scheduling in multi-tenant distributed systems: a taxonomy and future directions. ACM Comput. Surv. (CSUR) 53(1), 1–39 (2020)
Ghobaei-Arani, M., Jabbehdari, S., Pourmina, M.A.: An autonomic resource provisioning approach for service-based cloud applications: a hybrid approach. Future Gen. Comput. Syst. 78, 191–210 (2018)
Hilman, M.H., Rodriguez, M.A., Buyya, R.: Task runtime prediction in scientific workflows using an online incremental learning approach. In: Proceedings of the 2018 IEEE/ACM 11th International Conference on Utility and Cloud Computing (UCC), IEEE, pp 93–102 (2018)
Juve, G., Chervenak, A., Deelman, E., Bharathi, S., Mehta, G., Vahi, K.: Characterizing and profiling scientific workflows. Future Gen. Comput. Syst. 29(3), 682–692 (2013)
Singh, V., Gupta, I., Jana, P.K.: A novel cost-efficient approach for deadline-constrained workflow scheduling by dynamic provisioning of resources. Future Gen. Comput. Syst. 79, 95–110 (2018)
Garg, R., Mittal, M., et al.: Reliability and energy efficient workflow scheduling in cloud environment. Clust. Comput. 22(4), 1283–1297 (2019)
Faragardi, H.R., Sedghpour, M.R.S., Fazliahmadi, S., Fahringer, T., Rasouli, N.: Grp-heft: a budget-constrained resource provisioning scheme for workflow scheduling in iaas clouds. IEEE Trans. Parallel Distrib. Syst. 31, 1239–1254 (2019)
Arabnejad, V., Bubendorfer, K., Ng, B.: Scheduling deadline constrained scientific workflows on dynamically provisioned cloud resources. Future Gen. Comput. Syst. 75, 348–364 (2017)
Ahmad, W., Alam, B., Ahuja, S., Malik, S.: A dynamic vm provisioning and de-provisioning based cost-efficient deadline-aware scheduling algorithm for big data workflow applications in a cloud environment. Clust. Comput., pp 1–30 (2020)
Chen, W., Deelman, E.: Workflowsim: A toolkit for simulating scientific workflows in distributed environments. In: Proceedings of the 2012 IEEE 8th International Conference on E-Science, IEEE, pp 1–8 (2012)
Bharathi, S., Chervenak, A., Deelman, E., Mehta, G., Su, M.H., Vahi, K.: Characterization of scientific workflows. In: Proceedings of the 2008 third workshop on workflows in support of large-scale science, IEEE, pp 1–10 (2008)
Topcuoglu, H., Hariri, S., My, Wu.: Performance-effective and low-complexity task scheduling for heterogeneous computing. IEEE Trans. Parallel Distrib. Syst. 13(3), 260–274 (2002)
Hu, X.H., Ouyang, J.C., Yang, Z.H., Chen, Z.H.: An ipso algorithm for grid task scheduling based on satisfaction rate. In: Proceedings of the 2009 International Conference on Intelligent Human-Machine Systems and Cybernetics, IEEE, vol. 1, pp. 262–265 (2009)
Chen, W.N., Zhang, J.: An ant colony optimization approach to a grid workflow scheduling problem with various qos requirements. IEEE Trans. Syst. Man Cybern. Part C (Appl. Rev.) 39(1), 29–43 (2008)
Chuang, L.Y., Tsai, S.W., Yang, C.H. Catfish particle swarm optimization. In: Proceedings of the 2008 IEEE Swarm Intelligence Symposium, IEEE, pp 1–5. https://doi.org/10.1109/SIS.2008.4668277 (2008)
Verma, A., Kaushal, S.: A hybrid multi-objective particle swarm optimization for scientific workflow scheduling. Parallel Comput. 62, 1–19 (2017)
Elsherbiny, S., Eldaydamony, E., Alrahmawy, M., Reyad, A.E.: An extended intelligent water drops algorithm for workflow scheduling in cloud computing environment. Egypt. Inf. J. 19(1), 33–55 (2018)
Casas, I., Taheri, J., Ranjan, R., Wang, L., Zomaya, A.Y.: Ga-eti: an enhanced genetic algorithm for the scheduling of scientific workflows in cloud environments. J. Comput. Sci. 26, 318–331 (2018)
Zhou, J., Wang, T., Cong, P., Lu, P., Wei, T., Chen, M.: Cost and makespan-aware workflow scheduling in hybrid clouds. J. Syst. Architect. 100, 101631 (2019)
Biswas, T., Kuila, P., Ray, A.K., Sarkar, M.: Gravitational search algorithm based novel workflow scheduling for heterogeneous computing systems. Simul. Model. Pract. Theory 96, 101932 (2019)
Zheng, W., Sakellariou, R.: Budget-deadline constrained workflow planning for admission control. J. Grid Comput. 11(4), 633–651 (2013)
Verma, A., Kaushal, S.: Cost-time efficient scheduling plan for executing workflows in the cloud. J. Grid Comput. 13(4), 495–506 (2015)
Poola, D., Garg, S.K., Buyya, R., Yang, Y., Ramamohanarao, K.: Robust scheduling of scientific workflows with deadline and budget constraints in clouds. In: Proceedings of the 2014 IEEE 28th international conference on advanced information networking and applications, IEEE, pp. 858–865 (2014)
Khan, M.A.: Scheduling for heterogeneous systems using constrained critical paths. Parallel Comput. 38(4–5), 175–193 (2012)
Amazon EC2 (????a) Amazon EC2 Instance Types. https://aws.amazon.com/ec2/instance-types/, online; accessed 06 July 2019
Amazon EC2 (????b) Amazon EC2 pricing. https://aws.amazon.com/ec2/pricing/on-demand/, online; accessed 06 July 2019
Mao, M., Humphrey, M.: A performance study on the vm startup time in the cloud. In: Proceedings of the 2012 IEEE Fifth International Conference on Cloud Computing, IEEE, pp. 423–430 (2012)
Ghobaei-Arani, M., Rahmanian, A.A., Aslanpour, M.S., Dashti, S.E.: Csa-wsc: cuckoo search algorithm for web service composition in cloud environments. Soft. Comput. 22(24), 8353–8378 (2018)
Ghobaei-Arani, M., Rahmanian, A.A., Souri, A., Rahmani, A.M.: A moth-flame optimization algorithm for web service composition in cloud computing: simulation and verification. Softw. Pract. Exp. 48(10), 1865–1892 (2018)
Singh, V., Gupta, I., Jana, P.K.: An energy efficient algorithm for workflow scheduling in iaas cloud. J. Grid Comput. pp. 1–20 (2019)
Choudhary, A., Gupta, I., Singh, V., Jana, P.K.: A gsa based hybrid algorithm for bi-objective workflow scheduling in cloud computing. Future Gen. Comput. Syst. 83, 14–26 (2018)
Statology. How to Perform a Tukey–Kramer Post Hoc Test in Excel. https://www.statology.org/tukey-kramer-post-hoc-test-excel/, online; accessed 9 January 2021 (2020)
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Ndamlabin Mboula, J.E., Kamla, V.C. & Tayou Djamégni, C. Dynamic provisioning with structure inspired selection and limitation of VMs based cost-time efficient workflow scheduling in the cloud. Cluster Comput 24, 2697–2721 (2021). https://doi.org/10.1007/s10586-021-03289-1
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10586-021-03289-1