Skip to main content
SpringerLink
Log in

ElasticSim: A Toolkit for Simulating Workflows with Cloud Resource Runtime Auto-Scaling and Stochastic Task Execution Times

  • Published:
Journal of Grid Computing Aims and scope Submit manuscript

Abstract

Resource provisioning and scheduling are crucial for cloud workflow applications. Simulation is one of the most promising evaluation methods for different resource provisioning and scheduling algorithms. However, existing simulators for Cloud workflow applications fail to provide support for resource runtime auto-scaling and stochastic task execution time modeling. In this paper, a workflow simulator ElasticSim is introduced, which is an extension of the popular used CloudSim simulator by adding support for resource runtime auto-scaling and stochastic task execution time modeling. Most of existing workflow scheduling algorithms are static and are based on deterministic task execution times. By the aid of ElasticSim, the practical performance of existing static algorithms, when they are put into practice with stochastic task execution times, is evaluated. Experimental results show that about 2.8 % to 20 % additional resource rental cost is incurred for different cases and workflow deadlines are violated for most cases because of stochastic task execution times. Therefore, ElasticSim is a promising platform for evaluating the practical performance of workflow resource provisioning and scheduling algorithms, which supports resource runtime auto-scaling and stochastic task execution time modeling.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. A workflow generator. https://confluence.pegasus.isi.edu/display/pegasus/workflowgenerator, accessed, 2016.6.30

  2. Abrishami, S., Naghibzadeh, M., Epema, D.: Deadline-constrained workflow scheduling algorithms for iaas clouds. Futur. Gener. Comput. Syst. 29(1), 158–169 (2013)

    Article  Google Scholar 

  3. Bartz-Beielstein, T., Chiarandini, M., Paquete, L., Preuss, M.: Book Title: Experimental Methods for the Analysis of Optimization Algorithms. Springer (2010)

  4. Bharathi, S., Chervenak, A., Deelman, E., Mehta, G., Su, M. H., Vahi, K.: Characterization of scientific workflows. In: Third Workshop on Workflows in Support of Large-Scale Science, pp 1–10. IEEE (2008)

  5. Buyya, R.: Deadline based resource provisioningand scheduling algorithm for scientific workflows on clouds. IEEE Transactions on Cloud Computing 2(2), 222–235 (2014)

  6. Byun, E. K., Kee, Y. S., Kim, J. S., Deelman, E., Maeng, S.: BTS: Resource Capacity estimate for time-targeted science workflows. J. Parallel Distrib. Comput. 71(6), 848–862 (2011)

    Article  Google Scholar 

  7. Byun, E. K., Kee, Y. S., Kim, J. S., Maeng, S.: Cost optimized provisioning of elastic resources for application workflows. Futur. Gener. Comput. Syst. 27(8), 1011–1026 (2011)

    Article  Google Scholar 

  8. Cai, Z., Li, X., Gupta, J. N. D.: Heuristics for provisioning services to workflows in xaas clouds. IEEE Trans. Serv. Comput. 9(2), 250–263 (2016)

    Article  Google Scholar 

  9. Cai, Z., Li, X., Ruiz, R.: Cloud YARN resource provisioning for task-batch based workflows with deadlines. Technical report https://github.com/czcnjust/elasticsim/blob/master/technicalreport201500805.pdf (2016)

  10. Calheiros, R. N., Buyya, R.: Meeting deadlines of scientific workflows in public clouds with tasks replication. IEEE Trans. Parallel Distrib. Syst. 25(7), 1787–1796 (2014)

    Article  Google Scholar 

  11. Calheiros, R. N., Ranjan, R., Beloglazov, A., De Rose, C. A., Buyya, R.: Cloudsim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms. Software: Practice and Experience 41(1), 23–50 (2011)

    Google Scholar 

  12. Carrington, L., Snavely, A., Wolter, N.: A performance prediction framework for scientific applications. Futur. Gener. Comput. Syst. 22(3), 336–346 (2006)

    Article  Google Scholar 

  13. Chen, W., Deelman, E.: Workflowsim: a toolkit for simulating scientific workflows in distributed environments. In: IEEE International Conference on E-Science, pp 1–8 (2012)

  14. Chen, W., Silva, R. F. D., Deelman, E., Sakellariou, R.: Using imbalance metrics to optimize task clustering in scientific workflow executions. Futur. Gener. Comput. Syst. 46, 69–84 (2015)

    Article  Google Scholar 

  15. David, L., Puaut, I.: Static determination of probabilistic execution times. In: Euromicro Conference on Real-Time Systems, pp 223–230 (2004)

  16. Deelman, E., Gannon, D., Shields, M., Taylor, I.: Workflows and e-science: an overview of workflow system features and capabilities. Futur. Gener. Comput. Syst. 25(5), 528–540 (2009)

    Article  Google Scholar 

  17. Duan, R., Nadeem, F., Wang, J., Zhang, Y.: A hybrid intelligent method for performance modeling and prediction of workflow activities in grids. In: 9th IEEE/ACM International Symposium on Cluster Computing and the Grid, pp 339–347 (2009)

  18. Durillo, J. J., Prodan, R.: Multi-objective workflow scheduling in Amazon EC2. Clust. Comput. 17(2), 169–189 (2013)

    Article  Google Scholar 

  19. Galante, G., Erpen De Bona, L. C., Mury, A. R., Schulze, B., da Rosa Righi, R.: An analysis of public clouds elasticity in the execution of scientific applications: a survey. Journal of Grid Computing 14(2), 193–216 (2016)

    Article  Google Scholar 

  20. Iverson, M. A., Ozguner, F., Potter, L. C.: Statistical prediction of task execution times through analytic benchmarking for scheduling in a heterogeneous environment. IEEE Trans. Comput. 48(12), 1374–1379 (1999)

    Article  Google Scholar 

  21. Jia, Y., Buyya, R.: A taxonomy of workflow management systems for grid computing. Journal of Grid Computing 3(3-4), 171–200 (2005)

    Article  Google Scholar 

  22. Kamthe, A., Lee, S. Y.: A stochastic approach to estimating earliest start times of nodes for scheduling dags on heterogeneous distributed computing systems. Clust. Comput. 14(4), 377–395 (2005)

    Article  Google Scholar 

  23. Kecskemeti, G.: Dissect-cf: a simulator to foster energy-aware scheduling in infrastructure clouds. Simulation Modelling Practice and Theory 58P2, 188– 218 (2015)

    Article  Google Scholar 

  24. Kertesz, A., Dombi, J. D., Benyi, A.: A pliant-based virtual machine scheduling solution to improve the energy efficiency of iaas clouds. Journal of Grid Computing, 1–13 (2015)

  25. Kliazovich, D., Bouvry, P., Khan, S. U.: Greencloud: a packet-level simulator of energy-aware cloud computing data centers. J. Supercomput. 62(3), 1–5 (2010)

    Google Scholar 

  26. Lastovetsky, A., Twamley, J.: Towards a realistic performance model for networks of heterogeneous computers. In: International Federation for Information Processing Digital Library; High PERFORMANCE Computational Science and Engineering, pp 39–57 (2004)

  27. Li, X., Cai, Z.: Elastic resource provisioning for cloud workflow applications. IEEE Trans. Autom. Sci. Eng. (2015). doi:10.1109/TASE.2015.2500574. in press

    Google Scholar 

  28. Liu, J., Pacitti, E., Valduriez, P., Mattoso, M.: A survey of data-intensive scientific workflow management. Journal of Grid Computing 13(4), 457–493 (2015)

    Article  Google Scholar 

  29. Lorido-Botran, T., Miguel-Alonso, J., Lozano, J. A.: A review of auto-scaling techniques for elastic applications in cloud environments. Journal of Grid Computing 12(4), 559–592 (2014)

    Article  Google Scholar 

  30. Malawski, M., Juve, G., Deelman, E., Nabrzyski, J.: Algorithms for cost- and deadline-constrained provisioning for scientific workflow ensembles in iaas clouds. Futur. Gener. Comput. Syst. 48, 1–18 (2015)

    Article  Google Scholar 

  31. Núñez, A., Vázquez-Poletti, J. L., Caminero, A. C., Castañé, G. G., Carretero, J., Llorente, I.M.: icancloud: a flexible and scalable cloud infrastructure simulator. Journal of Grid Computing 10(1), 185–209 (2012)

    Article  Google Scholar 

  32. Singh, S., Chana, I.: A survey on resource scheduling in cloud computing: Issues and challenges. Journal of Grid Computing 14(2), 217–264 (2016)

    Article  Google Scholar 

  33. Skutella, M., Uetz, M.: Stochastic machine scheduling with precedence constraints. Siam Journal on Computing 34(4), 788–802 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  34. Su, S., Li, J., Huang, Q., Huang, X., Shuang, K., Wang, J.: Cost-efficient task scheduling for executing large programs in the cloud. Parallel Comput. 39(4C5), 177–188 (2013)

    Article  Google Scholar 

  35. Tang, X., Li, K., Liao, G., Fang, K., Wu, F.: A stochastic scheduling algorithm for precedence constrained tasks on grid. Futur. Gener. Comput. Syst. 27(8), 1083–1091 (2011)

    Article  Google Scholar 

  36. Tian, W., Xu, M., Chen, A., Li, G., Wang, X., Chen, Y.: Open-source simulators for cloud computing: Comparative study and challenging issues. Simul. Model. Pract. Theory 58, 239–254 (2015)

    Article  Google Scholar 

  37. Tian, W., Zhao, Y.: A toolkit for modeling and simulation of real-time virtual machine allocation in a cloud data center. IEEE Trans. Autom. Sci. Eng. 12(1), 153–161 (2015)

    Article  Google Scholar 

  38. Verma, A., Kaushal, S.: Cost-time efficient scheduling plan for executing workflows in the cloud. Journal of Grid Computing 13(4), 495–506 (2015)

    Article  MathSciNet  Google Scholar 

  39. Zheng, W., Sakellariou, R.: Stochastic dag scheduling using a monte carlo approach. J. Parallel Distrib. Comput. 73(12), 1673–1689 (2013)

    Article  MATH  Google Scholar 

  40. Zhou, A. C., He, B.: Simplified resource provisioning for workflows in iaas clouds. In: IEEE International Conference on Cloud Computing Technology and Science, pp 650–655 (2014)

Download references

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China

    Zhicheng Cai & Qianmu Li

  2. Key Laboratory of Computer Network and Information Integration (Southeast University), Ministry of Education, Nanjing, China

    Zhicheng Cai

  3. School of Computer Science and Engineering, Southeast University, Nanjing, China

    Xiaoping Li

Authors
  1. Zhicheng Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qianmu Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaoping Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhicheng Cai.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cai, Z., Li, Q. & Li, X. ElasticSim: A Toolkit for Simulating Workflows with Cloud Resource Runtime Auto-Scaling and Stochastic Task Execution Times. J Grid Computing 15, 257–272 (2017). https://doi.org/10.1007/s10723-016-9390-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10723-016-9390-y

Keywords

Search

Navigation