Skip to main content
SpringerLink
Log in

Budget Constraint Bag-of-Task Based Workflow Scheduling in Public Clouds

  • Conference paper
  • First Online:
Computer Supported Cooperative Work and Social Computing (ChineseCSCW 2019)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1042))

Abstract

Bag-of-Tasks (BoT) workflows have appeared in distributed computing platforms such as Spark, MapReduce, and Pegasus. Budget constraints usually exist for these applications. It is crucial to design scheduling algorithms to minimize makespans under budget constraints for BoT workflows. However, most existing workflow algorithms are tailored for general workflows without considering batch structures of Bot-workflows. The main challenge for scheduling BoT workflows is to distribute the budget to different BoTs appropriately considering BoT structures. In this paper, a configuration-and-serialization iterative adjusting based heuristic algorithm (CSIA) is proposed to minimize the makespans under budget constraints. CSIA allocates VM configurations and serial degrees to different BoTs appropriately to decrease the makespan. Experimental results illustrate that the proposal gets shorter makespans on several types of workflow instances than existing algorithms under budget constraints.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Spark lightning-fast unified analytics engine. http://spark.apache.org. Accessed 14 May 2019

  2. A workflow generator. https://confluence.pegasus.isi.edu/display/pegasus/WorkflowGenerator. Accessed 30 June 2016

  3. Abrishami, S., Naghibzadeh, M., Epema, D.H.J.: Deadline-constrained workflow scheduling algorithms for infrastructure; as a service clouds. Future Gener. Comput. Syst. 29(1), 158–169 (2013)

    Article  Google Scholar 

  4. Bharathi, S., Chervenak, A., Deelman, E., Mehta, G., Su, M.H., Vahi, K.: Characterization of scientific workflows. In: Third Workshop on Workflows in Support of Large-Scale Science, pp. 1–10. IEEE, Austin (2008)

    Google Scholar 

  5. Cai, Z., Li, Q., Li, X.: ElasticSim: a toolkit for simulating workflows with cloud resource runtime auto-scaling and stochastic task execution times. J. Grid Comput. 15, 1–16 (2016)

    Google Scholar 

  6. Cai, Z., Li, X., Ruiz, R.: Resource provisioning for task-batch based workflows with deadlines in public clouds. IEEE Trans. Cloud Comput. 7(3), 814–826 (2019)

    Article  Google Scholar 

  7. Cai, Z., Li, X., Ruiz, R., Li, Q.: A delay-based dynamic scheduling algorithm for bag-of-task workflows with stochastic task execution times in clouds. Future Gener. Comput. Syst. 71(C), 57–72 (2017)

    Article  Google Scholar 

  8. Calheiros, R.N., Buyya, R.: Meeting deadlines of scientific workflows in public clouds with tasks replication. IEEE Trans. Parallel Distrib. Syst. 25(7), 1787–1796 (2014)

    Article  Google Scholar 

  9. Chopra, N., Singh, S.: Heft based workflow scheduling algorithm for cost optimization within deadline in hybrid clouds. In: Fourth International Conference on Computing, pp. 1–6 (2014)

    Google Scholar 

  10. Ghafouri, R., Movaghar, A., Mohsenzadeh, M.: A budget constrained scheduling algorithm for executing workflow application in infrastructure as a service clouds. Peer-to-Peer Netw. Appl. 3, 1–28 (2018)

    Google Scholar 

  11. Li, X., Cai, Z.: Elastic resource provisioning for cloud workflow applications. IEEE Trans. Autom. Sci. Eng. 14(2), 1195–1210 (2017)

    Article  Google Scholar 

  12. Li, Z., Ge, J., Hu, H., Wei, S., Luo, B.: Cost and energy aware scheduling algorithm for scientific workflows with deadline constraint in clouds. IEEE Trans. Serv. Comput. 11(1), 713–726 (2018)

    Article  Google Scholar 

  13. Lin, X., Wu, C.Q.: On scientific workflow scheduling in clouds under budget constraint. In: IEEE International Conference on Parallel Processing, vol. 46, no. 1, pp. 90–99 (2013)

    Google Scholar 

  14. Mao, M., Humphrey, M.: Scaling and scheduling to maximize application performance within budget constraints in cloud workflows. In: IEEE International Symposium on Parallel and Distributed Processing, pp. 67–78 (2013)

    Google Scholar 

  15. Nudtasomboon, N., Randhawa, S.U.: Resource-constrained project scheduling with renewable and non-renewable resources and time-resource tradeoffs. Comput. Ind. Eng. 32(1), 227–242 (1997)

    Article  Google Scholar 

  16. Sahni, J., Vidyarthi, D.: A cost-effective deadline-constrained dynamic scheduling algorithm for scientific workflows in a cloud environment. IEEE Trans. Cloud Comput. 6(1), 2–18 (2018)

    Article  Google Scholar 

  17. Shi, J., Luo, J., Fang, D., Zhang, J.: A budget and deadline aware scientific workflow resource provisioning and scheduling mechanism for cloud. In: IEEE International Conference on Computer Supported Cooperative Work in Design, pp. 672–677 (2014)

    Google Scholar 

  18. Shi, J., Luo, J., Fang, D., Zhang, J., Zhang, J.: Elastic resource provisioning for scientific workflow scheduling in cloud under budget and deadline constraints. Cluster Comput. 19(1), 167–182 (2016)

    Article  Google Scholar 

  19. Zaharia, M., Chowdhury, M., Franklin, M.J., Shenker, S., Stoica, I.: Spark: cluster computing with working sets. In: USENIX Conference on Hot Topics in Cloud Computing, pp. 1–10 (2010)

    Google Scholar 

  20. Zeng, L., Veeravalli, B., Li, X.: ScaleStar: budget conscious scheduling precedence-constrained many-task workflow applications in cloud. In: IEEE International Conference on Advanced Information Networking and Applications, pp. 534–541 (2012)

    Google Scholar 

  21. Zhao, H., Sakellariou, R.: Scheduling multiple DAGs onto heterogeneous systems. In: International Parallel and Distributed Processing Symposium, pp. 1–14 (2006)

    Google Scholar 

Download references

Acknowledgements

Zhicheng Cai is supported by the National Natural Science Foundation of China (Grant No. 61602243), the Natural Science Foundation of Jiangsu Province (Grant No. BK20160846), the Fundamental Research Funds for the Central Universities (No. 30919011235) and the Fundamental Research Funds for the Central Universities (No. 30920120180101). Duan Liu is supported by the Postgraduate Research and Practice Innovation Program of Jiangsu Province (No. KYCX18_0434).

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, 210094, China

    Pengfei Sun, Zhicheng Cai & Duan Liu

Authors
  1. Pengfei Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhicheng Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Duan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhicheng Cai .

Editor information

Editors and Affiliations

  1. Shandong University, Jinan, China

    Yuqing Sun

  2. Fudan University, Shanghai, China

    Tun Lu

  3. Kunming University of Science and Technology, Kunming, China

    Zhengtao Yu

  4. Tongji University, Shanghai, China

    Hongfei Fan

  5. University of Shanghai for Science and Technology, Shanghai, China

    Liping Gao

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Sun, P., Cai, Z., Liu, D. (2019). Budget Constraint Bag-of-Task Based Workflow Scheduling in Public Clouds. In: Sun, Y., Lu, T., Yu, Z., Fan, H., Gao, L. (eds) Computer Supported Cooperative Work and Social Computing. ChineseCSCW 2019. Communications in Computer and Information Science, vol 1042. Springer, Singapore. https://doi.org/10.1007/978-981-15-1377-0_18

Download citation

  • DOI: https://doi.org/10.1007/978-981-15-1377-0_18

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-15-1376-3

  • Online ISBN: 978-981-15-1377-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation