Skip to main content
SpringerLink
Log in

Dynamic scheduling of bags-of-tasks with sensitive input data and end-to-end deadlines in a hybrid cloud

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

As organizations with existing on-premise infrastructure investments shift to the hybrid cloud computing paradigm, it is imperative to address the various challenges involved. One of the most important issues is the utilization of novel workload scheduling heuristics in order to effectively harness the security provided by the private cloud and the virtually unlimited resources of the public cloud. In this paper, we propose heuristics for the scheduling of real-time bag-of-tasks jobs that arrive dynamically at a hybrid cloud. The proposed scheduling strategies take into account the end-to-end deadlines of the jobs, as well as the monetary cost required for the utilization of the complementary public cloud resources. Furthermore, they take into consideration that some of the component tasks of the jobs may require input data that are sensitive and thus should not be transferred to the public cloud. The performance of the proposed heuristics is evaluated by simulation. For comparison purposes, two widely used baseline scheduling policies are also examined. In the simulation experiments, we consider jobs with either tight or loose deadlines and with different probabilities that the input data of their component tasks are sensitive.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Abdi S, PourKarimi L, Ahmadi M, Zargari F (2017) Cost minimization for deadline-constrained bag-of-tasks applications in federated hybrid clouds. Futur Gener Comput Syst 71:113–128. https://doi.org/10.1016/j.future.2017.01.036

    Article  Google Scholar 

  2. Bittencourt LF, Madeira ERM (2011) HCOC: A cost optimization algorithm for workflow scheduling in hybrid clouds. J Internet Serv Appl 2:207–227. https://doi.org/10.1007/s13174-011-0032-0

    Article  Google Scholar 

  3. Bittencourt LF, Madeira ERM, Da Fonseca NLS (2012) Scheduling in hybrid clouds. IEEE Commun. Mag 50(9):42–47. https://doi.org/10.1109/MCOM.2012.6295710

    Article  Google Scholar 

  4. Buttazzo GC (2011) Hard Real-Time Computing Systems: Predictable Scheduling Algorithms and Applications, 3rd edn. Springer. https://doi.org/10.1007/978-1-4614-0676-1

  5. Calheiros RN, Buyya R (2012) Cost-effective provisioning and scheduling of deadline-constrained applications in hybrid clouds. In: Proceedings of the 13th International Conference on Web Information Systems Engineering (WISE’12), pp 171–184. https://doi.org/10.1007/978-3-642-35063-4_13

  6. Chang YS, Fan CT, Sheu RK, Jhu SR, Yuan SM (2018) An agent-based workflow scheduling mechanism with deadline constraint on hybrid cloud environment. Int J Commun Syst 31(1):e3401. https://doi.org/10.1002/dac.3401

    Article  Google Scholar 

  7. Chen Y, Tsai WT (2015) Service-oriented computing and web software integration: from principles to development, 5th edn. Kendall Hunt Publishing

  8. Chen Y (2018) Service-oriented computing and system integration: software, IoT, big data, and AI as services, 6th edn. Kendall Hunt Publishing

  9. Chopra N, Singh S (2013) Deadline and cost based workflow scheduling in hybrid cloud. In: Proceedings of the 2nd International Conference on Advances in Computing, Communications and Informatics (ICACCI’13), pp 840–846. https://doi.org/10.1109/ICACCI.2013.6637285

  10. Duan R, Prodan R, Li X (2014) Multi-objective game theoretic scheduling of bag-of-tasks workflows on hybrid clouds. IEEE Trans Cloud Comput 2(1):29–42. https://doi.org/10.1109/TCC.2014.2303077

    Article  Google Scholar 

  11. Freund RF, Gherrity M, Ambrosius S, Campbell M, Halderman M, Hensgen D, Keith E, Kidd T, Kussow M, Lima JD, Mirabile F, Moore L, Rust B, Siegel HJ (1998) Scheduling resources in multi-user, heterogeneous, computing environments with SmartNet. In: Proceedings of the 7th Heterogeneous Computing Workshop (HCW’98), pp 184–199. https://doi.org/10.1109/HCW.1998.666558

  12. Gutierrez-Garcia JO, Sim KM (2013) A family of heuristics for agent-based elastic cloud bag-of-tasks concurrent scheduling. Futur Gener Comput Syst 29(7):1682–1699. https://doi.org/10.1016/j.future.

    Article  Google Scholar 

  13. Ibarra OH, Kim CE (1977) Heuristic algorithms for scheduling independent tasks on nonidentical processors. J ACM 24(2):280–289. https://doi.org/10.1145/322003.322011

    Article  MathSciNet  Google Scholar 

  14. Jararweh Y, Doulat A, AlQudah O, Ahmed E, Al-Ayyoub M, Benkhelifa E (2016) The future of mobile cloud computing: integrating cloudlets and mobile edge computing. In: Proceedings of the 23rd International Conference on Telecommunications (ICT’16), pp 1–5. https://doi.org/10.1109/ICT.2016.7500486

  15. Karatza HD (2007) Performance of gang scheduling policies in the presence of critical sporadic jobs in distributed systems. In: Proceedings of the 2007 International Symposium on Performance Evaluation of Computer and Telecommunication Systems (SPECTS’07), pp 547–554

  16. Kolodziej J (2012) Evolutionary hierarchical multi-criteria metaheuristics for scheduling in large-scale grid systems. Springer, Berlin

  17. Kotb Y, Al Ridhawi I, Aloqaily M, Baker T, Jararweh Y, Tawfik H (2019) Cloud-based multi-agent cooperation for IoT devices using workflow-nets. J Grid Comput 17:625–650. https://doi.org/10.1007/s10723-019-09485-z

    Article  Google Scholar 

  18. Mhedheb Y, Jrad F, Tao J, Zhao J, Kolodziej J, Streit A (2013) Load and thermal-aware VM scheduling on the cloud. In: Proceedings of the 13th International Conference on Algorithms and Architectures for Parallel Processing (ICA3PP’13), pp 101–114. https://doi.org/10.1007/978-3-319-03859-9_8

  19. Moschakis IA, Karatza HD (2015) Multi-criteria scheduling of bag-of-tasks applications on heterogeneous interlinked clouds with simulated annealing. J Syst Softw 101:1–14. https://doi.org/10.1016/j.jss.2014.11.014

    Article  Google Scholar 

  20. Papazachos ZC, Karatza HD (2011) Gang scheduling in multi-core clusters implementing migrations. Futur Gener Comput Syst 27 (8):1153–1165. https://doi.org/10.1016/j.future.2011.02.010

    Article  Google Scholar 

  21. Rahman M, Li X, Palit H (2011) Hybrid heuristic for scheduling data analytics workflow applications in hybrid cloud environment. In: Proceedings of the 2011 IEEE International Symposium on Parallel and Distributed Processing Workshops and PhD Forum, pp 966–974. https://doi.org/10.1109/IPDPS.2011.243

  22. Stavrinides GL, Karatza HD (2014) Scheduling real-time jobs in distributed systems - simulation and performance analysis. In: Proceedings of the 1st International Workshop on Sustainable Ultrascale Computing Systems (NESUS’14), pp 13–18

  23. Stavrinides GL, Karatza HD (2017) The effect of workload computational demand variability on the performance of a SaaS cloud with a multi-tier SLA. In: Proceedings of the IEEE 5th International Conference on Future Internet of Things and Cloud (FiCloud’17), pp 10–17. https://doi.org/10.1109/FiCloud.2017.26

  24. Stavrinides GL, Karatza HD (2018) The impact of workload variability on the energy efficiency of large-scale heterogeneous distributed systems. Simul Model Pract Theory 89:135–143. https://doi.org/10.1016/j.simpat.2018.09.013

    Article  Google Scholar 

  25. Stavrinides GL, Karatza HD (2018) Scheduling data-intensive workloads in large-scale distributed systems: trends and challenges, Studies in Big Data, vol 36, 1st edn., chap. 2. Springer, pp 19–43. https://doi.org/10.1007/978-3-319-73767-6_2

  26. Stavrinides GL, Karatza HD (2019) Cost-effective utilization of complementary cloud resources for the scheduling of real-time workflow applications in a fog environment. In: Proceedings of the 7th International Conference on Future Internet of Things and Cloud (FiCloud’19), pp 1–8. https://doi.org/10.1109/FiCloud.2019.00009

  27. Stavrinides GL, Karatza HD (2019) Performance evaluation of a SaaS cloud under different levels of workload computational demand variability and tardiness bounds. Simul Model Pract Theory 91:1–12. https://doi.org/10.1016/j.simpat.2018.11.006

    Article  Google Scholar 

  28. Stavrinides GL, Karatza HD (2019) Scheduling bag-of-task-chains in distributed systems. In: Proceedings of the 14th IEEE International Symposium on Autonomous Decentralized Systems (ISADS’19), pp 81–86

  29. Stavrinides GL, Karatza HD (2019) Scheduling different types of bag-of-tasks jobs in distributed systems. In: Proceedings of the 10th International Conference on Information and Communication Systems (ICICS’19), pp 13–18. https://doi.org/10.1109/IACS.2019.8809138

  30. Stavrinides GL, Karatza HD (2019) Scheduling single-task jobs along with bag-of-task-chains in distributed systems. In: Proceedings of the 3rd International Conference on Future Networks and Distributed Systems (ICFNDS’19), pp 32:1–32:6. https://doi.org/10.1145/3341325.3342023

  31. Stavrinides GL, Karatza HD (2019) An energy-efficient, QoS-aware and cost-effective scheduling approach for real-time workflow applications in cloud computing systems utilizing DVFS and approximate computations. Futur Gener Comput Syst 96:216–226. https://doi.org/10.1016/j.future.2019.02.019

  32. Stavrinides GL, Karatza HD (2020) Scheduling real-time bag-of-tasks applications with approximate computations in SaaS clouds. Concurr Comput Pract Exp 32(1):e4208. https://doi.org/10.1002/cpe.4208

    Article  Google Scholar 

  33. Tabak EK, Cambazoglu BB, Aykanat C (2014) Improving the performance of independent task assignment heuristics MinMin, MaxMin and Sufferage. IEEE Trans Parallel Distrib Syst 25(5):1244–1256. https://doi.org/10.1109/TPDS.2013.107

    Article  Google Scholar 

  34. Van den Bossche R, Vanmechelen K, Broeckhove J (2010) Cost-optimal scheduling in hybrid IaaS clouds for deadline constrained workloads. In: Proceedings of the 2010 IEEE 3rd International Conference on Cloud Computing (CLOUD’10), pp 228–235. https://doi.org/10.1109/CLOUD.2010.58

  35. Van den Bossche R, Vanmechelen K, Broeckhove J (2013) Online cost-efficient scheduling of deadline-constrained workloads on hybrid clouds. Futur Gener Comput Syst 29(4):973–985. https://doi.org/10.1016/j.future.2012.12.012

    Article  Google Scholar 

  36. Wang WJ, Chang YS, Lo WT, Lee YK (2013) Adaptive scheduling for parallel tasks with QoS satisfaction for hybrid cloud environments. J Supercomput 66 (2):783–811. https://doi.org/10.1007/s11227-013-0890-2

    Article  Google Scholar 

  37. Wang B, Song Y, Sun Y, Liu J (2016) Managing deadline-constrained bag-of-tasks jobs on hybrid clouds. In: Proceedings of the 24th High Performance Computing Symposium (HPC’16), pp 1–8. https://doi.org/10.22360/SpringSim.2016.HPC.039

  38. Zhang Y, Sun J (2017) Novel efficient particle swarm optimization algorithms for solving QoS-demanded bag-of-tasks scheduling problems with profit maximization on hybrid clouds. Concurr Comput Pract Exp 29(21):e4249. https://doi.org/10.1002/cpe.4249

    Article  Google Scholar 

  39. Zhang Y, Zhou J, Sun J (2019) Scheduling bag-of-tasks applications on hybrid clouds under due date constraints. J Syst Archit 101:101654. https://doi.org/10.1016/j.sysarc.2019.101654

    Article  Google Scholar 

  40. Zhang Y, Zhou J, Sun L, Mao J, Sun J (2019) A novel firefly algorithm for scheduling bag-of-tasks applications under budget constraints on hybrid clouds. IEEE Access 7:151888–151901. https://doi.org/10.1109/ACCESS.2019.2948468

  41. Zikos S, Karatza HD (2009) Communication cost effective scheduling policies of nonclairvoyant jobs with load balancing in a grid. J Syst Softw 82(12):2103–2116. https://doi.org/10.1016/j.jss.2009.07.006

    Article  Google Scholar 

  42. Zuo L, Shu L, Dong S, Chen Y, Yan L (2016) A multi-objective hybrid cloud resource scheduling method based on deadline and cost constraints. IEEE Access 5:22067–22080. https://doi.org/10.1109/ACCESS.2016.2633288

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Informatics, Aristotle University of Thessaloniki, 54124, Thessaloniki, Greece

    Georgios L. Stavrinides & Helen D. Karatza

Authors
  1. Georgios L. Stavrinides
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Helen D. Karatza
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Georgios L. Stavrinides.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Stavrinides, G.L., Karatza, H.D. Dynamic scheduling of bags-of-tasks with sensitive input data and end-to-end deadlines in a hybrid cloud. Multimed Tools Appl 80, 16781–16803 (2021). https://doi.org/10.1007/s11042-020-08974-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-08974-8

Keywords

Search

Navigation