Skip to main content
SpringerLink
Log in

Offloading Coalition Formation for Scheduling Scientific Workflow Ensembles in Fog Environments

  • Published:
Journal of Grid Computing Aims and scope Submit manuscript

Abstract

Fog computing provides a distributed computing paradigm that executes interactive and distributed applications, such as the Internet of Things (IoT) applications. Large-scale scientific applications, often in the form of workflow ensembles, have a distributed and interactive nature that demands a dispersed execution environment like fog computing. However, handling a large-scale application in heterogeneous environment of fog computing requires harmonizing heterologous resources over the continuum from the IoT to the cloud. This paper investigates offloading and task allocation problems for orchestrating the resources in a fog computing environment where the IoT application is considered in the form of workflow ensembles. We called Offload-Location a mechanism which has been designed to find offloading coalition structure alongside a matching algorithm for allocating the offloaded tasks to fog/cloud resources. The introduced solution attempts to minimize the execution time and minimize the price paid to servers for executing the tasks provided that Quality of Service (QoS) requirements of the ensemble’s deadline and budget are retaining. These objectives lead to maximizing the number of completed workflows of the ensemble as an ultimate goal. The appropriate performance of this mechanism is studied under different workflow applications and circumstances.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Abbasi, M., Pasand, E.M., Khosravi, M.R.: Workload allocation in iot-fog-cloud architecture using a multi-objective genetic algorithm. J. Grid Comput. 1–14 (2020)

  2. Apt, K.R., Witzel, A.: A generic approach to coalition formation. Int. Game Theor. Rev. 11(03), 347–367 (2009)

    Article  MathSciNet  Google Scholar 

  3. Aral, A., Brandic, I., Uriarte, R.B., De Nicola, R., Scoca, V.: Addressing application latency requirements through edge scheduling. J. Grid Comput. 17(4), 677–698 (2019)

    Article  Google Scholar 

  4. Arisdakessian, S., Wahab, O.A., Mourad, A., Otrok, H., Kara, N.: Fogmatch: An intelligent multi-criteria iot-fog scheduling approach using game theory. IEEE/ACM Trans. Netw. (2020)

  5. Bilbao, J.M.: Cooperative games on combinatorial structures, vol. 26. Springer Science & Business Media, Berlin (2012)

    Google Scholar 

  6. Bogomolnaia, A., Jackson, M.O., et al. : The stability of hedonic coalition structures. Games Econom. Behav. 38(2), 201–230 (2002)

    Article  MathSciNet  Google Scholar 

  7. Bryk, P., Malawski, M., Juve, G., Deelman, E.: Storage-aware algorithms for scheduling of workflow ensembles in clouds. J. Grid Comput. 14(2), 359–378 (2016)

    Article  Google Scholar 

  8. Buyya, R., Srirama, S.N.: Fog and edge computing: principles and paradigms. Wiley, New York (2019)

    Book  Google Scholar 

  9. Chen, L., Xu, J.: Socially trusted collaborative edge computing in ultra dense networks. In: Proceedings of the Second ACM/IEEE Symposium on Edge Computing, pp. 1–11 (2017)

  10. Cybershake project. https://strike.scec.org/scecpedia/CyberShake_Study_13.4

  11. Deelman, E., Gil, Y.: Workshop on the challenges of scientific workflows. Information Sciences Institute (2006)

  12. Demange, G., Gale, D.: The strategy structure of two-sided matching markets. Econometrica: J. Econom. Soc. 873–888 (1985)

  13. Driessen, T.S.: Cooperative games, solutions and applications, vol. 3. Springer Science & Business Media, Berlin (2013)

    Google Scholar 

  14. Fan, W., Liu, Y., Tang, B., Wu, F., Wang, Z.: Computation offloading based on cooperations of mobile edge computing-enabled base stations. IEEE Access 6, 22622–22633 (2017)

    Article  Google Scholar 

  15. Ferguson, T.S.: A course in game theory world scientific (2018)

  16. Gale, D., Shapley, L.S.: College admissions and the stability of marriage. Am. Math. Mon. 69(1), 9–15 (1962)

    Article  MathSciNet  Google Scholar 

  17. Gao, L., Moh, M.: Joint computation offloading and prioritized scheduling in mobile edge computing. In: 2018 International Conference on High Performance Computing & Simulation (HPCS), pp 1000–1007. IEEE (2018)

  18. Gao, X., Huang, X., Bian, S., Shao, Z., Yang, Y.: Pora: Predictive offloading and resource allocation in dynamic fog computing systems. IEEE Int. Things J. 7(1), 72–87 (2019)

    Article  Google Scholar 

  19. Genez, T.A., Bittencourt, L.F., Sakellariou, R., Madeira, E.R.: A flexible scheduler for workflow ensembles. In: Proceedings of the 9th International Conference on Utility and Cloud Computing, pp. 55–62 (2016)

  20. Goudarzi, M., Wu, H., Palaniswami, M.S., Buyya, R.: An application placement technique for concurrent iot applications in edge and fog computing environments. IEEE Trans. Mob. Comput. 1–1 (2020)

  21. Guo, K., Sheng, M., Quek, T.Q., Qiu, Z.: Task offloading and scheduling in fog ran: A parallel communication and computation perspective. IEEE Wirel. Commun. Lett. 9(2), 215–218 (2019)

    Article  Google Scholar 

  22. Hosseinzadeh, M., Ghafour, M.Y., Hama, H.K., Vo, B., Khoshnevis, A.: Multi-objective task and workflow scheduling approaches in cloud computing: a comprehensive review. J. Grid Comput. 1–30 (2020)

  23. Hu, P., Dhelim, S., Ning, H., Qiu, T.: Survey on fog computing: architecture, key technologies, applications and open issues. J. Netw. Comput. Appl. 98, 27–42 (2017)

    Article  Google Scholar 

  24. Huedo, E., Montero, R.S., Moreno-Vozmediano, R., Vázquez, C., Holer, V., Llorente, I.M.: Opportunistic deployment of distributed edge clouds for latency-critical applications. J. Grid Comput. 19(1), 1–16 (2021)

    Article  Google Scholar 

  25. Jošilo, S., Dán, G.: Decentralized scheduling for offloading of periodic tasks in mobile edge computing. In: 2018 IFIP Networking Conference (IFIP Networking) and Workshops, pp 1–9. IEEE (2018)

  26. Juve, G., Chervenak, A., Deelman, E., Bharathi, S., Mehta, G., Vahi, K.: Characterizing and profiling scientific workflows. Futur. Gener. Comput. Syst. 29(3), 682–692 (2013)

    Article  Google Scholar 

  27. Juve, G., Deelman, E., Vahi, K., Mehta, G., Berriman, B., Berman, B.P., Maechling, P.: Data sharing options for scientific workflows on amazon ec2. In: SC’10: Proceedings of the 2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis, pp 1–9. IEEE (2010)

  28. Leyton-Brown, K., Shoham, Y.: Essentials of game theory: A concise multidisciplinary introduction. Synt. Lect. Artif. Intell. Mach Learn. 2(1), 1–88 (2008)

    MATH  Google Scholar 

  29. Ligo project. https://pegasus.isi.edu/application-showcase/ligo/

  30. Liu, Y., Xu, C., Zhan, Y., Liu, Z., Guan, J., Zhang, H.: Incentive mechanism for computation offloading using edge computing: A stackelberg game approach. Comput. Netw. 129, 399–409 (2017)

    Article  Google Scholar 

  31. Malawski, M., Juve, G., Deelman, E., Nabrzyski, J.: Algorithms for cost-and deadline-constrained provisioning for scientific workflow ensembles in iaas clouds. Futur. Gener. Comput. Syst. 48, 1–18 (2015)

    Article  Google Scholar 

  32. Mashayekhy, L., Grosu, D.: A merge-and-split mechanism for dynamic virtual organization formation in grids. IEEE Trans. Parall. Distribut. Syst. 25(3), 540–549 (2014)

    Article  Google Scholar 

  33. Mashayekhy, L., Nejad, M.M., Grosu, D.: Cloud federations in the sky: Formation game and mechanism. IEEE Trans. Cloud Comput. 3(1), 14–27 (2015)

    Article  Google Scholar 

  34. McChesney, J., Wang, N., Tanwer, A., de Lara, E., Varghese, B.: Defog: fog computing benchmarks. In: Proceedings of the 4th ACM/IEEE Symposium on Edge Computing, pp. 47–58 (2019)

  35. Morales, L.E.P.: Efficient support for data-intensive scientific workflows on geo-distributed clouds. Ph.D thesis (2017)

  36. Nisan, N., Ronen, A.: Algorithmic mechanism design. Games Econom. Behav. 35 (1-2), 166–196 (2001)

    Article  MathSciNet  Google Scholar 

  37. Osborne, M.J., et al.: An Introduction to Game Theory, vol. 3. Oxford University Press, New York (2004)

    Google Scholar 

  38. Montage project. http://montage.ipac.caltech.edu

  39. Pegasus project. https://pegasus.isi.edu/application-showcase/

  40. Pietri, I., Malawski, M., Juve, G., Deelman, E., Nabrzyski, J., Sakellariou, R.: Energy-constrained provisioning for scientific workflow ensembles. In: 2013 International Conference on Cloud and Green Computing, pp 34–41. IEEE (2013)

  41. Ren, J., Zhang, D., He, S., Zhang, Y., Li, T.: A survey on end-edge-cloud orchestrated network computing paradigms: Transparent computing, mobile edge computing, fog computing, and cloudlet. ACM Comput. Surv. (CSUR) 52(6), 1–36 (2019)

    Article  Google Scholar 

  42. Rodriguez, M.A., Buyya, R.: Scheduling dynamic workloads in multi-tenant scientific workflow as a service platforms. Futur. Gener. Comput. Syst. 79, 739–750 (2018)

    Article  Google Scholar 

  43. Shakarami, A., Ghobaei-Arani, M., Masdari, M., Hosseinzadeh, M.: A survey on the computation offloading approaches in mobile edge/cloud computing environment: A stochastic-based perspective. J. Grid Comput. 1–33 (2020)

  44. Sipht project. http://newbio.cs.wisc.edu/sRNA/

  45. Stavrinides, G.L., Karatza, H.D.: A hybrid approach to scheduling real-time iot workflows in fog and cloud environments. Multimed. Tools Appl. 78(17), 24639–24655 (2019)

    Article  Google Scholar 

  46. Tianze, L., Muqing, W., Min, Z., Wenxing, L.: An overhead-optimizing task scheduling strategy for ad-hoc based mobile edge computing. IEEE Access 5, 5609–5622 (2017)

    Article  Google Scholar 

  47. Toczé, K., Nadjm-Tehrani, S.: A taxonomy for management and optimization of multiple resources in edge computing. Wirel. Commun. Mob. Comput. 2018 (2018)

  48. Velasquez, K., Abreu, D.P., Assis, M.R., Senna, C., Aranha, D.F., Bittencourt, L.F., Laranjeiro, N., Curado, M., Vieira, M., Monteiro, E., et al.: Fog orchestration for the internet of everything: state-of-the-art and research challenges. J. Int. Serv. Appl. 9(1), 14 (2018)

    Article  Google Scholar 

  49. Xie, Y., Zhu, Y., Wang, Y., Cheng, Y., Xu, R., Sani, A.S., Yuan, D., Yang, Y.: A novel directional and non-local-convergent particle swarm optimization based workflow scheduling in cloud–edge environment. Futur. Gener. Comput. Syst. 97, 361–378 (2019)

    Article  Google Scholar 

  50. Xu, X., Chen, Y., Yuan, Y., Huang, T., Zhang, X., Qi, L.: Blockchain-based cloudlet management for multimedia workflow in mobile cloud computing. Multimed. Tools Appl. 79, 9819–9844 (2019)

    Article  Google Scholar 

  51. Xu, X., Liu, Q., Luo, Y., Peng, K., Zhang, X., Meng, S., Qi, L.: A computation offloading method over big data for iot-enabled cloud-edge computing. Futur. Gener. Comput. Syst. 95, 522–533 (2019)

    Article  Google Scholar 

  52. Yi, S., Hao, Z., Zhang, Q., Zhang, Q., Shi, W., Li, Q.: Lavea: Latency-aware video analytics on edge computing platform. In: Proceedings of the Second ACM/IEEE Symposium on Edge Computing, SEC ’17. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3132211.3134459 (2017)

  53. Yousefpour, A., Fung, C., Nguyen, T., Kadiyala, K., Jalali, F., Niakanlahiji, A., Kong, J., Jue, J.P.: All one needs to know about fog computing and related edge computing paradigms: a complete survey. J. Syst. Archit. 98, 289–330 (2019)

    Article  Google Scholar 

  54. Yu, J., Buyya, R.: A taxonomy of scientific workflow systems for grid computing. ACM Sigmod Record 34(3), 44–49 (2005)

    Article  Google Scholar 

  55. Zhang, K., Mao, Y., Leng, S., Maharjan, S., Zhang, Y.: Optimal delay constrained offloading for vehicular edge computing networks. In: 2017 IEEE International Conference on Communications (ICC), pp 1–6. IEEE (2017)

  56. Zhou, B., Srirama, S.N., Buyya, R.: An auction-based incentive mechanism for heterogeneous mobile clouds. J. Syst. Softw. 152, 151–164 (2019)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Sharif University of Technology, Tehran, Iran

    Hajar Siar & Mohammad Izadi

Authors
  1. Hajar Siar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohammad Izadi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohammad Izadi.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Siar, H., Izadi, M. Offloading Coalition Formation for Scheduling Scientific Workflow Ensembles in Fog Environments. J Grid Computing 19, 34 (2021). https://doi.org/10.1007/s10723-021-09574-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10723-021-09574-y

Keywords

Search

Navigation