Abstract
With the emergence of the Fog paradigm, the relocation of computational capabilities to the network’s edge has become imperative to support the ever-growing requirements of latency-sensitive, data-intensive, and real-time decision-making applications. In dynamic and mobile environments, services must adapt to accommodate the mobility of users, resulting in frequent relocations across computing nodes to ensure seamless user experiences. However, these migrations incur additional costs and potentially degrade the Quality of Service (QoS) parameters. In this paper, we propose a mobility-aware workflow migration approach based on Deep Reinforcement Learning (DRL). This approach aims to minimize the system’s overall delay and energy consumption by optimizing the number of workflow task migrations, considering resource performance and network conditions in different regions. The problem is first formulated as a Markov Decision Process (MDP), and then a Double Deep Q-network (DDQN) algorithm is proposed to identify the optimal policy for workflow offloading and migration. Comprehensive experiments have been conducted and the results demonstrate that our approach outperforms significantly the existing approaches.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bao, W., et al.: Follow me fog: toward seamless handover timing schemes in a fog computing environment. IEEE Commun. Mag. 55(11), 72–78 (2017)
Bessai, K., Youcef, S., Oulamara, A., Godart, C., Nurcan, S.: Bi-criteria workflow tasks allocation and scheduling in cloud computing environments. In: 2012 IEEE Fifth International Conference on Cloud Computing, pp. 638–645 (2012)
Bonomi, F., Milito, R., Zhu, J., Addepalli, S.: Fog computing and its role in the Internet of Things. In: Proceedings of the First Edition of the MCC Workshop on Mobile Cloud Computing, pp. 13–16 (2012)
Boubaker, N.E.H., Zarour, K., Guermouche, N., Benmerzoug, D.: Fog and edge service migration approaches based on machine learning techniques: a short survey. In: Proceedings of the TACC Conference (2022)
Chen, W., Chen, Y., Liu, J.: Service migration for mobile edge computing based on partially observable Markov decision processes. Comput. Electr. Eng. 106, 108552 (2023)
Chen, W., Chen, Y., Wu, J., Tang, Z.: A multi-user service migration scheme based on deep reinforcement learning and SDN in mobile edge computing. Phys. Commun. 47, 101397 (2021)
Dastjerdi, A.V., Gupta, H., Calheiros, R.N., Ghosh, S.K., Buyya, R.: Fog computing: principles, architectures, and applications. In: Internet of Things, pp. 61–75 (2016)
De Vita, F., Bruneo, D., Puliafito, A., Nardini, G., Virdis, A., Stea, G.: A deep reinforcement learning approach for data migration in multi-access edge computing. In: 2018 ITU Kaleidoscope: Machine Learning for a 5G Future (ITU K), pp. 1–8 (2018)
Gao, Z., Jiao, Q., Xiao, K., Wang, Q., Mo, Z., Yang, Y.: Deep reinforcement learning based service migration strategy for edge computing. In: 2019 IEEE International Conference on Service-Oriented System Engineering (SOSE), pp. 116–1165 (2019)
Hong, C.H., Varghese, B.: Resource management in fog/edge computing: a survey on architectures, infrastructure, and algorithms. ACM Comput. Surv. (CSUR) 52(5), 1–37 (2019)
Jayanetti, A., Halgamuge, S., Buyya, R.: Deep reinforcement learning for energy and time optimized scheduling of precedence-constrained tasks in edge-cloud computing environments. Future Gener. Comput. Syst. 137, 14–30 (2022)
Juric, M.B., Mathew, B., Sarang, P.G.: Business Process Execution Language for Web Services: An Architect and Developer’s Guide to Orchestrating Web Services Using BPEL4WS. Packt Publishing Ltd. (2006)
Lan, D., Taherkordi, A., Eliassen, F., Chen, Z., Liu, L.: Deep reinforcement learning for intelligent migration of fog services in smart cities. In: Qiu, M. (ed.) ICA3PP 2020, Part II. LNCS, vol. 12453, pp. 230–244. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-60239-0_16
Liu, G., Wang, J., Tian, Y., Yang, Z., Wu, Z.: Mobility-aware dynamic service placement for edge computing. EAI Endorsed Trans. Internet Things 5, e2 (2019)
Mehran, N., Kimovski, D., Prodan, R.: MAPO: a multi-objective model for IoT application placement in a fog environment. In: Proceedings of the 9th International Conference on the Internet of Things, pp. 1–8 (2019)
Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015)
Moon, S., Park, J., Lim, Y.: Task migration based on reinforcement learning in vehicular edge computing. Wirel. Commun. Mob. Comput. 2021, 1–10 (2021)
Park, S.W., Boukerche, A., Guan, S.: A novel deep reinforcement learning based service migration model for mobile edge computing. In: 2020 IEEE/ACM 24th International Symposium on Distributed Simulation and Real Time Applications (DS-RT), pp. 1–8 (2020)
Peng, Y., Liu, L., Zhou, Y., Shi, J., Li, J.: Deep reinforcement learning-based dynamic service migration in vehicular networks. In: 2019 IEEE Global Communications Conference (GLOBECOM), pp. 1–6 (2019)
Puliafito, C., Mingozzi, E., Anastasi, G.: Fog computing for the internet of mobile things: issues and challenges. In: 2017 IEEE International Conference on Smart Computing (SMARTCOMP), pp. 1–6 (2017)
Van Hasselt, H., Guez, A., Silver, D.: Deep reinforcement learning with double Q-learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 30 (2016)
Wang, S.G., Sun, Q.B., Yang, F.C.: Web service dynamic selection by the decomposition of global QoS constraints. Ruanjian Xuebao/J. Softw. 22(7), 1426–1439 (2011)
Wu, C.L., Chiu, T.C., Wang, C.Y., Pang, A.C.: Mobility-aware deep reinforcement learning with glimpse mobility prediction in edge computing. In: ICC 2020–2020 IEEE International Conference on Communications (ICC), pp. 1–7 (2020)
Yi, S., Li, C., Li, Q.: A survey of fog computing: concepts, applications and issues. In: Proceedings of the 2015 Workshop on Mobile Big Data, pp. 37–42 (2015)
Zaki, A.M., Elsayed, S.A., Elgazzar, K., Hassanein, H.S.: Heuristic-based proactive service migration induced by dynamic computation load in edge computing. In: GLOBECOM 2022–2022 IEEE Global Communications Conference, pp. 5668–5673 (2022)
Acknowledgements
This work was supported by the ANR LabEx CIMI (grant ANR-11-LABX-0040) within the French State Programme “Investissements d’Avenir”.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Boubaker, N.E.H., Zarour, K., Guermouche, N., Benmerzoug, D. (2024). Double Deep Q-Network-Based Time and Energy-Efficient Mobility-Aware Workflow Migration Approach. In: Sellami, M., Vidal, ME., van Dongen, B., Gaaloul, W., Panetto, H. (eds) Cooperative Information Systems. CoopIS 2023. Lecture Notes in Computer Science, vol 14353. Springer, Cham. https://doi.org/10.1007/978-3-031-46846-9_6
Download citation
DOI: https://doi.org/10.1007/978-3-031-46846-9_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-46845-2
Online ISBN: 978-3-031-46846-9
eBook Packages: Computer ScienceComputer Science (R0)