Abstract
One of the challenges in the Computing Continuum paradigm is the optimal distribution of the generated tasks between the devices in each layer (cloud-fog-edge). In this paper, we propose to use Reinforcement Learning (RL) to solve the Task Assignment Problem (TAP) at the edge layer and then we propose a novel multi-layer extension of RL (ML-RL) techniques that allows edge agents to query an upper-level agent with more knowledge to improve the performance in complex and uncertain situations. We first formulate the task assignment process considering the trade-off between energy consumption and execution time. We then present a greedy solution as a baseline and implement our two RL proposals in the PureEdgeSim simulator. Finally, several simulations of each algorithm are evaluated with different numbers of devices to verify scalability. The simulation results show that reinforcement learning solutions outperformed the heuristic-based solutions and our multi-layer approach can significantly improve performance in high device density scenarios.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Adhikari, M., Mukherjee, M., Srirama, S.: DPTO: a deadline and priority-aware task offloading in fog computing framework leveraging multilevel feedback queueing. IEEE Internet Things J. 7, 5773–5782 (2020)
Alfakih, T., Hassan, M., Gumaei, A., Savaglio, C., Fortino, G.: Task offloading and resource allocation for mobile edge computing by deep reinforcement learning based on SARSA. IEEE Access 8, 54074–54084 (2020)
Argerich, M.F., Fürst, J., Cheng, B.: Tutor4RL: guiding reinforcement learning with external knowledge. In: AAAI Spring Symposium: Combining Machine Learning with Knowledge Engineering (2020)
Bellendorf, J., Ádám Mann, Z.: Classification of optimization problems in fog computing. Futur. Gener. Comput. Syst. 107, 158–176 (2020). https://doi.org/10.1016/j.future.2020.01.036
Ghanavati, S., Abawajy, J., Izadi, D.: An energy aware task scheduling model using ant-mating optimization in fog computing environment. IEEE Trans. Serv. Comput. 15(4), 2007-2017 (2020)
Jia, Z., Yu, J., Ai, X., Xu, X., Yang, D.: Cooperative multiple task assignment problem with stochastic velocities and time windows for heterogeneous unmanned aerial vehicles using a genetic algorithm. Aerosp. Sci. Technol. 76, 112–125 (2018). https://doi.org/10.1016/j.ast.2018.01.025
Liang, J., Long, Y., Mei, Y., Wang, T., Jin, Q.: A distributed intelligent hungarian algorithm for workload balance in sensor-cloud systems based on urban fog computing. IEEE Access 7, 77649–77658 (2019). https://doi.org/10.1109/ACCESS.2019.2922322
Liu, L., Qi, D., Zhou, N., Wu, Y.: A task scheduling algorithm based on classification mining in fog computing environment. Wirel. Commun. Mobile Comput. 2018, 1–11 (2018). https://doi.org/10.1155/2018/2102348
Liu, X., Qin, Z., Gao, Y.: Resource allocation for edge computing in IoT networks via reinforcement learning. In: ICC 2019–2019 IEEE International Conference on Communications (ICC), pp. 1–6 (2019)
Mechalikh, C., Taktak, H., Moussa, F.: Pureedgesim: a simulation framework for performance evaluation of cloud, edge and mist computing environments. Comput. Sci. Inf. Syst. 18, 42 (2020). https://doi.org/10.2298/CSIS200301042M
Rahbari, D., Nickray, M.: Low-latency and energy-efficient scheduling in fog-based IoT applications. Turk. J. Electr. Eng. Comput. Sci. 27, 1406–1427 (2019)
Rahbari, D., Nickray, M.: Task offloading in mobile fog computing by classification and regression tree. Peer-to-Peer Netw. Appl. 13, 104–122 (2020)
Ren, C., Lyu, X., Ni, W., Tian, H., Song, W., Liu, R.P.: Distributed online optimization of fog computing for internet of things under finite device buffers. IEEE Internet Things J. 7(6), 5434–5448 (2020). https://doi.org/10.1109/JIOT.2020.2979353
Robles-Enciso, A.: Pureedgesim RL extension (2021). https://github.com/alb1183/ML-RL-PureEdgeSim
Robles-Enciso, A.: ML-RL Simulations results (2022). https://github.com/alb1183/ML-RL-simulations
Sen, T., Shen, H.: Machine learning based timeliness-guaranteed and energy-efficient task assignment in edge computing systems. In: 2019 IEEE 3rd International Conference on Fog and Edge Computing (ICFEC), pp. 1–10 (2019)
Shakarami, A., Ghobaei-Arani, M., Shahidinejad, A.: A survey on the computation offloading approaches in mobile edge computing: a machine learning-based perspective. Comput. Netw. 182, 107496 (2020). https://doi.org/10.1016/j.comnet.2020.107496
Wang, J., Zhao, L., Liu, J., Kato, N.: Smart resource allocation for mobile edge computing: a deep reinforcement learning approach. IEEE Trans. Emerg. Top. Comput. 9(3), 1529–1541 (2019)
Wen, Z., Yang, R., Garraghan, P., Lin, T., Xu, J., Rovatsos, M.: Fog orchestration for internet of things services. IEEE Internet Comput. 21(2), 16–24 (2017). https://doi.org/10.1109/MIC.2017.36
Zhang, G., Shen, F., Liu, Z., Yang, Y., Wang, K., Zhou, M.T.: FEMTO: fair and energy-minimized task offloading for fog-enabled IoT networks. IEEE Internet Things J. 6(3), 4388–4400 (2019). https://doi.org/10.1109/JIOT.2018.2887229
Acknowledgments
This work was supported by the FPI Grant 21463/FPI/20 of the Seneca Foundation in Region of Murcia (Spain), partially funded by project PID2020–112675RB–C44 and PTAS–20211009 MCIN/AEI/10.13039/501100011033 and by the “European Union NextGenerationEU/PRTR”.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Robles-Enciso, A., Skarmeta, A.F. (2022). Task Offloading in Computing Continuum Using Collaborative Reinforcement Learning. In: González-Vidal, A., Mohamed Abdelgawad, A., Sabir, E., Ziegler, S., Ladid, L. (eds) Internet of Things. GIoTS 2022. Lecture Notes in Computer Science, vol 13533. Springer, Cham. https://doi.org/10.1007/978-3-031-20936-9_7
Download citation
DOI: https://doi.org/10.1007/978-3-031-20936-9_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20935-2
Online ISBN: 978-3-031-20936-9
eBook Packages: Computer ScienceComputer Science (R0)