Abstract
In recent years, numerous Deep Reinforcement Learning (DRL) neural network models have been proposed to optimize computational offloading and resource allocation in Mobile Edge Computing (MEC). However, the diversity of computational tasks and the complexity of 5G networks pose significant challenges for current DRL algorithms apply to MEC scenarios. This research focuses on a single MEC server-multi-user scenario and develops a realistic small-scale MEC offloading system. In order to alleviate the problem of overestimation of action value in current Deep Q-learning Network (DQN), we propose a normalized model of Complex network based on Double DQN (DDQN) algorithm to determine the optimal computational offloading and resource allocation strategy. Simulation results demonstrate that DDQN outperforms conventional approaches such as fixed parameter policies and DQN regarding convergence speed, energy consumption and latency. This research showcases the potential of DDQN for achieving efficient optimization in MEC environments.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Liu, J., Zhang, Q.: Offloading schemes in mobile edge computing for ultra-reliable low latency communications. IEEE Access 6, 12825–12837 (2018). https://doi.org/10.1109/ACCESS.2018.2800032
Yang, J., Shah, A.A., Pezaros, D.: A survey of energy optimization approaches for computational task offloading and resource allocation in MEC networks. Electronics 12(17), 3548 (2023). https://doi.org/10.3390/electronics12173548
Landers, M., Doryab, A.: Deep reinforcement learning verification: a survey. ACM Comput. Surv. 55(14s), Article 330, 31 (2023). https://doi.org/10.1145/3596444
Kumaran, K., Sasikala, E.: Learning based latency minimization techniques in mobile edge computing (MEC) systems: a comprehensive survey. In: 2021 International Conference on System, Computation, Automation and Networking (ICSCAN), Puducherry, India, pp. 1–6 (2021). https://doi.org/10.1109/ICSCAN53069.2021.9526410
Liu, C.-F., Bennis, M., Poor, H.V.: Latency and reliability-aware task offloading and resource allocation for mobile edge computing. In: 2017 IEEE Globe com Workshops (GC Wkshps), Singapore, pp. 1–7 (2017). https://doi.org/10.1109/GLOCOMW.2017.8269175
Dab, B., Aitsaadi, N., Langar, R.: Q-learning algorithm for joint computation offloading and resource allocation in edge cloud. In: 2019 IFIP/IEEE Symposium on Integrated Network and Service Management (IM), pp. 45–52. IEEE (2019)
Huang, L., Feng, X., Zhang, C., et al.: Deep reinforcement learning-based joint task offloading and bandwidth allocation for multi-user mobile edge computing. Digit. Commun. Netw. 5(1), 10–17 (2019)
Liang, Y., He, Y., Zhong, X.: Decentralized computation offloading and resource allocation in MEC by deep reinforcement learning. In: 2020 IEEE/CIC International Conference on Communications in China (ICCC), pp. 244–249. IEEE (2020)
Liang, S., Wan, H., Qin, T., et al.: Multi-user computation offloading for mobile edge computing: A deep reinforcement learning and game theory approach. In: 2020 IEEE 20th International Conference on Communication Technology (ICCT), pp. 1534–1539. IEEE (2020)
Wu, Y.C., Dinh, T.Q., Fu, Y., et al.: A hybrid DQN and optimization approach for strategy and resource allocation in MEC networks. IEEE Trans. Wireless Commun. 20(7), 4282–4295 (2021)
Li, C., Xia, J., Liu, F., et al.: Dynamic offloading for multiuser muti-CAP MEC networks: a deep reinforcement learning approach. IEEE Trans. Veh. Technol. 70(3), 2922–2927 (2021)
Gan, S., Siew, M., Xu, C., et al.: Differentially Private Deep Q-Learning for Pattern Privacy Preservation in MEC Offloading (2023). arXiv preprint arXiv:2302.04608
Silver, D., Huang, A., Maddison, C.J., et al.: Mastering the game of go with deep neural networks and tree search. Nature 529(7587), 484–489 (2016)
Al-Absi, M.A., Al-Absi, A.A., Sain, M., et al.: Moving ad hoc networks—a comparative study. Sustainability 13(11), 6187 (2021)
Jiang, P., Ergu, D., Liu, F., et al.: A review of Yolo algorithm developments. Procedia Comput. Sci. 199, 1066–1073 (2022)
Nath, S., Li, Y., Wu, J., et al.: Multi-user multi-channel computation offloading and resource allocation for mobile edge computing. In: ICC 2020–2020 IEEE International Conference on Communications (ICC), pp. 1–6. IEEE (2020)
Silver, D., et al.: Mastering the game of go with deep neural networks and tree search. Nature 529(7587), 484–489 (2016)
Hao, W., Yang, S.: Small cell cluster-based resource allocation for wireless backhaul in two-tier heterogeneous networks with massive MIMO. IEEE Trans. Veh. Technol. 67(1), 509–523 (2017)
Zeng, H., Zhang, M., Xia, Y., et al.: Decoupling the depth and scope of graph neural networks. Adv. Neural. Inf. Process. Syst. 34, 19665–19679 (2021)
Dennis, A.K.: Raspberry Pi Computer Architecture Essentials. Packt Publishing Ltd., Birmingham (2016)
Acknowledgments
This research was supported in part by the Inner Mongolia Science and Technology Key Project No. 2021GG0218, ROIS NII Open Collaborative Research 23S0601, and in part by JSPS KAKENHI Grant No. 21H03424.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Zhang, C., Peng, C., Lin, M., Du, Z., Wu, C. (2024). Double DQN Reinforcement Learning-Based Computational Offloading and Resource Allocation for MEC. In: Wu, C., Chen, X., Feng, J., Wu, Z. (eds) Mobile Networks and Management. MONAMI 2023. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 559. Springer, Cham. https://doi.org/10.1007/978-3-031-55471-1_18
Download citation
DOI: https://doi.org/10.1007/978-3-031-55471-1_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-55470-4
Online ISBN: 978-3-031-55471-1
eBook Packages: Computer ScienceComputer Science (R0)