Abstract
Mobile Edge Computing (MEC) is regarded as a promising paradigm for reducing service latency in Mobile users data processing by providing computing resources at the network edge. Existing deep reinforcement learning (DRL) algorithms struggle to effectively handle the joint optimization of computational offloading and resource allocation (JCORA). To overcome this challenge, we propose a Long Short-Term Deterministic Policy Gradient (LSTDPG) approach to tackle JCORA. Building upon the Deep Deterministic Policy Gradients (DDPG) algorithm, LSTDPG incorporates two key features. Firstly, it utilizes a Temporal Attention Network composed of Long Short-Term Memory (LSTM) networks, which facilitates high-quality state representation and function approximation. Secondly, an Episode-Based Prioritized Experience Replay (ePER) method is introduced to expedite and stabilize the convergence of model training. Experimental results demonstrate that the proposed LSTDPG outperforms several state-of-the-art DRL agents in terms of task completion time and energy consumption.
This work is supported by Heilongjiang Provincial Science and Technology Program (No. 2022ZX01A16) and Sichuan Science and Technology Program (No. 2022 YFG0148).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Hassan, N., Yau, K.L.A., Wu, C.: Edge computing in 5G: a review. IEEE Access 7, 127276–127289 (2019)
Chen, Y., Zhang, N., Zhang, Y., Chen, X., Wu, W., Shen, X.: Energy efficient dynamic offloading in mobile edge computing for internet of things. IEEE Trans. Cloud Comput. 9(3), 1050–1060 (2019)
Jošilo, S., Dán, G.: Wireless and computing resource allocation for selfish computation offloading in edge computing. In: IEEE INFOCOM 2019-IEEE Conference on Computer Communications, pp. 2467–2475. IEEE (2019)
Xiong, X., Zheng, K., Lei, L., Hou, L.: Resource allocation based on deep reinforcement learning in IoT edge computing. IEEE J. Sel. Areas Commun. 38(6), 1133–1146 (2020)
Tran, T.X., Pompili, D.: Joint task offloading and resource allocation for multi-server mobile-edge computing networks. IEEE Trans. Veh. Technol. 68(1), 856–868 (2018)
Lu, H., He, X., Du, M., Ruan, X., Sun, Y., Wang, K.: Edge QoE: computation offloading with deep reinforcement learning for internet of things. IEEE Internet Things J. 7(10), 9255–9265 (2020)
Bi, S., Huang, L., Zhang, Y.J.A.: Joint optimization of service caching placement and computation offloading in mobile edge computing systems. IEEE Trans. Wireless Commun. 19(7), 4947–4963 (2020)
Ahani, G., Yuan, D.: BS-assisted task offloading for D2D networks with presence of user mobility. In: 2019 IEEE 89th Vehicular Technology Conference (VTC2019-Spring), pp. 1–5. IEEE (2019)
Wang, S., Hu, Z., Deng, Y., Hu, L.: Game-theory-based task offloading and resource scheduling in cloud-edge collaborative systems. Appl. Sci. 12(12), 6154 (2022)
He, Q., et al.: A game-theoretical approach for user allocation in edge computing environment. IEEE Trans. Parallel Distrib. Syst. 31(3), 515–529 (2019)
Vijayaram, B., Vasudevan, V.: Wireless edge device intelligent task offloading in mobile edge computing using hyper-heuristics. EURASIP J. Adv. Sign. Proc. 2022(1), 1–23 (2022)
LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)
Zhao, X., et al.: Deep learning based mobile data offloading in mobile edge computing systems. Futur. Gener. Comput. Syst. 99, 346–355 (2019)
Sadiki, A., Bentahar, J., Dssouli, R., En-Nouaary, A., Otrok, H.: Deep reinforcement learning for the computation offloading in MIMO-based edge computing. Ad Hoc Netw. 141, 103080 (2023)
Lin, B., Lin, K., Lin, C., Lu, Y., Huang, Z., Chen, X.: Computation offloading strategy based on deep reinforcement learning for connected and autonomous vehicle in vehicular edge computing. J. Cloud Comput. 10(1), 33 (2021)
Cao, S., Chen, S., Chen, H., Zhang, H., Zhan, Z., Zhang, W.: HCOME: research on hybrid computation offloading strategy for MEC based on DDPG. Electronics 12(3), 562 (2023)
Ebrahim, M.A., Ebrahim, G.A., Mohamed, H.K., Abdellatif, S.O.: A deep learning approach for task offloading in Multi-UAV aided mobile edge computing. IEEE Access 10, 101716–101731 (2022)
Chen, Y., Han, S., Chen, G., Yin, J., Wang, K.N., Cao, J.: A deep reinforcement learning-based wireless body area network offloading optimization strategy for healthcare services. Health Inf. Sci. Syst. 11(1), 8 (2023)
Hao, Y., Ni, Q., Li, H., Hou, S.: Energy-efficient multi-user mobile-edge computation offloading in massive MIMO enabled HetNets. In: ICC 2019–2019 IEEE International Conference on Communications (ICC), pp. 1–6. IEEE (2019)
Guo, S., Liu, J., Yang, Y., Xiao, B., Li, Z.: Energy-efficient dynamic computation offloading and cooperative task scheduling in mobile cloud computing. IEEE Trans. Mob. Comput. 18(2), 319–333 (2018)
Wang, Y., Sheng, M., Wang, X., Wang, L., Li, J.: Mobile-edge computing: Partial computation offloading using dynamic voltage scaling. IEEE Trans. Commun. 64(10), 4268–4282 (2016)
Graves, A., Graves, A.: Long short-term memory. Supervised sequence labelling with recurrent neural networks, pp. 37–45 (2012)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Schaul, T., Quan, J., Antonoglou, I., Silver, D.: Prioritized experience replay. arXiv preprint: arXiv:1511.05952 (2015)
Harris, C.R., et al.: Array programming with NumPy. Nature 585(7825), 357–362 (2020)
Fujimoto, S., Hoof, H., Meger, D.: Addressing function approximation error in actor-critic methods. In: International Conference on Machine Learning, pp. 1587–1596. PMLR (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Lei, X., Li, Q., Bo, P., Zhou, Y.Z., Chen, C., Peng, S.L. (2024). Long Short-Term Deterministic Policy Gradient for Joint Optimization of Computational Offloading and Resource Allocation in MEC. In: Tari, Z., Li, K., Wu, H. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2023. Lecture Notes in Computer Science, vol 14492. Springer, Singapore. https://doi.org/10.1007/978-981-97-0811-6_20
Download citation
DOI: https://doi.org/10.1007/978-981-97-0811-6_20
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-0810-9
Online ISBN: 978-981-97-0811-6
eBook Packages: Computer ScienceComputer Science (R0)