Skip to main content

Advertisement

Springer Nature Link
Log in

An optimized task offloading strategy based on deep reinforcement learning combined with channel reliability prediction

  • Original Paper
  • Published:
Wireless Networks Aims and scope Submit manuscript

Abstract

Mobile edge computing provides a new solution to meet the computing demands of emerging applications such as the industrial Internet, which cannot be fully met by the resources on the device side. However, the edge nodes and transmission channels are not always completely reliable. When the network channel is unreliable, it is easy to cause task transmission failure and lower service quality. In this paper, an optimized task offloading strategy based on deep reinforcement learning combined with channel reliability prediction is proposed, named deep deterministic policy gradient-based strategy combined with hindsight experience replay and LSTM (HL-DDPG). The HL-DDPG strategy uses long short-term memory (LSTM) to mine the time dependence between the channel reliability states. The task offloading is then modeled using the Markov decision process (MDP), and the joint optimization problem is solved using the DRL method based on the Actor-Critic framework. Meanwhile, Hindsight experience replay (HER) is used to improve the learning ability of the algorithm. The experimental results show that compared with four baseline algorithms, HL-DDPG has a lower overall offloading error probability and a lower task timeout rate, which effectively improves the reliability of the edge offloading system and reduces the risk of task transmission failure.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Explore related subjects

Discover the latest articles and news from researchers in related subjects, suggested using machine learning.

References

  1. Deng, X., Yin, J., Guan, P., Xiong, N. N., Zhang, L., & Mumtaz, S. (2021). Intelligent delay-aware partial computing task offloading for multi-user industrial Internet of Things through edge computing. IEEE Internet of Things Journal, 10(4), 2954–2966.

    Article  MATH  Google Scholar 

  2. Zhou, H., Jiang, K., Liu, X., Li, X., & Leung, V. C. (2021). Deep reinforcement learning for energy-efficient computation offloading in mobile-edge computing. IEEE Internet of Things Journal, 9(2), 1517–1530.

    Article  MATH  Google Scholar 

  3. Tu, Y., Chen, H., Yan, L., & Zhou, X. (2022). Task offloading based on LSTM prediction and deep reinforcement learning for efficient edge computing in IoT. Future Internet, 14(2), 30.

    Article  MATH  Google Scholar 

  4. Gong, H., Ge, H., Ma, S., Sun, A., Chen, X., & Liu, L. (2022, March). Task Offloading Strategy Based on TD3 Algorithm in Cloud-Edge Collaborative MEC. In: 2022 4th International Conference on Natural Language Processing (ICNLP) (pp. 452–459). IEEE

  5. Shinde, S. S., & Tarchi, D. (2023). A Markov Decision Process Solution for Energy-Saving Network Selection and Computation Offloading in Vehicular Networks. IEEE Transactions on Vehicular Technology.

  6. Ladosz, P., Weng, L., Kim, M., & Oh, H. (2022). Exploration in deep reinforcement learning: A survey. Information Fusion, 85, 1–22.

    Article  MATH  Google Scholar 

  7. Li, H., Wei, T., Ren, A., Zhu, Q., & Wang, Y. (2017, November). Deep reinforcement learning: Framework, applications, and embedded implementations. In: 2017 IEEE/ACM International Conference on Computer-Aided Design (ICCAD) (pp. 847–854). IEEE

  8. Alam, M. G. R., Hassan, M. M., Uddin, M. Z., Almogren, A., & Fortino, G. (2019). Autonomic computation offloading in mobile edge for IoT applications. Future Generation Computer Systems, 90, 149–157.

    Article  Google Scholar 

  9. Qian, Y., Hu, L., Chen, J., Guan, X., Hassan, M. M., & Alelaiwi, A. (2019). Privacy-aware service placement for mobile edge computing via federated learning. Information Sciences, 505, 562–570.

    Article  Google Scholar 

  10. Wang, T., Liang, Y., Zhang, Y., Zheng, X., Arif, M., Wang, J., & Jin, Q. (2020). An intelligent dynamic offloading from cloud to edge for smart iot systems with big data. IEEE Transactions on Network Science and Engineering, 7(4), 2598–2607.

    Article  MATH  Google Scholar 

  11. Wang, S., Chen, M., Yin, C., Saad, W., Hong, C. S., Cui, S., & Poor, H. V. (2021). Federated learning for task and resource allocation in wireless high-altitude balloon networks. IEEE Internet of Things Journal, 8(24), 17460–17475.

    Article  MATH  Google Scholar 

  12. Liu, X., Jiang, J., & Li, L. (2021, October). Computation Offloading and Task Scheduling with Fault-Tolerance for Minimizing Redundancy in Edge Computing. In: 2021 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW) (pp. 198–209). IEEE.

  13. Min, M., Xiao, L., Chen, Y., Cheng, P., Wu, D., & Zhuang, W. (2019). Learning-based computation offloading for IoT devices with energy harvesting. IEEE Transactions on Vehicular Technology, 68(2), 1930–1941.

    Article  MATH  Google Scholar 

  14. Bian, Yuwei, et al. Dependency-Aware Task Scheduling and Offloading Scheme based on Graph Neural Network For MEC-Assisted Network. In: 2023 IEEE/CIC International Conference on Communications in China (ICCC Workshops). IEEE, 2023.

  15. Gu, L., & Mohajer, A. (2024). Joint throughput maximization, interference cancellation, and power efficiency for multi-IRS-empowered UAV communications. Signal, Image and Video Processing, 18(5), 4029–4043.

    Article  MATH  Google Scholar 

  16. Liu, W., Li, B., Xie, W., et al. (2023). Energy efficient computation offloading in aerial edge networks with multi-agent cooperation[J]. IEEE Transactions on Wireless Communications, 22, 5725–5739.

    Article  MATH  Google Scholar 

  17. Pu, X., Lei, T., Wen, W., et al. (2023). Incentive mechanism and resource allocation for collaborative task offloading in energy-efficient mobile edge computing[J]. IEEE Transactions on Vehicular Technology, 72, 13775–13780.

    Article  Google Scholar 

  18. Wu, G., Xu, Z., Zhang, H., et al. (2023). Multi-agent DRL for joint completion delay and energy consumption with queuing theory in MEC-based IIoT[J]. Journal of Parallel and Distributed Computing, 176, 80–94.

    Article  MATH  Google Scholar 

  19. Wang, Q., Li, W., & Mohajer, A. (2024). Load-aware continuous-time optimization for multi-agent systems: Toward dynamic resource allocation and real-time adaptability. Computer Networks, 250, 110526.

    Article  MATH  Google Scholar 

  20. Yang, T., Sun, J., & Mohajer, A. (2024). Queue stability and dynamic throughput maximization in multi-agent heterogeneous wireless networks. Wireless Networks, 30, 1–27.

    Article  MATH  Google Scholar 

  21. Liu, C. F., Bennis, M., Debbah, M., & Poor, H. V. (2019). Dynamic task offloading and resource allocation for ultra-reliable low-latency edge computing. IEEE Transactions on Communications, 67(6), 4132–4150.

    Article  MATH  Google Scholar 

  22. Merluzzi, M., di Pietro, N., Di Lorenzo, P., Strinati, E. C., & Barbarossa, S. (2019, December). Network energy efficient mobile edge computing with reliability guarantees. In: 2019 IEEE Global Communications Conference (GLOBECOM) (pp. 1–6). IEEE.

  23. Dong, L., Wu, W., Guo, Q., Satpute, M. N., Znati, T., & Du, D. Z. (2019). Reliability-aware offloading and allocation in multilevel edge computing system. IEEE Transactions on Reliability, 70(1), 200–211.

    Article  Google Scholar 

  24. Wu, H. (2018). Performance modeling of delayed offloading in mobile wireless environments with failures [J]. IEEE Communications Letters, 22(11), 2334–2337.

    Article  MATH  Google Scholar 

  25. Tang, W., Gao, D., Siyu, Yu., Jianbo, Lu., Wei, Z., Li, Z., & Chen, N. (2024). Reliable and adaptive computation offload strategy with load and cost coordination for edge computing. Pervasive and Mobile Computing, 102, 101932.

    Article  MATH  Google Scholar 

  26. Wenxiu, Xu., Yin, Y., Chen, N., & Huan, Tu. (2023). Collaborative inference acceleration integrating DNN partitioning and task offloading in mobile edge computing. International Journal of Software Engineering and Knowledge Engineering, 33(11&12), 1835–1863.

    Google Scholar 

  27. Wang, J., Liu, K., Li, B., Liu, T., Li, R., & Han, Z. (2019). Delay-sensitive multi-period computation offloading with reliability guarantees in fog networks. IEEE Transactions on Mobile Computing, 19(9), 2062–2075.

    Article  MATH  Google Scholar 

  28. Sadek, A. K., Han, Z., & Liu, K. R. (2009). Distributed relay-assignment protocols for coverage expansion in cooperative wireless networks. IEEE Transactions on Mobile Computing, 9(4), 505–515.

    Article  MATH  Google Scholar 

  29. Hazarika, B., Singh, K., Biswas, S., & Li, C. P. (2022). DRL-based resource allocation for computation offloading in IoV networks. IEEE Transactions on Industrial Informatics, 18(11), 8027–8038.

    Article  MATH  Google Scholar 

  30. You, C., Huang, K., Chae, H., & Kim, B. H. (2016). Energy-efficient resource allocation for mobile-edge computation offloading. IEEE Transactions on Wireless Communications, 16(3), 1397–1411.

    Article  MATH  Google Scholar 

  31. Tang, M., & Wong, V. W. (2020). Deep reinforcement learning for task offloading in mobile edge computing systems. IEEE Transactions on Mobile Computing, 21(6), 1985–1997.

    Article  MATH  Google Scholar 

  32. Chen, X., Ge, H., Liu, L., Li, S., Han, J., & Gong, H. (2021, April). Computing offloading decision based on DDPG algorithm in mobile edge computing. In: 2021 IEEE 6th International Conference on Cloud Computing and Big Data Analytics (ICCCBDA) (pp. 391–399). IEEE

Download references

Funding

This study was funded by the National Natural Science Foundation of China (No.62162003). This study was funded by the Nanning Science and Technology project (No. 20221031).

Author information

Authors and Affiliations

  1. School of Computer and Electronic Information, Guangxi University, Nanning, 530004, China

    Weicheng Tang, Yubin Yang, Donghui Gao, Juan Chen, Suqun Huang & Ningjiang Chen

  2. Key Laboratory of Parallel, Distributed and Intelligent Computing (Guangxi University), Education Department of Guangxi Zhuang Autonomous Region, Nanning, 530004, China

    Ningjiang Chen

  3. Guangxi Intelligent Digital Services Research Center of Engineering Technology, Nanning, 530004, China

    Ningjiang Chen

  4. School of Information Technology, Guangxi Police College, Nanning, 530004, China

    Weicheng Tang

Authors
  1. Weicheng Tang
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Yubin Yang
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Donghui Gao
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Juan Chen
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Suqun Huang
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Ningjiang Chen
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Ningjiang Chen.

Ethics declarations

Conflict of interest

Author Weicheng Tang declares that he has no conflict of interest. Author Yubin Yang declares that he has no conflict of interest. Author Donghui Gao declares that he has no conflict of interest. Author Juan Chen declares that he has no conflict of interest. Author Suqun Huang declares that he has no conflict of interest. Author Ningjiang Chen declares that he has no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tang, W., Yang, Y., Gao, D. et al. An optimized task offloading strategy based on deep reinforcement learning combined with channel reliability prediction. Wireless Netw 31, 1663–1682 (2025). https://doi.org/10.1007/s11276-024-03838-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11276-024-03838-7

Keywords