Abstract
In view of the existing computation offloading research on fog computing network scenarios, most scenarios focus on reducing energy consumption and delay and lack the joint consideration of smart device rechargeability. This paper proposes a deep deterministic policy gradient-based intelligent rechargeable fog computation offloading mechanism that is combined with simultaneous wireless information and power transfer technology. Specifically, an optimization problem that minimizes the total energy consumption for completing all tasks in a multiuser scenario is formulated, and the joint optimization of the task offloading ratio, uplink channel bandwidth, power split ratio and computing resource allocation is fully considered. Based on the above nonconvex optimization problem with a continuous action space, a communication, computation and energy harvesting co-aware intelligent computation offloading algorithm is developed. It can achieve the optimal energy consumption and delay, and similar to a double deep Q-network, an inverting gradient updating-based dual actor-critic neural network design can improve the convergence and stability of the training process. Finally, the simulation results validate that the proposed mechanism can converge quickly and can effectively reduce the energy consumption with the lowest task delay.







Similar content being viewed by others
References
Cao, B., Li, Y., Zhang, L., Zhang, L., Mumtaz, S., Zhou, Z., & Peng, M. (2019). When Internet of Things meets blockchain: Challenges in distributed consensus. IEEE Network, 33(6), 133–139.
Zhao, J., Li, Q., Gong, Y., & Zhang, K. (2019). Computation offloading and resource allocation for cloud assisted mobile edge computing in vehicular networks. IEEE Transactions on Vehicular Technology, 68(8), 7944–7956.
Cao, B., Zhang, L., Li, Y., Feng, D., & Cao, W. (2019). Intelligent offloading in multi-access edge computing: A state-of-the-art review and framework. IEEE Communications Magazine, 57(3), 56–62.
Chen, L., Zhou, P., Gao, L., & Xu, J. (2018). Adaptive fog configuration for the industrial Internet of Things. IEEE Transactions on Industrial Informatics, 14(10), 4656–4664.
Xiang, H., Peng, M., Sun, Y., & Yan, S. (2020). Mode selection and resource allocation in sliced fog radio access networks: A reinforcement learning approach. IEEE Transactions on Vehicular Technology, 69(4), 4271–4284.
Fang, F., Wang, K., Ding, Z., & Leung, V. C. (2021). Energy-efficient resource allocation for NOMA-MEC networks with imperfect CSI. IEEE Transactions on Communications, 69(5), 3436–3449.
Liu, L., Chang, Z., Guo, X., Mao, S., & Ristaniemi, T. (2018). Multiobjective optimization for computation offloading in fog computing. IEEE Internet of Things Journal, 5(1), 283–294.
Zhao, Z., Bu, S., Zhao, T., Yin, Z., Peng, M., Ding, Z., & Quek, T. Q. (2019). On the design of computation offloading in fog radio access networks. IEEE Transactions on Vehicular Technology, 68(7), 7136–7149.
Zhang, L., Cao, B., Li, Y., Peng, M., & Feng, G. (2021). A multi-stage stochastic programming-based offloading policy for fog enabled IoT-eHealth. IEEE Journal on Selected Areas in Communications, 39(2), 411–425.
Liu, Y., Yu, F. R., Li, X., Ji, H., & Leung, V. C. (2018). Distributed resource allocation and computation offloading in fog and cloud networks with non-orthogonal multiple access. IEEE Transactions on Vehicular Technology, 67(12), 12137–12151.
Chen, S., You, Z., & Ruan, X. (2020). Privacy and energy co-aware data aggregation computation offloading for fog-assisted IoT networks. IEEE Access, 8, 72424–72434.
Tan, L., Hu, R., & Hanzo, L. (2019). Twin-timescale artificial intelligence aided mobility-aware edge caching and computing in vehicular networks. IEEE Transactions on Vehicular Technology, 68(4), 3086–3099.
Wei, Y., Yu, F. R., Song, M., & Han, Z. (2019). Joint optimization of caching, computing, and radio resources for fog-enabled IoT using natural actor-critic deep reinforcement learning. IEEE Internet of Things Journal, 6(2), 2061–2073.
Volodymyr, M., Koray, K., David, S., Rusu, A. A., Veness, J., Bellemare, M. G., et al. (2015). Human-level control through deep reinforcement learning. Nature, 518(7540), 529–533.
Wu, C., Yoshinaga, T., Ji, Y., Murase, T., & Zhang, Y. (2017). A reinforcement learning-based data storage scheme for vehicular ad hoc networks. IEEE Transactions on Vehicular Technology, 66(7), 6336–6348.
Lei, L., Xu, H., Xiong, X., Zheng, K., Xiang, W., & Wang, X. (2019). Multi-user resource control with deep reinforcement learning in IoT edge computing. IEEE Internet of Things Journal, 6(6), 10119–10133.
Chen, X., Zhang, H., Wu, C., Mao, S., Ji, Y., & Bennis, M. (2019). Optimized computation offloading performance in virtual edge computing systems via deep reinforcement learning. IEEE Internet of Things Journal, 6(3), 4005–4018.
Huang, L., Bi, S., & Zhang, Y. (2020). Deep reinforcement learning for online computation offloading in wireless powered mobile-edge computing networks. IEEE Transactions on Mobile Computing, 19(11), 2581–2593.
Liu, Y., Yu, H., Xie, S., & Zhang, Y. (2019). Deep reinforcement learning for offloading and resource allocation in vehicle edge computing and networks. IEEE Transactions on Vehicular Technology, 68(11), 11158–11168.
Chen, S., Chen, J., & Zhao, C. (2021). Deep reinforcement learning based cloud-edge collaborative computation offloading mechanism. Tien Tzu Hsueh Pao, 49(1), 157–166.
Zhou, F., & Hu, R. Q. (2020). Computation efficiency maximization in wireless-powered mobile edge computing networks. IEEE Transactions on Wireless Communications, 19(5), 3170–3184.
Chen, S., Zheng, Y., Wang, K., & Lu, W. (2019). Delay guaranteed energy-efficient computation offloading for industrial IoT in fog computing. In Proceedings of the IEEE international conference on communications (ICC) (pp. 1–6).
Funding
Funding was provided by National Natural Science Foundation of China (Grant No. 61971235), China Postdoctoral Science Foundation (Grant No. 2018M630590), Jiangsu Planned Projects for Postdoctoral Research Funds (Grant No. 2021K501C), 333 High-level Talents Training Project of Jiangsu Province, 1311 Talents Plan of NJUPT, and Postgraduate Research & Practice Innovation Program of Jiangsu Province (Grant No. KYCX22_1017).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Chen, S., Ge, X., Wang, Q. et al. DDPG-based intelligent rechargeable fog computation offloading for IoT. Wireless Netw 28, 3293–3304 (2022). https://doi.org/10.1007/s11276-022-03054-1
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11276-022-03054-1