Kexin Li
ABSTRACT
Recently, researchers have focused on a new paradigm called Multi-access Edge Computing (MEC), which ensures reduce the execution time for computation constrained edge devices through computation offloading techniques. Despite the exciting research on optimizing centralized task scheduling problems in the MEC architecture bringing several advantages, a few challenges, such as efficiently optimizing algorithms in dynamic and large-scale networks, are still a conundrum. This article investigates a computation offloading scheduling problem in a dynamic network. The tasks generated by the edge devices can be executed locally or sent to edge servers for execution based on the time-varying MEC network. Moreover, we formulate the computation offloading optimization problem as a Markov Decision Process (MDP) model. Furthermore, to improve the learning and convergence efficiency, we propose an Experience-Based replay Reinforcement Learning algorithm (EBRL) by collecting significant transformations and leveraging the most valuable knowledge from the experience pool. Experimental results show that our proposed algorithm effectively achieves faster convergence speed and reduces the system delay than other benchmarks in a dynamic MEC network.
Supplemental Material
- S. Bi and Yja Zhang. 2018. An ADMM Based Method for Computation Rate Maximization in Wireless Powered Mobile-Edge Computing Networks. In 2018 IEEE International Conference on Communications (ICC).Google Scholar
- T. D. Bruin, J. Kober, K. Tuyls, and R. Babuska. 2018. Experience Selection in Deep Reinforcement Learning for Control. Journal of Machine Learning Research 19 (2018), 1–56.Google Scholar
- [3] Xianfu Chen, Honggang Zhang, Celimuge Wu, Shiwen Mao, Yusheng Ji, and Medhi Bennis.[n.d.]. ([n. d.]).Google Scholar
- Hossein Ebrahimi Dinaki, Shervin Shirmohammadi, and Mahmoud Reza Hashemi. 2020. Boosted Metaheuristic Algorithms for QoE-Aware Server Selection in Multiplayer Cloud Gaming. IEEE Access 8(2020), 60468–60483. https://doi.org/10.1109/ACCESS.2020.2983080Google ScholarCross Ref
- Jianbo Du, Liqiang Zhao, Jie Feng, and Xiaoli Chu. 2018. Computation Offloading and Resource Allocation in Mixed Fog/Cloud Computing Systems With Min-Max Fairness Guarantee. IEEE Transactions on Communications 66, 4 (2018), 1594–1608. https://doi.org/10.1109/TCOMM.2017.2787700Google ScholarCross Ref
- W. Fang, S. Ding, Y. Li, W. Zhou, and N. Xiong. 2019. OKRA: optimal task and resource allocation for energy minimization in mobile edge computing systems. Wireless Networks 25, 5 (2019), 2851–2867.Google ScholarDigital Library
- J. M. Hancock. 2004. Jaccard Distance (Jaccard Index, Jaccard Similarity Coefficient). Dictionary of Bioinformatics and Computational Biology.Google Scholar
- Liang Huang, Suzhi Bi, and Ying-Jun Angela Zhang. 2020. Deep Reinforcement Learning for Online Computation Offloading in Wireless Powered Mobile-Edge Computing Networks. IEEE Transactions on Mobile Computing 19, 11 (2020), 2581–2593. https://doi.org/10.1109/TMC.2019.2928811Google ScholarCross Ref
- Shaik Hussain and Kamaludin Yusof. 2021. Dynamic Q-learning and Fuzzy CNN Based Vertical Handover Decision for Integration of DSRC, mmWave 5G and LTE in Internet of Vehicles (IoV). Journal of Communications 16 (01 2021), 155–166. https://doi.org/10.12720/jcm.16.5.155-166Google Scholar
- L. Lei, H. Xu, X. Xiong, K. Zheng, W. Xiang, and X. Wang. 2019. Multi-user Resource Control with Deep Reinforcement Learning in IoT Edge Computing. IEEE Internet of Things Journal 6, 6 (2019), 10119–10133.Google ScholarCross Ref
- T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra. 2015. Continuous control with deep reinforcement learning. Computer ence (2015).Google Scholar
- Haodong Lu, Xiaoming He, Miao Du, Xiukai Ruan, Yanfei Sun, and Kun Wang. 2020. Edge QoE: Computation Offloading With Deep Reinforcement Learning for Internet of Things. IEEE Internet of Things Journal 7, 10 (2020), 9255–9265. https://doi.org/10.1109/JIOT.2020.2981557Google ScholarCross Ref
- Pavel Mach and Zdenek Becvar. 2017. Mobile Edge Computing: A Survey on Architecture and Computation Offloading. IEEE Communications Surveys Tutorials 19, 3 (2017), 1628–1656. https://doi.org/10.1109/COMST.2017.2682318Google ScholarDigital Library
- Y. Mao, J. Zhang, and K. B. Letaief. 2016. Dynamic Computation Offloading for Mobile-Edge Computing With Energy Harvesting Devices. IEEE Journal on Selected Areas in Communications 34, 12(2016), 3590–3605.Google ScholarDigital Library
- Zhaolong Ning, Peiran Dong, Xiangjie Kong, and Feng Xia. 2019. A Cooperative Partial Computation Offloading Scheme for Mobile Edge Computing Enabled Internet of Things. IEEE Internet of Things Journal 6, 3 (2019), 4804–4814. https://doi.org/10.1109/JIOT.2018.2868616Google ScholarCross Ref
- Fernaz Narin Nur, Saiful Islam, Nazmun Nessa Moon, Asif Karim, Sami Azam, and Bharanidharan Shanmugam. 2019. Priority-Based Offloading and Caching in Mobile Edge Cloud. Journal of Communications Software and Systems 15, 2 (4 2019), 193–201. https://doi.org/10.24138/jcomss.v15i2.707Google ScholarCross Ref
- Tadashi Ogino, Shinji Kitagami, and Norio Shiratori. 2019. A Multi-agent Based Flexible IoT Edge Computing Architecture and Application to ITS. J. Commun. 14(2019), 47–52.Google ScholarCross Ref
- Shengli Pan, Zhiyong Zhang, Zongwang Zhang, and Deze Zeng. 2019. Dependency-Aware Computation Offloading in Mobile Edge Computing: A Reinforcement Learning Approach. IEEE Access 7(2019), 134742–134753. https://doi.org/10.1109/ACCESS.2019.2942052Google ScholarCross Ref
- Chandan Pradhan, Ang Li, Changyang She, Yonghui Li, and Branka Vucetic. 2020. Computation Offloading for IoT in C-RAN: Optimization and Deep Learning. IEEE Transactions on Communications 68, 7 (2020), 4565–4579. https://doi.org/10.1109/TCOMM.2020.2983142Google ScholarCross Ref
- Hurmat Ali Shah, Lian Zhao, and Il-Min Kim. 2021. Joint Network Control and Resource Allocation for Space-Terrestrial Integrated Network Through Hierarchal Deep Actor-Critic Reinforcement Learning. IEEE Transactions on Vehicular Technology 70, 5 (2021), 4943–4954. https://doi.org/10.1109/TVT.2021.3071983Google ScholarCross Ref
- N. Sharma and K. R. Anupama. 2011. On the use of NSGA-II for multi-objective resource allocation in MIMO-OFDMA systems. Wireless Networks 17, 5 (2011), 1191–1201.Google ScholarDigital Library
- Ming Tang and Vincent W.S. Wong. 2020. Deep Reinforcement Learning for Task Offloading in Mobile Edge Computing Systems. IEEE Transactions on Mobile Computing(2020), 1–1. https://doi.org/10.1109/TMC.2020.3036871Google Scholar
- Jun Wang, Daquan Feng, Shengli Zhang, An Liu, and Xiang-Gen Xia. 2021. Joint Computation Offloading and Resource Allocation for MEC-Enabled IoT Systems With Imperfect CSI. IEEE Internet of Things Journal 8, 5 (2021), 3462–3475. https://doi.org/10.1109/JIOT.2020.3022802Google ScholarCross Ref
- Xiaojie Wang, Zhaolong Ning, and Song Guo. 2021. Multi-Agent Imitation Learning for Pervasive Edge Computing: A Decentralized Computation Offloading Algorithm. IEEE Transactions on Parallel and Distributed Systems 32, 2 (2021), 411–425. https://doi.org/10.1109/TPDS.2020.3023936Google ScholarCross Ref
- Huiqiang Xie and Zhijin Qin. 2021. A Lite Distributed Semantic Communication System for Internet of Things. IEEE Journal on Selected Areas in Communications 39, 1(2021), 142–153. https://doi.org/10.1109/JSAC.2020.3036968Google ScholarDigital Library
- Xi Zhang and Qixuan Zhu. 2019. Game-Theory Based Power and Spectrum Virtualization for Optimizing Spectrum Efficiency in Mobile Cloud-Computing Wireless Networks. IEEE Transactions on Cloud Computing 7, 4 (2019), 1025–1038. https://doi.org/10.1109/TCC.2017.2727044Google ScholarCross Ref
Index Terms
- Experience-based Computation Offloading by Deep Reinforcement Learning for Multi-access Edge Computing Network
Recommendations
Incentive mechanism for computation offloading using edge computing
IoT-based services benefit from cloud which offers a virtually unlimited capabilities, such as storage, processing, and communication. However, the challenges are still open for mobile users to receive computation from the cloud with satisfied quality-...
Learning for Smart Edge: Cognitive Learning-Based Computation Offloading
AbstractWith the development of intelligent applications, more and more intelligent applications are computation intensive, data intensive and delay sensitive. Compared with traditional cloud computing, edge computing can reduce communication delay by ...
Optimization for computational offloading in multi-access edge computing: A deep reinforcement learning scheme
AbstractOwing to their limited computing power and battery level, wireless users (WUs) can hardly handle compute-intensive workflows by the local processor. Multi-access edge computing (MEC) servers attached to base stations have ample ...
Graphical abstractDisplay Omitted
Highlights- A multi-MEC cooperation framework comprising multiple multi-access edge computing server for computation offloading.
Comments