skip to main content
research-article

AI-Enabled Task Offloading for Improving Quality of Computational Experience in Ultra Dense Networks

Published: 14 March 2022 Publication History

Abstract

Multi-access edge computing (MEC) and ultra-dense networking (UDN) are recognized as two promising paradigms for future mobile networks that can be utilized to improve the spectrum efficiency and the quality of computational experience (QoCE). In this paper, we study the task offloading problem in an MEC-enabled UDN architecture with the aim to minimize the task duration while satisfying the energy budget constraints. Due to the dynamics associated with the environment and parameter uncertainty, designing an optimal task offloading algorithm is highly challenging. Consequently, we propose an online task offloading algorithm based on a state-of-the-art deep reinforcement learning (DRL) technique: asynchronous advantage actor-critic (A3C). It is worthy of remark that the proposed method requires neither instantaneous channel state information (CSI) nor prior knowledge of the computational capabilities of the base stations. Simulations show that the our method is able to learn a good offloading policy to obtain a near-optimal task allocation while meeting energy budget constraints of mobile devices in the UDN environment.

References

[1]
A. Anjum, T. Abdullah, M. F. Tariq, Y. Baltaci, and N. Antonopoulos. 2019. Video stream analysis in clouds: An object detection and classification framework for high performance video analytics. IEEE Transactions on Cloud Computing 7, 4 (2019), 1152–1167.
[2]
Min Chen and Yixue Hao. 2018. Task offloading for mobile edge computing in software defined ultra-dense network. IEEE Journal on Selected Areas in Communications 36, 3 (2018), 587–597.
[3]
Xianfu Chen, Honggang Zhang, Celimuge Wu, Shiwen Mao, Yusheng Ji, and Medhi Bennis. 2019. Optimized computation offloading performance in virtual edge computing systems via deep reinforcement learning. IEEE Internet of Things Journal 6, 3 (2019), 4005–4018. arxiv:1805.06146
[4]
X. Chen, Z. Zhao, C. Wu, M. Bennis, H. Liu, Y. Ji, and H. Zhang. 2019. Multi-tenant cross-slice resource orchestration: A deep reinforcement learning approach. IEEE Journal on Selected Areas in Communications 37, 10 (2019), 2377–2392.
[5]
Cisco. March 2020. Cisco annual internet report (2018–2023) white paper. (March 2020).
[6]
R. Deng, R. Lu, C. Lai, T. H. Luan, and H. Liang. 2016. Optimal workload allocation in fog-cloud computing toward balanced delay and power consumption. IEEE Internet of Things Journal 3, 6 (Dec 2016), 1171–1181.
[7]
Thinh Quang Dinh, Jianhua Tang, Quang Duy La, and Tony Q. S. Quek. 2017. Offloading in mobile edge computing: Task allocation and computational frequency scaling. IEEE Transactions on Communications 65, 8 (2017), 3571–3584.
[8]
ETSI. 2016. Mobile Edge Computing: Technical Requirements. (Mar. 2016).
[9]
Geoffrey Hinton, Nitish Srivastava, and Kevin Swersky. 2012. Neural networks for machine learning lecture 6a overview of mini-batch gradient descent. Cited on 14, 8 (2012).
[10]
Liang Huang, Suzhi Bi, and Ying Jun Zhang. 2019. Deep reinforcement learning for online computation offloading in wireless powered mobile-edge computing networks. IEEE Transactions on Mobile Computing 518060 (2019), 1–1. arxiv:1808.01977
[11]
C. Kai, H. Li, L. Xu, Y. Li, and T. Jiang. 2019. Joint subcarrier assignment with power allocation for sum rate maximization of D2D communications in wireless cellular networks. IEEE Transactions on Vehicular Technology 68, 5 (2019), 4748–4759.
[12]
S. Ko, K. Han, and K. Huang. 2018. Wireless networks for mobile edge computing: Spatial modeling and latency analysis. IEEE Transactions on Wireless Communications 17, 8 (Aug. 2018), 5225–5240.
[13]
Gang Li and Jun Cai. 2020. An online incentive mechanism for collaborative task offloading in mobile edge computing. IEEE Transactions on Wireless Communications 19, 1 (2020), 624–636.
[14]
Chen Feng Liu, Mehdi Bennis, Merouane Debbah, and H. Vincent Poor. 2019. Dynamic task offloading and resource allocation for ultra-reliable low-latency edge computing. IEEE Transactions on Communications 67, 6 (2019), 4132–4150. arxiv:1812.08076
[15]
Nguyen Cong Luong, Dinh Thai Hoang, Shimin Gong, Dusit Niyato, Ping Wang, Ying Chang Liang, and Dong In Kim. 2019. Applications of deep reinforcement learning in communications and networking: A survey. IEEE Communications Surveys and Tutorials 21, 4 (2019), 3133–3174. arxiv:1810.07862
[16]
Y. Mao, J. Zhang, and K. B. Letaief. 2016. Dynamic computation offloading for mobile-edge computing with energy harvesting devices. IEEE Journal on Selected Areas in Communications 34, 12 (Dec 2016), 3590–3605.
[17]
Y. Mao, J. Zhang, and K. B. Letaief. 2017. Joint task offloading scheduling and transmit power allocation for mobile-edge computing systems. In 2017 IEEE Wireless Communications and Networking Conference (WCNC). 1–6.
[18]
Y. Mao, J. Zhang, S. H. Song, and K. B. Letaief. 2017. Stochastic joint radio and computational resource management for multi-user mobile-edge computing systems. IEEE Transactions on Wireless Communications 16, 9 (Sept. 2017), 5994–6009.
[19]
Peter M. Mell and Timothy Grance. 2011. SP 800-145. The NIST definition of cloud computing. (2011).
[20]
Volodymyr Mnih, Adrià Puigdomènech Badia, Mehdi Mirza, Alex Graves, Timothy P. Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. 2016. Asynchronous methods for deep reinforcement learning. CoRR abs/1602.01783 (2016). arxiv:1602.01783http://arxiv.org/abs/1602.01783.
[21]
Lingjun Pu, Xu Chen, Jingdong Xu, and Xiaoming Fu. 2016. D2D fogging: An energy-efficient and incentive-aware task offloading framework via network-assisted D2D collaboration. IEEE Journal on Selected Areas in Communications 34, 12 (2016), 3887–39014.
[22]
Jinke Ren, Guanding Yu, Yunlong Cai, and Yinghui He. 2018. Latency optimization for resource allocation in mobile-edge computation offloading. IEEE Transactions on Wireless Communications 17, 8 (2018), 5506–5519. arxiv:1704.00163
[23]
Y. Sun, S. Zhou, and J. Xu. 2017. EMM: Energy-aware mobility management for mobile edge computing in ultra dense networks. IEEE Journal on Selected Areas in Communications 35, 11 (Nov 2017), 2637–2646.
[24]
Chenmeng Wang, Chengchao Liang, F. Richard Yu, Qianbin Chen, and Lun Tang. 2017. Computation offloading and resource allocation in wireless cellular networks with mobile edge computing. IEEE Transactions on Wireless Communications 16, 8 (2017), 4924–4938.
[25]
Feng Wang, Jie Xu, and Shuguang Cui. 2020. Optimal energy allocation and task offloading policy for wireless powered mobile edge computing systems. IEEE Transactions on Wireless Communications 1276, c (2020), 1–1. arxiv:1907.07565
[26]
Y. Wang, P. Lang, D. Tian, J. Zhou, X. Duan, Y. Cao, and D. Zhao. 2020. A game-based computation offloading method in vehicular multi-access edge computing networks. IEEE Internet of Things Journal (2020), 1–1.
[27]
Q. Xu, Z. Su, Q. Zheng, M. Luo, and B. Dong. 2017. Secure content delivery with edge nodes to save caching resources for mobile users in green cities. IEEE Transactions on Industrial Informatics (2017), 1–1.
[28]
Changyan Yi, Jun Cai, and Zhou Su. 2020. A multi-user mobile computation offloading and transmission scheduling mechanism for delay-sensitive applications. IEEE Transactions on Mobile Computing 19, 1 (2020), 29–43.
[29]
W. Zhan, C. Luo, G. Min, C. Wang, Q. Zhu, and H. Duan. 2020. Mobility-aware multi-user offloading optimization for mobile edge computing. IEEE Transactions on Vehicular Technology 69, 3 (2020), 3341–3356.
[30]
Wenhan Zhan, Chunbo Luo, Geyong Min, Chao Wang, Qingxin Zhu, and Hancong Duan. 2020. Mobility-aware multi-user offloading optimization for mobile edge computing. IEEE Transactions on Vehicular Technology 69, 3 (2020), 1–1.
[31]
Yufeng Zhan, Song Guo, Peng Li, and Jiang Zhang. 2020. A deep reinforcement learning based offloading game in edge computing. IEEE Trans. Comput. 9340, c (2020), 1–14.
[32]
Biling Zhang, Senior Member, Lili Wang, and Zhu Han. 2020. Contracts for joint downlink and uplink traffic offloading with asymmetric information. 8716, c (2020), 1–13.
[33]
H. Zhang, Y. Xiao, S. Bu, D. Niyato, F. R. Yu, and Z. Han. 2017. Computing resource allocation in three-tier IoT fog networks: A joint optimization approach combining Stackelberg game and matching. IEEE Internet of Things Journal 4, 5 (Oct 2017), 1204–1215.
[34]
Qi Zhang, Lin Gui, Fen Hou, Jiacheng Chen, Shichao Zhu, and Feng Tian. 2020. Dynamic task offloading and resource allocation for mobile edge computing in dense cloud RAN. IEEE Internet of Things Journal 4662, c (2020), 1–1.
[35]
Xuefei Zhang, Jian Zhang, Zhitong Liu, Qimei Cui, Xiaofeng Tao, and Shuo Wang. 2020. MDP-based task offloading for vehicular edge computing under certain and uncertain transition probabilities. IEEE Transactions on Vehicular Technology 9545, c (2020), 1–1.
[36]
Jianchao Zheng, Yueming Cai, Yuan Wu, and Xuemin Shen. 2019. Dynamic computation offloading for mobile cloud computing: A stochastic game-theoretic approach. IEEE Transactions on Mobile Computing 18, 4 (2019), 771–786.

Cited By

View all
  • (2024)CORA: Channel Occupancy-Aware Resource Allocation in LoRa Wireless NetworksIEEE Transactions on Vehicular Technology10.1109/TVT.2024.342545973:11(17083-17094)Online publication date: Nov-2024
  • (2024)CBDTF: A Distributed and Trustworthy Data Trading Framework for Mobile CrowdsensingIEEE Transactions on Vehicular Technology10.1109/TVT.2023.332760473:3(4207-4218)Online publication date: Mar-2024
  • (2024)Distributed DNN Inference With Fine-Grained Model Partitioning in Mobile Edge Computing NetworksIEEE Transactions on Mobile Computing10.1109/TMC.2024.335787423:10(9060-9074)Online publication date: 1-Oct-2024
  • Show More Cited By

Index Terms

  1. AI-Enabled Task Offloading for Improving Quality of Computational Experience in Ultra Dense Networks

          Recommendations

          Comments

          Information & Contributors

          Information

          Published In

          cover image ACM Transactions on Internet Technology
          ACM Transactions on Internet Technology  Volume 22, Issue 3
          August 2022
          631 pages
          ISSN:1533-5399
          EISSN:1557-6051
          DOI:10.1145/3498359
          • Editor:
          • Ling Liu
          Issue’s Table of Contents

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          Published: 14 March 2022
          Accepted: 01 October 2021
          Revised: 01 March 2021
          Received: 01 October 2020
          Published in TOIT Volume 22, Issue 3

          Permissions

          Request permissions for this article.

          Check for updates

          Author Tags

          1. Edge-AI
          2. quality of computational experience
          3. UDN
          4. deep reinforcement learning
          5. task offloading
          6. MEC

          Qualifiers

          • Research-article
          • Refereed

          Funding Sources

          • National Science Foundation of China (NSFC)
          • National Key R&D Program of China

          Contributors

          Other Metrics

          Bibliometrics & Citations

          Bibliometrics

          Article Metrics

          • Downloads (Last 12 months)124
          • Downloads (Last 6 weeks)10
          Reflects downloads up to 17 Feb 2025

          Other Metrics

          Citations

          Cited By

          View all
          • (2024)CORA: Channel Occupancy-Aware Resource Allocation in LoRa Wireless NetworksIEEE Transactions on Vehicular Technology10.1109/TVT.2024.342545973:11(17083-17094)Online publication date: Nov-2024
          • (2024)CBDTF: A Distributed and Trustworthy Data Trading Framework for Mobile CrowdsensingIEEE Transactions on Vehicular Technology10.1109/TVT.2023.332760473:3(4207-4218)Online publication date: Mar-2024
          • (2024)Distributed DNN Inference With Fine-Grained Model Partitioning in Mobile Edge Computing NetworksIEEE Transactions on Mobile Computing10.1109/TMC.2024.335787423:10(9060-9074)Online publication date: 1-Oct-2024
          • (2024)A survey on computation offloading in edge systems: From the perspective of deep reinforcement learning approachesComputer Science Review10.1016/j.cosrev.2024.10065653(100656)Online publication date: Aug-2024
          • (2024)IRS-assisted energy efficient communication for UAV mobile edge computingComputer Networks10.1016/j.comnet.2024.110387246(110387)Online publication date: Jun-2024
          • (2024)Binary task offloading strategy for cloud robots using improved game theory in cloud-edge collaborationThe Journal of Supercomputing10.1007/s11227-024-06034-880:10(14752-14772)Online publication date: 26-Mar-2024
          • (2023)Distributed DRL-Based Computation Offloading Scheme for Improving QoE in Edge Computing EnvironmentsSensors10.3390/s2308416623:8(4166)Online publication date: 21-Apr-2023
          • (2023)Graph Tasks Offloading and Resource Allocation in Multi-Access Edge Computing: A DRL-and-Optimization-Aided ApproachIEEE Transactions on Network Science and Engineering10.1109/TNSE.2023.3272351(1-13)Online publication date: 2023
          • (2023)The Smart Workflow Analysis Framework For Channel Allocation In Ultra Dense Cloud Computing2023 2nd International Conference for Innovation in Technology (INOCON)10.1109/INOCON57975.2023.10101079(1-6)Online publication date: 3-Mar-2023
          • (2023)Energy allocation and task scheduling in edge devices based on forecast solar energy with meteorological informationJournal of Parallel and Distributed Computing10.1016/j.jpdc.2023.03.005177:C(171-181)Online publication date: 1-Jul-2023
          • Show More Cited By

          View Options

          Login options

          Full Access

          View options

          PDF

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader

          Full Text

          View this article in Full Text.

          Full Text

          HTML Format

          View this article in HTML Format.

          HTML Format

          Figures

          Tables

          Media

          Share

          Share

          Share this Publication link

          Share on social media