Skip to main content

Advertisement

Springer Nature Link
Log in

A task unloading strategy of IoT devices using deep reinforcement learning based on mobile cloud computing environment

  • Published:
Wireless Networks Aims and scope Submit manuscript

Abstract

Aiming at the task unloading mode in cloud computing environment, the task unloading problem for IoT devices is studied. Through theoretical analysis, we can know that in the task unloading problem, it is usually contradictory to improve the utilization of cloud resources and reduce the task delay. In order to solve this problem, a task unloading scheme for Internet of things devices using deep reinforcement learning algorithm is proposed. The deep reinforcement learning algorithm is used to model the task unloading problem. The return value with weight is introduced into the algorithm, and the utilization rate of cloud resources and the delay of unloading task are weighed by adjusting the return value of the weight. First of all, the improved k-means clustering algorithm with weighted density is used to cluster the physical machines. The physical machines of each cluster have similar bandwidth and task waiting time. Then, deep reinforcement learning is used to select the best physical machine cluster from the current unloading tasks. Finally, the improved PSO algorithm is used to select the optimal physical machine from the optimal cluster, and Pareto is used to improve the convergence speed. Experimental results show that compared with the traditional method, the proposed algorithm has a good performance, and can achieve the goal of increasing the utilization of physical machine resources and reducing task delay.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Lin, X., Jiang, J., Li, C. H. Y., et al. (2020). Circa: Collaborative code offloading among multiple mobile devices. Wireless Networks, 26, 823–841. https://doi.org/10.1007/s11276-018-1824-y.

    Article  Google Scholar 

  2. Xu, X., Gu, R., Dai, F., et al. (2019). Multi-objective computation offloading for Internet of Vehicles in cloud-edge computing. Wireless Networks. https://doi.org/10.1007/s11276-019-02127-y.

    Article  Google Scholar 

  3. Long, Hu, Tian, Y., Yang, J., et al. (2019). Ready player one: UAV-clustering-based multi-task offloading for vehicular VR/AR gaming. IEEE Network, 33(3), 42–48.

    Article  Google Scholar 

  4. Hongyan Yu, Quyuan Wang, Songtao Guo. (2018) Energy-efficient task offloading and resource scheduling for mobile edge computing[C]. In 2018 IEEE International Conference on Networking, Architecture and Storage (NAS). IEEE, https://doi.org/10.1109/NAS.2018.8515731.

  5. Hao, Y., Chen, M., Long, Hu, et al. (2018). Energy efficient task caching and offloading for mobile edge computing. IEEE Access, 6(99), 11365–11373.

    Article  Google Scholar 

  6. Suzhi Bi, Ying-Jun Angela Zhang. (2018) An ADMM based method for computation rate maximization in wireless powered mobile-edge computing networks[C]. In 2018 IEEE International Conference on Communications (ICC). (pp. 1–7) IEEE.

  7. Yang, Y., Zhang, T., Wang, J., et al. (2019). Fog services and enabling technologies. IEEE Communications Magazine, 57(5), 18–18.

    Article  Google Scholar 

  8. Zhang, Ke, Leng, S., He, Y., et al. (2018). Mobile Edge Computing and Networking for Green and Low-Latency Internet of Things[J]. IEEE Communications Magazine, 56(5), 39–45.

    Article  Google Scholar 

  9. Wang, J., Jia, Hu, Min, G., et al. (2019). Computation offloading in multi-access edge computing using a deep sequential model based on reinforcement learning. IEEE Communications Magazine, 57(5), 64–69.

    Article  Google Scholar 

  10. Ometov, A., Moltchanov, D., Komarov, M., et al. (2019). Packet level performance assessment of mmWave backhauling technology for 3GPP NR systems. IEEE Access, 7(99), 9860–9871.

    Article  Google Scholar 

  11. Zhang, W., Wen, Y., Guan, K., et al. (2013). Energy-optimal mobile cloud computing under stochastic wireless channel. IEEE Transactions on Wireless Communications, 12(9), 4569–4581.

    Article  Google Scholar 

  12. Zhefeng Jiang, Shiwen Mao. (2015) Energy delay trade-off in cloud offloading for mutli-core mobile devices[C]. In 2015 IEEE Global Communications Conference GLOBECOM 2015. (pp. 1–6) IEEE.

  13. Liu J, Mao Y, Zhang J, et al. (2016) Delay-optimal computation task scheduling for mobile-edge computing systems[C]. In 2016 IEEE International Symposium on Information Theory (ISIT), (pp. 1451–1455) IEEE.

  14. Haleh Shahzad, Ted H Szymanski. (2016) A Dynamic Programming Offloading Algorithm for Mobile Cloud Computing[C]. In2016 Canadian Conf. on Electrical and Computer Engineering. (pp. 1–5) IEEE.

  15. Wang, N., Varghese, B., Matthaiou, M., et al. (2017). ENORM: A framework for edge node resource management[J]. IEEE Transactions on Services Computing. https://doi.org/10.1109/TSC.2017.2753775.

    Article  Google Scholar 

  16. Lyu, X., Tian, H., Sengul, C., et al. (2017). Multiuser joint task offloading and resource optimization in proximate clouds. IEEE Transactions on Vehicular Technology, 66(4), 3435–3447.

    Article  Google Scholar 

  17. Ciobanu, R. I., Negru, C., Pop, F., et al. (2017). Drop computing: Ad-hoc dynamic collaborative computing. Future Generation Computer Systems. https://doi.org/10.1016/j.future.2017.11.044.

    Article  Google Scholar 

  18. Chae D, Kim J, Kim J, et al. (2014) CMcloud: Cloud platform for cost-effective offloading of mobile applications[C]. In 2014 14th IEEE/ACM international symposium on Cluster, cloud and grid computing (CCGrid), (pp. 434–444) IEEE.

  19. Yu R Z,Xue G L,Zhang X. (2018) Applicationprovisioninfog computing-enabledinternet-of-things:A networkperspective [C]. In 2018 IEEEINFOCOM-IEEE Conference on Computer Communications. (pp. 783–791) Honolulu:IEEE.

  20. Chen, Xu, Jiao, L., Li, W., et al. (2015). Efficient multi-user computation offloading for mobile-edge cloud computing. IEEE/ACM Transactions on Networking, 24(5), 2795–2808.

    Article  Google Scholar 

  21. Guo, Y. (2019). Tasks offloading strategy with caching mechanism in mobile margin computing. Computer Applications and Software, 36(6), 114–119.

    Google Scholar 

  22. Liu, W., Cao, J., Yang, L., et al. (2017). AppBooster: Boosting the performance of interactive mobile applications with computation offloading and parameter tuning [J]. IEEE Transactions on Parallel and Distributed Systems, 28(6), 1593–1606.

    Article  Google Scholar 

  23. Heungsik Eom, Renato Figueiredo, Huaqian Cai, et al. (2015) MALMOS: Machine learning-based mobile offloading scheduler with online training[C]. In 2015 3rd IEEE International Conference on Mobile Cloud Computing, Services, and Engineering (MobileCloud). (pp. 51–60) IEEE.

  24. Crutcher A, Koch C, Coleman K, et al. (2017) Hyperprofile-based computation offloading for mobile edge networks [C]. In 2017 IEEE 14th International Conference on Mobile Ad Hoc and Sensor Systems (MASS), (pp. 525–529) IEEE.

  25. Wei F, Chen S X, Zou W X. (2018) Agreedy algorithm for task off-loading in mobile edge computing system[J].China Communications, 15(11):149–157.

  26. You, C., Huang, K., Chae, H., et al. (2017). Energy-efficient resource allocation for mobile-edge computation offloading. IEEE Transactions on Wireless Communications, 16(3), 1397–1411.

    Article  Google Scholar 

  27. Xia, K., Yin, H., & Zhang, Y. (2019). Deep semantic segmentation of kidney and space-occupying lesion area based on SCNN and ResNet models combined with SIFT-flow algorithm. Journal of Medical Systems, 43(1), 2.

    Article  Google Scholar 

  28. Xia, K., Yin, H., Qian, P., Jiang, Y., & Wang, S. (2019). liver semantic segmentation algorithm based on improved deep adversarial networks in combination of weighted loss function on abdominal CT images. IEEE Access, 7, 96349–96358.

    Article  Google Scholar 

  29. Hashim, H. A., El-Ferik, S., & Abido, M. A. (2018). A fuzzy logic feedback filter design tuned with PSO for L1 adaptive controller[J]. Expert Systems with Applications, 42(23), 9077–9085.

    Article  Google Scholar 

  30. Zhou, Y. F., & Chen, N. (2019). The LAP under facility disruptions during early post-earthquake rescue using PSO-GA hybrid algorithm [J]. Fresenius Environmental Bulletin, 28(12A), 9906–9914.

    Google Scholar 

Download references

Acknowledgements

This work was supported by the Key Research and Development Project of Shanxi Province (No. 201803D31055).

Author information

Authors and Affiliations

  1. Department of Computing, Taiyuan Normal University, Shanxi, 030619, Jingzhong, China

    Hui Qi, Xiaofang Mu & Ying Shi

Authors
  1. Hui Qi
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Xiaofang Mu
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Ying Shi
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Hui Qi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Qi, H., Mu, X. & Shi, Y. A task unloading strategy of IoT devices using deep reinforcement learning based on mobile cloud computing environment. Wireless Netw 30, 3587–3597 (2024). https://doi.org/10.1007/s11276-020-02471-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11276-020-02471-4

Keywords