Skip to main content

Advertisement

SpringerLink
Log in

A deep reinforcement learning-based wireless body area network offloading optimization strategy for healthcare services

  • Research
  • Published:
Health Information Science and Systems Aims and scope Submit manuscript

Abstract

Wireless body area network (WBAN) is widely adopted in healthcare services, providing remote real-time and continuous healthcare monitoring. With the massive increase of detective sensor data, WBAN is largely restricted by limited storage and computation capacity, resulting in severely decreased efficiency and reliability. Mobile edge computing (MEC) technique can be combined with WBAN to resolve this issue. This paper studies the joint optimization problem of computational offloading and resource allocation (JCORA) in MEC for healthcare service scenarios. We formulate JCORA as a Markov decision process and propose a deep deterministic policy gradient-based WBAN offloading strategy (DDPG-WOS) to optimize time delay and energy consumption in interfered transmission channels. This scheme employs MEC to mitigate the computation pressure on a single WBAN and increase the transmission ability. Further, DDPG-WOS optimizes the offloading strategy-making process by considering the channel condition, transmission quality, computation ability and energy consumption. Simulation results verify the effectiveness of the proposed optimization schema in reducing energy consumption and computation latency and increasing the utility of WBAN compared to two competitive solutions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Tang C, Yin J. A localization algorithm of weighted maximum likelihood estimation for wireless sensor network. J Inf Comput Sci. 2011;8(16):4293–300.

    Google Scholar 

  2. Du J, Michalska S, Subramani S, Wang H, Zhang Y. Neural attention with character embeddings for hay fever detection from twitter. Health Inf Sci Syst. 2019;7(1):1–7.

    Article  Google Scholar 

  3. Qadri YA, Nauman A, Zikria YB, Vasilakos AV, Kim SW. The future of healthcare internet of things: a survey of emerging technologies. IEEE Commun Surv Tutor. 2020;22(2):1121–67.

    Article  Google Scholar 

  4. Teshome AK, Kibret B, Lai DT. A review of implant communication technology in WBAN: progress and challenges. IEEE Rev Biomed Eng. 2018;12:88–99.

    Article  Google Scholar 

  5. Hammood D, Alkhayyat A. An overview of the survey/review studies in wireless body area network. IEEE; 2020. pp. 18–23.

  6. Yin J, Cao J, Siuly S, Wang H. An integrated mci detection framework based on spectral-temporal analysis. Int J Autom Comput. 2019;16(6):786–99.

    Article  Google Scholar 

  7. Wang W, Qin T, Wang Y. Encryption-free data transmission and hand-over in two-tier body area networks. Comput Methods Programs Biomed. 2020;192:105411.

    Article  Google Scholar 

  8. Tang C, Cheng Y, Yin J. An optimized algorithm of grid calibration in WSN node deployment based on the energy consumption distribution model. J Inf Comput Sci. 2012;9(4):1035–42.

    Google Scholar 

  9. Brik B, Frangoudis PA, Ksentini A. Service-oriented MEC applications placement in a federated edge cloud architecture. In: ICC 2020-2020 IEEE international conference on communications (ICC). IEEE; 2020. pp. 1–6.

  10. Liao Y, Han Y, Yu Q, Ai Q, Liu Q, Leeson MS. Wireless body area network mobility-aware task offloading scheme. IEEE Access. 2018;6:61366–76.

    Article  Google Scholar 

  11. Vimalachandran P, Liu H, Lin Y, Ji K, Wang H, Zhang Y. Improving accessibility of the Australian my health records while preserving privacy and security of the system. Health Inf Sci Syst. 2020;8(1):1–9.

    Article  Google Scholar 

  12. You M, Yin J, Wang H, Cao J, Miao Y. A minority class boosted framework for adaptive access control decision-making. In: International conference on web information systems engineering. Springer; 2021. pp. 143–157.

  13. Alnoman A, Sharma SK, Ejaz W, Anpalagan A. Emerging edge computing technologies for distributed IoT systems. IEEE Netw. 2019;33(6):140–7.

    Article  Google Scholar 

  14. Tawhid M, Ahad N, Siuly S, Wang K, Wang H. Data mining based artificial intelligent technique for identifying abnormalities from brain signal data. In: international conference on web information systems engineering. Springer; 2021. pp. 198–206.

  15. Chen Y, Liu Z, Zhang Y, Wu Y, Chen X, Zhao L. Deep reinforcement learning-based dynamic resource management for mobile edge computing in industrial internet of things. IEEE Trans Ind Inf. 2020;17(7):4925–34.

    Article  Google Scholar 

  16. You M, Yin J, Wang H, Cao J, Wang K, Miao Y, Bertino E. A knowledge graph empowered online learning framework for access control decision-making. World Wide Web, 2022. pp. 1–22.

  17. Yuan X, Tian H, Wang H, Su H, Liu J, Taherkordi A. Edge-enabled WBANs for efficient QOS provisioning healthcare monitoring: a two-stage potential game-based computation offloading strategy. IEEE Access. 2020;8:92718–30.

    Article  Google Scholar 

  18. Nath S, Wu J. Deep reinforcement learning for dynamic computation offloading and resource allocation in cache-assisted mobile edge computing systems. Intell Converg Netw. 2020;1(2):181–98.

    Article  Google Scholar 

  19. Xu Y-H, Xie J-W, Zhang Y-G, Hua M, Zhou W. Reinforcement learning (RL)-based energy efficient resource allocation for energy harvesting-powered wireless body area network. Sensors. 2019;20(1):44.

    Article  Google Scholar 

  20. Yadav R, Zhang W, Elgendy IA, Dong G, Shafiq M, Laghari AA, Prakash S. Smart healthcare: Rl-based task offloading scheme for edge-enable sensor networks. IEEE Sens J. 2021;21(22):24910–8.

    Article  Google Scholar 

  21. Heidari A, Jabraeil Jamali MA, Jafari Navimipour N, Akbarpour S. Deep Q-learning technique for offloading offline/online computation in blockchain-enabled green IoT-edge scenarios. Appl Sci. 2022;12(16):8232.

    Article  Google Scholar 

  22. Sarki R, Ahmed K, Wang H, Zhang Y. Automated detection of mild and multi-class diabetic eye diseases using deep learning. Health Inf Sci Syst. 2020;8(1):1–9.

    Article  Google Scholar 

  23. Liu J, Ahmed M, Mirza MA, Khan WU, Xu D, Li J, Aziz A, Han Z. RL/DRL meets vehicular task offloading using edge and vehicular cloudlet: a survey. IEEE Internet Things J. 2022;9(11):8315–38.

    Article  Google Scholar 

  24. Yin J, You M, Cao J, Wang H, Tang M, Ge Y-F. Data-driven hierarchical neural network modeling for high-pressure feedwater heater group. In: Australasian database conference. Springer; 2020. pp. 225–233.

  25. Yin J, Tang M, Cao J, You M, Wang H, Alazab M. Knowledge-driven cybersecurity intelligence: software vulnerability co-exploitation behaviour discovery. In: IEEE transactions on industrial informatics; 2022.

  26. Lyu X, Tian H, Sengul C, Zhang P. Multiuser joint task offloading and resource optimization in proximate clouds. IEEE Trans Veh Technol. 2016;66(4):3435–47.

    Article  Google Scholar 

  27. Zheng J, Cai Y, Wu Y, Shen X. Dynamic computation offloading for mobile cloud computing: a stochastic game-theoretic approach. IEEE Trans Mob Comput. 2018;18(4):771–86.

    Article  Google Scholar 

  28. Wu H, Sun Y, Wolter K. Energy-efficient decision making for mobile cloud offloading. IEEE Trans Cloud Comput. 2018;8(2):570–84.

    Article  Google Scholar 

  29. Zanette A. Exponential lower bounds for batch reinforcement learning: Batch rl can be exponentially harder than online rl. In: International conference on machine learning. PMLR; 2021. pp. 12287–12297.

  30. Shakarami A, Ghobaei-Arani M, Shahidinejad A. A survey on the computation offloading approaches in mobile edge computing: a machine learning-based perspective. Comput Netw. 2020;182:107496.

    Article  Google Scholar 

  31. Chen J, Xing H, Xiao Z, Xu L, Tao T. A DRL agent for jointly optimizing computation offloading and resource allocation in MEC. IEEE Internet Things J. 2021;8(24):17508–24.

    Article  Google Scholar 

  32. Pandey D, Wang H, Yin X, Wang K, Zhang Y, Shen J. Automatic breast lesion segmentation in phase preserved DCE-MRIs. Health Inf Sci Syst. 2022;10(1):1–19.

    Article  Google Scholar 

  33. Huang L, Bi S, Zhang Y-JA. Deep reinforcement learning for online computation offloading in wireless powered mobile-edge computing networks. IEEE Trans Mob Comput. 2019;19(11):2581–93.

    Article  Google Scholar 

  34. Xu X, Li D, Dai Z, Li S, Chen X. A heuristic offloading method for deep learning edge services in 5G networks. IEEE Access. 2019;7:67734–44.

    Article  Google Scholar 

  35. Lu H, Gu C, Luo F, Ding W, Liu X. Optimization of lightweight task offloading strategy for mobile edge computing based on deep reinforcement learning. Future Gener Comput Syst. 2020;102:847–61.

    Article  Google Scholar 

  36. Ale L, King SA, Zhang N, Sattar AR, Skandaraniyam J. D3PG: Dirichlet DDPG for task partitioning and offloading with constrained hybrid action space in mobile edge computing. IEEE Internet Things J. 2022;9:19260.

    Article  Google Scholar 

  37. Li Y, Qi F, Wang Z, Yu X, Shao S. Distributed edge computing offloading algorithm based on deep reinforcement learning. IEEE Access. 2020;8:85204–15.

    Article  Google Scholar 

  38. Chen X, Ge H, Liu L, Li S, Han J, Gong H. Computing offloading decision based on DDPG algorithm in mobile edge computing. In: 2021 IEEE 6th international conference on cloud computing and big data analytics (ICCCBDA), 2021. pp. 391–399.

  39. Hu H, Wu D, Zhou F, Jin S, Hu RQ. Dynamic task offloading in MEC-enabled IoT networks: a hybrid DDPG-d3qn approach. In: 2021 IEEE global communications conference (GLOBECOM), 2021. pp. 1–6.

  40. Zhang L, Jiang Y, Zheng F-C, Bennis M, You X. Computation offloading and resource allocation in f-rans: a federated deep reinforcement learning approach. In: 2022 IEEE international conference on communications workshops (ICC Workshops), 2022. pp. 97–102.

  41. Qiu Y, Zhang H, Long K. Computation offloading and wireless resource management for healthcare monitoring in fog-computing-based internet of medical things. IEEE Internet Things J. 2021;8(21):15875–83.

    Article  Google Scholar 

  42. Zhang H, Guo J, Yang L, Li X, Ji H. Computation offloading considering fronthaul and backhaul in small-cell networks integrated with MEC. In: 2017 IEEE conference on computer communications workshops (INFOCOM WKSHPS). IEEE, 2017. pp. 115–120.

  43. Yu Y, Tang J, Huang J, Zhang X, So DKC, Wong K-K. Multi-objective optimization for UAV-assisted wireless powered IoT networks based on extended DDPG algorithm. IEEE Trans Commun. 2021;69(9):6361–74.

    Article  Google Scholar 

  44. Ye H, Li GY, Juang B-HF. Deep reinforcement learning based resource allocation for V2V communications. IEEE Trans Veh Technol. 2019;68(4):3163–73.

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the Science and Technology Program of Guangzhou, China (Grant No. 202102080279).

Author information

Authors and Affiliations

  1. School of Cyber Security, Guangdong Polytechnic Normal University, Guangzhou, 510000, China

    Yingqun Chen, Shaodong Han & Guihong Chen

  2. School of Automation Science and Engineering, South China University of Technology, Guangzhou, 510641, China

    Guihong Chen

  3. Institute for Sustainable Industries and Liveable Cities, Victoria University, Melbourne, 3011, Australia

    Jiao Yin

  4. School of Health and Biomedical Sciences, RMIT University, Melbourne, 3082, Australia

    Kate Nana Wang

  5. Department of Computer Science and Information Technology, La Trobe University, Melbourne, 3086, Australia

    Jinli Cao

Authors
  1. Yingqun Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shaodong Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guihong Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jiao Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kate Nana Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jinli Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guihong Chen.

Ethics declarations

Conflict of interest

The authors have no conflicts of interest to declare that are relevant to the content of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, Y., Han, S., Chen, G. et al. A deep reinforcement learning-based wireless body area network offloading optimization strategy for healthcare services. Health Inf Sci Syst 11, 8 (2023). https://doi.org/10.1007/s13755-023-00212-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s13755-023-00212-3

Keywords

Search

Navigation