Skip to main content

Advertisement

SpringerLink
Log in

Reinforcement learning–enabled efficient data gathering in underground wireless sensor networks

  • Original Article
  • Published:
Personal and Ubiquitous Computing Aims and scope Submit manuscript

Abstract

Wireless underground sensor networks (WUSNs) consist of sensors that are buried in and communicate through soil medium, while the channel quality of WUSNs is greatly impacted by the underground environment, such as soil moisture and composition. Due to the precipitation and harsh weather, the underground environments change frequently, which make wireless communication in WUSNs much complicated than that in terrestrial over-the-air wireless sensor networks. To achieve reliable and energy-efficient data gathering in dynamic WUSNs, this article proposes an optimal transmission policy, where path loss of sensory data transmission, energy constraint, and network load balancing are the factors to be considered. Specifically, we capture the effect of underground environments on wireless communications, and evaluate path probability, energy consumption, and load balancing factor with respect to reliability and efficiency of transmission paths. The transmission topology can be reduced to a multi-objective and multi-constrained optimization problem and solved through an improved maximum flow minimum cost algorithm. By using reinforcement learning, we derive an adaptive transmission policy for underground sensors to efficiently use their energy and avoid transmitting sensory data in unreliable paths under a dynamic environment. Through simulations and comparison upon publicly available real data, our technique achieves more reliable wireless communication with significant reduction of packet loss, and enables more energy-efficient data gathering than other techniques, especially when soil moisture varies frequently.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Chen C, Ding Y, Wang Z, Zhao J, Guo B, Zhang D (2020) VTracer: when online vehicle trajectory compression meets mobile edge computing. IEEE Syst J 14(2):1635–1646

    Article  Google Scholar 

  2. Chen C, Ding Y, Xie X, Zhang S, Wang Z, Feng L (2020) TrajCompressor: an online map-matching-based trajectory compression framework leveraging vehicle heading direction and change. IEEE Trans Intell Transp Syst 21(5):2012–2028

    Article  Google Scholar 

  3. Chen C, Zhang D, Ma X, Guo B, Wang L, Wang Y, Sha E (2017) CrowdDeliver: planning city-wide package delivery paths leveraging the crowd of taxis. IEEE Trans Intell Transp Syst 18(6):1478–1496

    Google Scholar 

  4. Chen Z, He C, He Z, Chen M (2018) BD-ADOPT: a hybrid DCOP algorithm with Best-First and Depth-First search strategies. Artif Intell Rev 50:161–199

    Article  Google Scholar 

  5. Chowdhury AR, Pramanik A, Roy GC (2020) On wireless communication in underground mine system. In: Proceedings of the 2nd International Conference on Communication, Devices and Computing, pp 159–169

  6. Dascalu S, Harris Jr FC, McMahon Jr M, Fritzinger E, Strachan S (2014) An overview of the Nevada Climate Change Portal. In: 7Th International Congress on Environmental Modelling and Software, vol 1, pp 75–82

  7. Guo H, Ben B (2019) Reinforcement learning-enabled reliable wireless sensor networks in dynamic underground environments. In: IEEE Military Communications Conference, pp 646–651

  8. Guo S, Chen C, Wang J, Liu Y, Ke X, Yu Z, Zhang D, Chiu DM (2019) ROD-Revenue: seeking strategies analysis and revenue prediction in ride-on-demand service using multi-source urban data. IEEE Transactions on Mobile Computing, pp 2202–2220

  9. Guo W, Zhu W, Yu Z, Wang J, Guo B (2019) A survey of task allocation: contrastive perspectives from wireless sensor networks and mobile crowdsensing. IEEE Access 7:78406–78420

    Article  Google Scholar 

  10. Hongwei H, Jingkang S, Fei W, Dongming Z, Dongmei Z (2020) Theoretical and experimental studies on the signal propagation in soil for wireless underground sensor networks. Sensor 20(9):2580

    Article  Google Scholar 

  11. Huang Y, Chen Z, Han T, Liu X (2018) One energy-efficient random-walk topology evolution method for underground wireless sensor networks. Int J Distrib Sens Netw 14(9):155014771880062

  12. Idrees AK, Alhussaini R, Salman MA (2020) Energy-efficient two-layer data transmission reduction protocol in periodic sensor networks of IoTs personal and ubiquitous computing

  13. Jalal D, Ezzedine T (2018) Leaks detection and localization in water distribution network based on wireless underground sensor networks. In: International Conference on Internet of Things, Embedded Systems and Communications, pp 42–46

  14. Ketshabetswe L, Adamu M, Mangwala M, Chuma J, Sigweni B (2019) Communication Protocols for wireless sensor networks: a survey and comparison. Heliyon 5:1–43

    Article  Google Scholar 

  15. Kisseleff S, Akyildiz IF, Gerstacker WH (2018) Survey on advances in magnetic induction-based wireless underground sensor networks. IEEE Internet Things J 5(6):4843– 4856

    Article  Google Scholar 

  16. Kumar] D. P., Amgoth T, Annavarapu CSR (2019) Machine learning algorithms for wireless sensor networks: a survey. Inf Fus 49:1–25

    Article  Google Scholar 

  17. Leong AS, Ramaswamy A, Quevedo DE, Karl H, Shi L (2020) Deep reinforcement learning for wireless sensor scheduling in cyber-physical systems. Automatica 113:108759

  18. Li K, Ni W, Abolhasan M, Tovar E (2019) Reinforcement learning for scheduling wireless powered sensor communications. IEEE Trans Green Commun Netw 3(2):264–274

    Article  Google Scholar 

  19. Liu G, Sun Z, Jiang T (2019) Joint time and energy allocation for QoS-aware throughput maximization in MIMO-based wireless powered underground sensor networks. IEEE Trans Commun 67(2):1400–1412

    Article  Google Scholar 

  20. Liu G, Wang Z, Jiang T (2016) Qos-aware throughput maximization in wireless powered underground sensor networks. IEEE Trans Commun 64(11):4776–4789

    Article  Google Scholar 

  21. Chincoli M, Liotta A (2018) Self-learning power control in wireless sensor networks. Sensors 18(2):375

    Article  Google Scholar 

  22. Qiu S, Mias C, Guo W, Geng X (2019) HS2 Railway embankment monitoring: effect of soil condition on underground signals. SN Appl 1(6):1–7

    Google Scholar 

  23. Raza U, Salam A (2020) On-site and external energy harvesting in underground wireless electronics 9(4)

  24. Salam A, Shah S (2019) Urban underground infrastructure monitoring IoT: the path loss analysis. In: IEEE World Forum on Internet of Things, pp 398–401

  25. Salam A, Vuran MC, Irmak S (2019) Di-Sense: In situ real-time permittivity estimation and soil moisture sensing using wireless underground communications. Comput Netw 151:31–41

    Article  Google Scholar 

  26. Sharma R, Prakash S (2020) Enhancement of relay nodes communication approach in WSN-IoT for underground coal mine. J Inf Optim Sci 41(2):521–531

    MathSciNet  Google Scholar 

  27. Szymanski TH (2013) Max-flow min-cost routing in a future-Internet with improved QoS guarantees. IEEE Trans Commun 61(4):1485–1497

    Article  Google Scholar 

  28. Tam NT, Binh HTT, Dung DA, Lan PN, Vinh LT, Yuan B, Yao X (2019) A hybrid clustering and evolutionary approach for wireless underground sensor network lifetime maximization. Inf Sci 504:372–393

    Article  Google Scholar 

  29. Tam NT, Binh HTT, Hung TH, Dung DA, Vinh LT (2019) Prolong the network lifetime of wireless underground sensor networks by optimal relay node placement. In: Applications of evolutionary computation, vol 11454, pp 439–453

  30. Trang HTH, Dung LT, Hwang SQ (2018) Connectivity analysis of underground sensors in wireless underground sensor networks. Ad Hoc Netw 71:104–116

    Article  Google Scholar 

  31. Wang KI, Wu S, Ivoghlian A, Salcic Z, Austin A, Zhou X (2019) LWS: A LoRaWAN wireless underground sensor network simulator for agriculture applications. In: IEEE Smartworld, Ubiquitous Intelligence Computing, Advanced Trusted Computing, Scalable Computing Communications, Cloud Big Data Computing, Internet of People and Smart City Innovation, pp 475–482

  32. Wang S, Shin Y (2019) Efficient routing protocol based on reinforcement learning for magnetic induction underwater sensor networks. IEEE Access 7:82027–82037

    Article  Google Scholar 

  33. Wohwe Sambo D, Forster A, Yenke BO, Sarr I, Gueye B, Dayang P (2020) Wireless underground sensor networks path loss model for precision agriculture (WUSN-PLM). IEEE Sens J 20(10):5298–5313

    Article  Google Scholar 

  34. Xia X, Chen Z, Liu H, Wang H, Zeng F (2016) A routing protocol for multisink wireless sensor networks in underground coalmine tunnels. Sensors 16:16–32

    Article  Google Scholar 

  35. Yu X, Han W, Zhang Z (2016) Path loss estimation for wireless underground sensor network in agricultural application. Agric Res 6:97–102

    Article  Google Scholar 

  36. Yu X, Zhang Z, Han W (2019) Survey on transmission performance for soil wireless sensor network signal in different communication. IAENG Int J Comput 46(3):475–484

    Google Scholar 

  37. Yu XQ, Zhang ZL, Han WT (2017) Evaluation of communication in wireless underground sensor networks. IOP Conf Ser Earth Environ Sci 69:012083

  38. Yu Z, Zhang D, Yu Z, Yang D (2015) Participant selection for offline event marketing leveraging location-based social networks. IEEE Trans Syst Man Cybern Syst 45(6):853–864

    Article  Google Scholar 

  39. Yuan B, Chen H, Yao X (2017) Optimal relay placement for lifetime maximization in wireless underground sensor networks. Inf Sci 418:463–479

    Article  Google Scholar 

  40. Zhou Z, Zhao D, Liu L, Hung PC (2018) Energy-aware composition for wireless sensor networks as a service. Fut Gen Comput Syst 80:299–310

    Article  Google Scholar 

  41. Zhou Z, Zhao D, Shu L, Tsang KF (2015) A novel two-tier cooperative caching mechanism for the optimization of multi-attribute periodic queries in wireless sensor networks. Sensors 15(7):15033–15066

    Article  Google Scholar 

  42. Zungeru AM, Mangwala M, Chuma J (2017) Optimal node placement in wireless underground sensor networks. Int J Appl Eng Res 12(20):9290–9297

    Google Scholar 

Download references

Funding

This work was supported by the National Key R&D Program of China (2019YFB2101803) and by the National Natural Science Foundation of China (61772479).

Author information

Authors and Affiliations

  1. School of Information Engineering, China University of Geosciences, Beijing, China

    Deng Zhao & Zhangbing Zhou

  2. Telecom SudParis, UMR 5157 Samovar, University of Paris-Saclay, Paris, France

    Zhangbing Zhou & Walid Gaaloul

  3. State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, China

    Shangguang Wang

  4. Faculty of Information Technology, Beijing University of Technology, Beijing, China

    Bo Liu

Authors
  1. Deng Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhangbing Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shangguang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bo Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Walid Gaaloul
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhangbing Zhou.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, D., Zhou, Z., Wang, S. et al. Reinforcement learning–enabled efficient data gathering in underground wireless sensor networks. Pers Ubiquit Comput 27, 581–598 (2023). https://doi.org/10.1007/s00779-020-01443-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00779-020-01443-x

Keywords

Search

Navigation