Skip to main content

Advertisement

Springer Nature Link
Log in

Reinforcement Learning-Based Routing in Underwater Acoustic Sensor Networks

  • Published:
Wireless Personal Communications Aims and scope Submit manuscript

Abstract

Underwater acoustic sensor networks (UASNs) play an essential role in exploring the marine world applications in rivers, ponds, and oceans. In recent years, prominent developments have addressed and resolved the numerous issues of critical underwater applications. Routing is one of the vital problems discussed widely, and various protocols are designed to solve the routing-related issues effectively. In this paper, we propose reinforcement-learning based routing using Steiner-points to minimize energy consumption and enhance the network lifetime of UASNs. Initially, the sensor nodes form a group of three nodes and calculate the Steiner point for each such group. The isolated nodes and the Steiner points in the network are vertices of the Steiner tree, which is the routing path to be constructed dynamically by the sink node agent to travel with the assistance of AUV. The sink node agent in the unknown environment initiates the Steiner tree construction process by visiting either the isolated neighbor node or neighbor sensor node’s Steiner point to collect the data from sensor nodes based on the reward function and the environmental conditions. If agent visits the isolated node, then it collects the data from that node itself. If the agent visits the Steiner point, the agent broadcasts its arrival through the beacon message and intended members of that group communicate to the agent and forwards the sensed data. The agent processes the data to remove the data’s redundancy and selects the next point to be visited and constructs the Steiner tree dynamically. The performance evaluation of the proposed scheme minimized each sensor node’s energy consumption and enhanced the network’s lifetime.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Ayaz, M., Abdullah, A., Faye, I., & Batira, Y. (2012). An efficient dynamic addressing based routing protocol for underwater wireless sensor networks. Computer Communications, 35, 475–486.

    Article  Google Scholar 

  2. Akyildiz, I. F., Pompili, D., & Melodia, T. (2005). Underwater acoustic sensor networks: Research challenges. Ad hoc Networks, 3(3), 257–279.

    Article  Google Scholar 

  3. Wait, A. D. (2002). Sonar for practicing engineers (3rd ed.). Wiley.

  4. Etter, P. C. (2013). Underwater acoustic modeling and simulation (3rd ed.). CRC Press, Taylor & Francis Group.

  5. Khan, A., Ali, I., Ghani, A., Khan, N., Alsaqer, M., Rahman, A. U., & Mahmood, H. (2018). Routing protocols for underwater wireless sensor networks: Taxonomy-research challenges-routing strategies and future directions. Sensors, 18(5), 1–38.

    Article  Google Scholar 

  6. Xie, P., Cui, J. H., Lao, L., & (2006). VBF: Vector-based forwarding protocol for underwater sensor networks (NETWORKING 2006). In Networking technologies, services, and protocols; performance of computer and communication networks; mobile and wireless communications systems (pp. 1216–1221). Springer.

  7. Hwang, F. K. (1986). A linear time algorithm for full Steiner trees. Operations Research Letters, 5, 235–237.

    Article  MathSciNet  Google Scholar 

  8. Forster, A. (2007). Machine learning techniques applied to wireless ad-hoc networks: Guide and survey. In 2007 3rd International conference on intelligent sensors, sensor networks and information (pp. 365–370).

  9. Kim, S. (2018). A better-performing Q-learning game-theoretic distributed routing for underwater wireless sensor networks. International Journal of Distributed Sensor Networks, 14(1), 1–12.

    Google Scholar 

  10. Faheem, M., Ngadi, M. A., & Gungor, V. C. (2019). Energy efficient multi-objective evolutionary routing scheme for reliable data gathering in Internet of underwater acoustic sensor networks. Ad Hoc Networks, 93, 101912.

    Article  Google Scholar 

  11. Khan, A., Khan, M., Ahmed, S., Abd Rahman, M. A., & Khan, M. (2019). Energy harvesting based routing protocol for underwater sensor networks. PLoS ONE, 14(7), 1–18.

    Article  Google Scholar 

  12. Hoa, T.-D., & Dong-Seong, K. (2019). Energy-efficient cooperative routing in underwater acoustic sensor networks. Journal of Advances in Information Technology, 10(4), 137–141.

    Article  Google Scholar 

  13. Sangeeta, A., Kumari, P., Kumar, M., & Veena, A. (2019). Fault resilient routing based on moth flame optimization scheme for underwater wireless sensor networks. Wireless Networks, 66, 1–15.

    Google Scholar 

  14. Di Valerio, V., Petrioli, C., Pescosolido, L., & Van Der Shaar, M. (2015). A reinforcement learning-based data-link protocol for underwater acoustic communications. In WUWNET’15 proceedings of the 10th international conference on underwater networks & systems—Arlington, VA, USA (pp. 1–5).

  15. Jin, Z., Ma, Y., Su, Y., Li, S., & Fu, X. (2017). A Q-learning-based delay-aware routing algorithm to extend the lifetime of underwater sensor networks. Sensors, 17(7), 1–15.

    Article  Google Scholar 

  16. Ali, M., Khan, A., Mahmood, H., & Bhatti, N. (2019). An energy balanced efficient and reliable routing protocol for underwater wireless sensor networks. International Journal of Distributed Sensor Networks, 15(6), 1–11.

    Article  Google Scholar 

  17. Wadud, Z., Ismail, M., Qazi, A. B., Khan, F. A., Derhab, A., Ahmad, I., & Ahmad, A. M. (2019). Cooperative, reliable, and stability-aware routing for underwater wireless sensor networks. IEEE Access, 7, 175980–175999.

    Article  Google Scholar 

  18. Kumari, S., Mishra, P. K., & Anand, A. (2019). Integrated load balancing and void healing routing with Cuckoo search optimization scheme for underwater wireless sensor networks. Wireless Personal Communications, 66, 1–17.

    Google Scholar 

  19. Jin, Z., Zhao, Q., & Yishan, S. (2019). RCAR: A reinforcement-learning-based routing protocol for congestion-avoided underwater acoustic sensor networks. IEEE Sensors Journal, 19(22), 10881–10891.

    Article  Google Scholar 

  20. Han, G., Du, J., Lin, C., Wu, H., & Guizani, M. (2020). An energy-balanced trust cloud migration scheme for underwater acoustic sensor networks. IEEE Transactions on Wireless Communications, 19(3), 1636–1649.

    Article  Google Scholar 

  21. Tiansi, H., & Yunsi, F. (2008). QELAR: A Q-learning-based energy-efficient and lifetime-aware routing protocol for underwater sensor networks. In 2008 IEEE international performance, computing and communications conference (pp. 247–255).

  22. Yuan, Z., Tao, C., & Wei, X. (2020). Anypath routing protocol design via Q-learning for underwater sensor networks. IEEE Internet Things Journal, 47, 1–16.

    Google Scholar 

  23. Winter, P., & Zachariasen, M. (1997). Large Euclidean Steiner minimum trees in an hour. ISMP.

  24. Kalinin, M. I., & Kononogov, S. A. (2005). Boltzmann’s constant, the energy meaning of temperature, and thermodynamic Ir-reversibility. Measurement Techniques, 48, 632–636.

    Article  Google Scholar 

Download references

Funding

Not applicable

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, S.G. Balekundri Institute of Technology, Belagavi, Karnataka, 590010, India

    B. S. Halakarnimath

  2. Department of Electronics and Communication Engineering, Basaveshwar Engineering College, Bagalkot, Karnataka, 587102, India

    A. V. Sutagundar

Authors
  1. B. S. Halakarnimath
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. A. V. Sutagundar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to A. V. Sutagundar.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Halakarnimath, B.S., Sutagundar, A.V. Reinforcement Learning-Based Routing in Underwater Acoustic Sensor Networks. Wireless Pers Commun 120, 419–446 (2021). https://doi.org/10.1007/s11277-021-08467-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11277-021-08467-3

Keywords