Skip to main content

Advertisement

Springer Nature Link
Log in

RL-EAR: reinforcement learning-based energy-aware routing for software-defined wireless sensor network

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Efficient energy utilization in wireless sensor networks is paramount for the success of Internet of things (IoT) applications. Traditional energy-saving routing protocols typically compute isolated routes, which may only partially optimize energy consumption globally. Software-defined wireless sensor networking (SDWSN) is emerging as a crucial architecture for enabling IoT, leveraging software-defined network principles to facilitate routing operations. However, existing routing algorithms often need more capability to identify the most optimized paths. In this paper, we propose a reinforcement learning-based energy-aware routing (RL-EAR) algorithm to optimize the routing paths of SDWSN. Our method utilizes a reward function considering factors such as remaining energy, hop count, and congestion to optimize routes from any node to the controller node, aiming to enhance network lifetime. The SDWSN controller, acting as the agent, receives rewards based on its actions and refines routes over time through interactions with the data plane. The simulation results of the RL-EAR algorithm demonstrate its prominence over existing routing algorithms. Metrics such as network lifetime, packet delivery ratio, and the number of dead nodes over time are utilized to evaluate the effectiveness of our proposed algorithm. As a result, RL-EAR increases network lifetime by 42% compared to existing routing algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Algorithm 1
Algorithm 2
Algorithm 3
Algorithm 4
Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data availability

No datasets were generated or analyzed during the current study.

References

  1. Kandris D, Nakas C, Vomvas D, Koulouras G (2020) Applications of wireless sensor networks: an up-to-date survey. Appl Syst Innov 3(1):14

    Article  Google Scholar 

  2. Nunes BAA, Mendonca M, Nguyen X-N, Obraczka K, Turletti T (2014) A survey of software-defined networking: past, present, and future of programmable networks. IEEE Commun Surv Tutor 16(3):1617–1634

    Article  Google Scholar 

  3. Narwaria A, Mazumdar AP (2023) Software-defined wireless sensor network: a comprehensive survey. J Netw Comput Appl 215:103636

    Article  MATH  Google Scholar 

  4. Narwaria A, Mazumdar AP, Sharma S (2023) Energy efficient multi-objective task allocation in software-defined wireless sensor network. In: 2023 IEEE 8th International Conference for Convergence in Technology (I2CT), pp 1–6

  5. Rahimifar A, Kavian YS, Kaabi H, Soroosh M (2023) A smart duty cycle for lifetime enhancement and control overhead in SDWSN. Iran J Sci Technol Trans Electr Eng 47(3):1207–1223

    Article  Google Scholar 

  6. Moerland TM, Broekens J, Plaat A, Jonker CM et al (2023) Model-based reinforcement learning: a survey. Found Trends® Mach Learn 16(1):1–118

    Article  MATH  Google Scholar 

  7. Savaglio C, Pace P, Aloi G, Liotta A, Fortino G (2019) Lightweight reinforcement learning for energy efficient communications in wireless sensor networks. IEEE Access 7:29355–29364

    Article  MATH  Google Scholar 

  8. Rahimifar A, Seifi Kavian Y, Kaabi H, Soroosh M (2021) Predicting the energy consumption in software defined wireless sensor networks: a probabilistic Markov model approach. J Ambient Intell Humaniz Comput 12(10):9053–9066

    Article  Google Scholar 

  9. Younus MU, Khan MK, Bhatti AR (2021) Improving the software-defined wireless sensor networks routing performance using reinforcement learning. IEEE Internet Things J 9(5):3495–3508

    Article  MATH  Google Scholar 

  10. Kobo HI, Abu-Mahfouz AM, Hancke GP (2017) A survey on software-defined wireless sensor networks: challenges and design requirements. IEEE Access 5:1872–1899

    Article  MATH  Google Scholar 

  11. Vijayan K, Kshirsagar PR, Sonekar SV, Chakrabarti P, Unhelkar B, Margala M (2024) Optimizing IoT-enabled WSN routing strategies using whale optimization-driven multi-criterion correlation approach employs the reinforcement learning agent. Opt Quantum Electron 56(4):568

    Article  Google Scholar 

  12. Priyadarshi R (2024) Exploring machine learning solutions for overcoming challenges in IoT-based wireless sensor network routing: a comprehensive review. Wirel Netw 30:1–27

    Article  MATH  Google Scholar 

  13. Jurado-Lasso FF, Clarke K, Cadavid AN, Nirmalathas A (2022) Energy-aware routing for software-defined multihop wireless sensor networks. IEEE Sens J 21(8):10174–10182

    Article  Google Scholar 

  14. Yang X, Yan J, Wang D, Xu Y, Hua G (2024) WOAD3QN-RP: an intelligent routing protocol in wireless sensor networks-a swarm intelligence and deep reinforcement learning based approach. Expert Syst Appl 246:123089

    Article  MATH  Google Scholar 

  15. Garcia F, Rachelson E (2013) Markov decision processes. Markov Decision Processes in Artificial Intelligence, pp 1–38

  16. Xiang W, Wang N, Zhou Y (2016) An energy-efficient routing algorithm for software-defined wireless sensor networks. IEEE Sens J 16(20):7393–7400

    Article  MATH  Google Scholar 

  17. Kamarei M, Tavakoli F, Patooghy A (2022) Route packets uneven, consume energy even: A lifetime expanding routing algorithm for WSNS. Wireless Pers Commun 125(4):3133–3151

    Article  MATH  Google Scholar 

  18. Kamarei M, Patooghy A, Alsharif A, Hakami V (2020) Simple: a unified single and multi-path routing algorithm for wireless sensor networks with source location privacy. IEEE Access 8:33818–33829

    Article  Google Scholar 

  19. Wenxing L, Muqing W, Yuewei W (2016) Energy-efficient algorithm based on multi-dimensional energy space for software-defined wireless sensor networks. In: 2016 International Symposium on Wireless Communication Systems (ISWCS), IEEE, pp 309–314

  20. Kim S, Son J, Talukder A, Hong CS (2016) Congestion prevention mechanism based on q-leaning for efficient routing in SDN. In: 2016 International Conference on Information Networking (ICOIN), IEEE, pp 124–128

  21. Stampa G, Arias M, Sánchez-Charles D, Muntés-Mulero V, Cabellos A (2017) A deep-reinforcement learning approach for software-defined networking routing optimization. arXiv:1709.07080

  22. Mostafaei H, Obaidat MS (2018) Learning automaton-based self-protection algorithm for wireless sensor networks. IET Netw 7(5):353–361

    Article  MATH  Google Scholar 

  23. Guo W, Yan C, Lu T (2019) Optimizing the lifetime of wireless sensor networks via reinforcement-learning-based routing. Int J Distrib Sens Netw 15(2):1550147719833541

    Article  MATH  Google Scholar 

  24. Lee S, Seo Y, Lee K, Abbeel P, Shin J (2022) Offline-to-online reinforcement learning via balanced replay and pessimistic q-ensemble. In: Conference on Robot Learning, PMLR, pp 1702–1712

  25. Fang C, Cheng C, Tang Z, Li C (2019) Research on routing algorithm based on reinforcement learning in SDN. In: Journal of Physics: Conference Series, vol 1284. IOP Publishing, p 012053

  26. Younus MU, Khan MK, Anjum MR, Afridi S, Arain ZA, Jamali AA (2020) Optimizing the lifetime of software defined wireless sensor network via reinforcement learning. IEEE Access 9:259–272

    Article  Google Scholar 

  27. Jurado-Lasso FF, Clarke K, Cadavid AN, Nirmalathas A (2021) Energy-aware routing for software-defined multihop wireless sensor networks. IEEE Sens J 21(8):10174–10182

    Article  Google Scholar 

  28. Dunkels A, Gronvall B, Voigt T (2004) Contiki-a lightweight and flexible operating system for tiny networked sensors. In: 29th Annual IEEE International Conference on Local Computer Networks, IEEE, pp 455–462

  29. Eriksson J, Österlind F, Finne N, Tsiftes N, Dunkels A, Voigt T, Sauter R, Marrón PJ (2009) Cooja/mspsim: interoperability testing for wireless sensor networks. In: Proceedings of the 2nd International Conference on Simulation Tools and Techniques, pp 1–7

  30. Kumar A, Zhou A, Tucker G, Levine S (2020) Conservative q-learning for offline reinforcement learning. Adv Neural Inf Process Syst 33:1179–1191

    MATH  Google Scholar 

  31. Casas-Velasco DM, Rendon OMC, Fonseca NL (2020) Intelligent routing based on reinforcement learning for software-defined networking. IEEE Trans Netw Serv Manage 18(1):870–881

    Article  MATH  Google Scholar 

  32. Narwaria A, Mazumdar AP, Kalla G (2021) C3hac: A controller placement approach for sdwsn. In: TENCON 2021-2021 IEEE Region 10 Conference (TENCON), IEEE, pp 935–940

  33. Knox WB, Stone P (2012) Reinforcement learning from simultaneous human and MDP reward. In: AAMAS, vol 1004. Valencia, pp 475–482

  34. Su X, Ren Y, Cai Z, Liang Y, Guo L (2022) A Q-learning based routing approach for energy efficient information transmission in wireless sensor network. IEEE Trans Netw Serv Manag 20(2):1949–1961

    Article  MATH  Google Scholar 

  35. Van Seijen H, Fatemi M, Romoff J, Laroche R, Barnes T, Tsang J (2017) Hybrid reward architecture for reinforcement learning. Advances in Neural Information Processing Systems 30

  36. Narwaria A, Soni K, Mazumdar AP (2024) A position and energy aware multi-objective controller placement and re-placement scheme in distributed SDWSN. J Supercomput 80:1–29

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Malaviya National Institute of Technology, Jaipur, Rajasthan, 302017, India

    Abhishek Narwaria, Varsha Kumari & Arka Prokash Mazumdar

  2. Department of Computer and Communication Engineering, Manipal University Jaipur, Jaipur, Rajasthan, 303007, India

    Abhishek Narwaria

Authors
  1. Abhishek Narwaria
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Varsha Kumari
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Arka Prokash Mazumdar
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

A. N. contributed to the conceptualization, methodology, writing, reviewing, editing, and software development of the study. V. K. was responsible for methodology, software implementation, and the creation of figures and graphs. A. P. M. contributed to conceptualization, reviewing, editing, and provided supervision for the research.

Corresponding author

Correspondence to Arka Prokash Mazumdar.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Narwaria, A., Kumari, V. & Mazumdar, A.P. RL-EAR: reinforcement learning-based energy-aware routing for software-defined wireless sensor network. J Supercomput 81, 485 (2025). https://doi.org/10.1007/s11227-025-06998-1

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11227-025-06998-1

Keywords