Skip to main content

Advertisement

SpringerLink
Log in

Reinforcement learning based energy efficient protocol for wireless multimedia sensor networks

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

With the advancements in sensor networks, Wireless multimedia sensor networks (WMSNs) have emerged and shifted the objectives of sensor nodes to multimedia devices which can retrieve audio, images, and video. In WMSNs, the sensor nodes are tiny microphones and cameras which can transmit image, audio or video using the network. However, these nodes are battery constrained (i.e., may become dead after passing certain iterations). Therefore, improvement of the network lifetime is a challenging issue of WMSNs. In this paper, a reinforcement-based energy-aware protocol is designed and implemented. To successfully implement the reinforcement-based protocol, a State-Action-Reward-State-Action (SARSA) is used for learning a Markov decision process. Extensive experiments are considered to evaluate the significant improvement of the proposed protocol. Comparisons are also drawn between the competitive protocols and the proposed protocol. From comparative analysis, it is found that the proposed protocol conserves more energy as compared to the competitive protocols.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Ahmed G, Zou J, Fareed MMS, Zeeshan M (2016) Sleep-awake energy efficient distributed clustering algorithm for wireless sensor networks. Comput. Electr. Eng. 56:385–398

    Article  Google Scholar 

  2. Amuthan A, Arulmurugan A (2018) Semi-markov inspired hybrid trust prediction scheme for prolonging lifetime through reliable cluster head selection in wsns. J. King Saud Univ. - Comput. Inf. Sci.

  3. Aslam N, Xia K, Hadi MU (2019) Optimal wireless charging inclusive of intellectual routing based on sarsa learning in renewable wireless sensor networks. IEEE Sensors J 19:8340–8351

    Article  Google Scholar 

  4. Baradaran AA, Navi K (2020) Hqca-wsn: high-quality clustering algorithm and optimal cluster head selection using fuzzy logic in wireless sensor networks. Fuzzy Sets Syst 389:114–144. Clustering

    Article  MathSciNet  Google Scholar 

  5. Boyan JA, Littman ML (1994) Packet routing in dynamically changing networks: a reinforcement learning approach. In: Advances in Neural Information Processing Systems, pp 671–678

  6. Choung O-H, Lee SW, Jeong Y (2017) Exploring feature dimensions to learn a new policy in an uninformed reinforcement learning task. Sci Rep 7(1):17676

    Article  Google Scholar 

  7. Dattatraya KN, Rao KR (2019) Hybrid based cluster head selection for maximizing network lifetime and energy efficiency in wsn. Journal of King Saud University - Computer and Information Sciences

  8. Ding X-X, Ling M, Wang Z-J, Song F-L (2017) Dk-leach: an optimized cluster structure routing method based on leach in wireless sensor networks. Wirel Pers Commun 96(4):6369–6379

    Article  Google Scholar 

  9. Fanian F, Kuchaki Rafsanjani M, Borumand Saeid A (2021) Fuzzy multi-hop clustering protocol: selection fuzzy input parameters and rule tuning for wsns. Applied Soft Computing 99:106923

    Article  Google Scholar 

  10. Goswami P, Yan Z, Mukherjee A, Yang L, Routray S, Palai G (2019) An energy efficient clustering using firefly and hml for optical wireless sensor network. Optik 182:181–185

    Article  Google Scholar 

  11. Gupta HP, Rao SV, Venkatesh T (2016) Sleep scheduling protocol for k-coverage of three-dimensional heterogeneous wsns. IEEE Trans Veh Technol 65:8423–8431

    Article  Google Scholar 

  12. Ha I, Djuraev M, Ahn B (2017) An optimal data gathering method for mobile sinks in wsns. Wirel Pers Commun 97(1):1401–1417

    Article  Google Scholar 

  13. Han R, Yang W, Wang Y, You K (2017) Dce: a distributed energy-efficient clustering protocol for wireless sensor network based on double-phase cluster-head election. Sensors 17(5):998

    Article  Google Scholar 

  14. Harb H, Makhoul A, Laiymani D, Jaber A (2017) A distance-based data aggregation technique for periodic sensor networks. ACM Transactions on Sensor Networks (TOSN) 13(4):32

    Article  Google Scholar 

  15. Hassan A, Anter A, Kayed M (2021) A robust clustering approach for extending the lifetime of wireless sensor networks in an optimized manner with a novel fitness function. Sus Comput Inf Syst 30:100482

    Google Scholar 

  16. Karunanithy K, Velusamy B (2020) Cluster-tree based energy efficient data gathering protocol for industrial automation using wsns and iot. J. Ind. Inf. Integr. 19:100156

    Google Scholar 

  17. Khan T, Singh K, Hasan MH, Ahmad K, Reddy GT, Mohan S, Ahmadian A (2021) Eters: a comprehensive energy aware trust-based efficient routing scheme for adversarial wsns. Futur Gener Comput Syst 125:921–943

    Article  Google Scholar 

  18. Kia G, Hassanzadeh A (2019) A multi-threshold long life time protocol with consistent performance for wireless sensor networks. AEU - International Journal of Electronics and Communications 101:114–127

    Article  Google Scholar 

  19. Kinoshita K, Inoue N, Tanigawa Y, Tode H, Watanabe T (2016) Fair routing for overlapped cooperative heterogeneous wireless sensor networks. IEEE Sensors J 16:3981–3988

    Article  Google Scholar 

  20. Kozlowski M, McConville R, Santos-Rodriguez R, Piechocki R (2018) Energy efficiency in reinforcement learning for wireless sensor networks. arXiv:1812.02538

  21. Krishnan AM, Kumar PG (2016) An effective clustering approach with data aggregation using multiple mobile sinks for heterogeneous wsn. Wirel Pers Commun 90(2):423–434

    Article  Google Scholar 

  22. Liu F, Chang Y (2019) An energy aware adaptive kernel density estimation approach to unequal clustering in wireless sensor networks. IEEE Access 7:40569–40580

    Article  Google Scholar 

  23. Maheshwari P, Sharma AK, Verma K (2021) Energy efficient cluster based routing protocol for wsn using butterfly optimization algorithm and ant colony optimization. Ad Hoc Networks 110:102317

    Article  Google Scholar 

  24. Mann PS, Singh S (2017) Energy efficient clustering protocol based on improved metaheuristic in wireless sensor networks. J Netw Comput Appl 83:40–52

    Article  Google Scholar 

  25. Peng W, Li C, Zhang G, Yi J (2020) Interval type-2 fuzzy logic based transmission power allocation strategy for lifetime maximization of wsns. Engineering Applications of Artificial Intelligence 87:103269

    Article  Google Scholar 

  26. Raj Priyadarshini R, Sivakumar N (2018) Cluster head selection based on minimum connected dominating set and bi-partite inspired methodology for energy conservation in wsns. Journal of King Saud University - Computer and Information Sciences

  27. Shankar T, Shanmugavel S, Rajesh A (2016) Hybrid hsa and pso algorithm for energy efficient cluster head selection in wireless sensor networks. Swarm Evol. Comput. 30:1–10

    Article  Google Scholar 

  28. Singh M, Soni SK (2021) Network lifetime enhancement of wsns using correlation model and node selection algorithm. Ad Hoc Networks 114:102441

    Article  Google Scholar 

  29. Sood T, Sharma K (2020) Luet: A novel lines-of-uniformity based clustering protocol for heterogeneous-wsn for multiple-applications. Journal of King Saud University - Computer and Information Sciences

  30. Wang Q, Xu K, Takahara G, Hassanein H (2006) On lifetime-oriented device provisioning in heterogeneous wireless sensor networks: approaches and challenges. IEEE Netw 20:26–33

    Article  Google Scholar 

  31. Wang T, Zhang G, Yang X, Vajdi A (2018) Genetic algorithm for energy-efficient clustering and routing in wireless sensor networks. J Syst Softw 146:196–214

    Article  Google Scholar 

  32. Xiao G, Sun N, Lv L, Ma J, Chen Y (2015) An heed-based study of cell-clustered algorithm in wireless sensor network for energy efficiency. Wirel Pers Commun 81(1):373–386

    Article  Google Scholar 

  33. Yarinezhad R, Hashemi SN (2019) Solving the load balanced clustering and routing problems in wsns with an fpt-approximation algorithm and a grid structure. Pervasive Mob Comput 58:101033

    Article  Google Scholar 

  34. Yarinezhad R, Sabaei M (2021) An optimal cluster-based routing algorithm for lifetime maximization of internet of things. Journal of Parallel and Distributed Computing 156:7–24

    Article  Google Scholar 

  35. Yao Y, Cao Q, Vasilakos AV (2015) Edal: an energy-efficient, delay-aware, and lifetime-balancing data collection protocol for heterogeneous wireless sensor networks. IEEE/ACM Transactions on Networking (TON) 23(3):810–823

    Article  Google Scholar 

  36. Younis O, Fahmy S (2004) Heed: a hybrid, energy-efficient, distributed clustering approach for ad hoc sensor networks. IEEE Trans Mob Comput 3:366–379

    Article  Google Scholar 

  37. Zhu X, Shen L, Yum TP (2009) Hausdorff clustering and minimum energy routing for wireless sensor networks. IEEE Trans Veh Technol 58:990–997

    Article  Google Scholar 

  38. Zhou Y, Wang N, Xiang W (2017) Clustering hierarchy protocol in wireless sensor networks using an improved pso algorithm. IEEE Access 5:2241–2253

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Science and Engineering Department, Thapar Institute of Engineering and Technology, Patiala, Punjab, India

    Upasna Joshi & Rajiv Kumar

Authors
  1. Upasna Joshi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rajiv Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Upasna Joshi.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Joshi, U., Kumar, R. Reinforcement learning based energy efficient protocol for wireless multimedia sensor networks. Multimed Tools Appl 81, 2827–2840 (2022). https://doi.org/10.1007/s11042-021-11387-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11387-w

Keywords

Search

Navigation