Abstract
With the advancements in sensor networks, Wireless multimedia sensor networks (WMSNs) have emerged and shifted the objectives of sensor nodes to multimedia devices which can retrieve audio, images, and video. In WMSNs, the sensor nodes are tiny microphones and cameras which can transmit image, audio or video using the network. However, these nodes are battery constrained (i.e., may become dead after passing certain iterations). Therefore, improvement of the network lifetime is a challenging issue of WMSNs. In this paper, a reinforcement-based energy-aware protocol is designed and implemented. To successfully implement the reinforcement-based protocol, a State-Action-Reward-State-Action (SARSA) is used for learning a Markov decision process. Extensive experiments are considered to evaluate the significant improvement of the proposed protocol. Comparisons are also drawn between the competitive protocols and the proposed protocol. From comparative analysis, it is found that the proposed protocol conserves more energy as compared to the competitive protocols.




Similar content being viewed by others
References
Ahmed G, Zou J, Fareed MMS, Zeeshan M (2016) Sleep-awake energy efficient distributed clustering algorithm for wireless sensor networks. Comput. Electr. Eng. 56:385–398
Amuthan A, Arulmurugan A (2018) Semi-markov inspired hybrid trust prediction scheme for prolonging lifetime through reliable cluster head selection in wsns. J. King Saud Univ. - Comput. Inf. Sci.
Aslam N, Xia K, Hadi MU (2019) Optimal wireless charging inclusive of intellectual routing based on sarsa learning in renewable wireless sensor networks. IEEE Sensors J 19:8340–8351
Baradaran AA, Navi K (2020) Hqca-wsn: high-quality clustering algorithm and optimal cluster head selection using fuzzy logic in wireless sensor networks. Fuzzy Sets Syst 389:114–144. Clustering
Boyan JA, Littman ML (1994) Packet routing in dynamically changing networks: a reinforcement learning approach. In: Advances in Neural Information Processing Systems, pp 671–678
Choung O-H, Lee SW, Jeong Y (2017) Exploring feature dimensions to learn a new policy in an uninformed reinforcement learning task. Sci Rep 7(1):17676
Dattatraya KN, Rao KR (2019) Hybrid based cluster head selection for maximizing network lifetime and energy efficiency in wsn. Journal of King Saud University - Computer and Information Sciences
Ding X-X, Ling M, Wang Z-J, Song F-L (2017) Dk-leach: an optimized cluster structure routing method based on leach in wireless sensor networks. Wirel Pers Commun 96(4):6369–6379
Fanian F, Kuchaki Rafsanjani M, Borumand Saeid A (2021) Fuzzy multi-hop clustering protocol: selection fuzzy input parameters and rule tuning for wsns. Applied Soft Computing 99:106923
Goswami P, Yan Z, Mukherjee A, Yang L, Routray S, Palai G (2019) An energy efficient clustering using firefly and hml for optical wireless sensor network. Optik 182:181–185
Gupta HP, Rao SV, Venkatesh T (2016) Sleep scheduling protocol for k-coverage of three-dimensional heterogeneous wsns. IEEE Trans Veh Technol 65:8423–8431
Ha I, Djuraev M, Ahn B (2017) An optimal data gathering method for mobile sinks in wsns. Wirel Pers Commun 97(1):1401–1417
Han R, Yang W, Wang Y, You K (2017) Dce: a distributed energy-efficient clustering protocol for wireless sensor network based on double-phase cluster-head election. Sensors 17(5):998
Harb H, Makhoul A, Laiymani D, Jaber A (2017) A distance-based data aggregation technique for periodic sensor networks. ACM Transactions on Sensor Networks (TOSN) 13(4):32
Hassan A, Anter A, Kayed M (2021) A robust clustering approach for extending the lifetime of wireless sensor networks in an optimized manner with a novel fitness function. Sus Comput Inf Syst 30:100482
Karunanithy K, Velusamy B (2020) Cluster-tree based energy efficient data gathering protocol for industrial automation using wsns and iot. J. Ind. Inf. Integr. 19:100156
Khan T, Singh K, Hasan MH, Ahmad K, Reddy GT, Mohan S, Ahmadian A (2021) Eters: a comprehensive energy aware trust-based efficient routing scheme for adversarial wsns. Futur Gener Comput Syst 125:921–943
Kia G, Hassanzadeh A (2019) A multi-threshold long life time protocol with consistent performance for wireless sensor networks. AEU - International Journal of Electronics and Communications 101:114–127
Kinoshita K, Inoue N, Tanigawa Y, Tode H, Watanabe T (2016) Fair routing for overlapped cooperative heterogeneous wireless sensor networks. IEEE Sensors J 16:3981–3988
Kozlowski M, McConville R, Santos-Rodriguez R, Piechocki R (2018) Energy efficiency in reinforcement learning for wireless sensor networks. arXiv:1812.02538
Krishnan AM, Kumar PG (2016) An effective clustering approach with data aggregation using multiple mobile sinks for heterogeneous wsn. Wirel Pers Commun 90(2):423–434
Liu F, Chang Y (2019) An energy aware adaptive kernel density estimation approach to unequal clustering in wireless sensor networks. IEEE Access 7:40569–40580
Maheshwari P, Sharma AK, Verma K (2021) Energy efficient cluster based routing protocol for wsn using butterfly optimization algorithm and ant colony optimization. Ad Hoc Networks 110:102317
Mann PS, Singh S (2017) Energy efficient clustering protocol based on improved metaheuristic in wireless sensor networks. J Netw Comput Appl 83:40–52
Peng W, Li C, Zhang G, Yi J (2020) Interval type-2 fuzzy logic based transmission power allocation strategy for lifetime maximization of wsns. Engineering Applications of Artificial Intelligence 87:103269
Raj Priyadarshini R, Sivakumar N (2018) Cluster head selection based on minimum connected dominating set and bi-partite inspired methodology for energy conservation in wsns. Journal of King Saud University - Computer and Information Sciences
Shankar T, Shanmugavel S, Rajesh A (2016) Hybrid hsa and pso algorithm for energy efficient cluster head selection in wireless sensor networks. Swarm Evol. Comput. 30:1–10
Singh M, Soni SK (2021) Network lifetime enhancement of wsns using correlation model and node selection algorithm. Ad Hoc Networks 114:102441
Sood T, Sharma K (2020) Luet: A novel lines-of-uniformity based clustering protocol for heterogeneous-wsn for multiple-applications. Journal of King Saud University - Computer and Information Sciences
Wang Q, Xu K, Takahara G, Hassanein H (2006) On lifetime-oriented device provisioning in heterogeneous wireless sensor networks: approaches and challenges. IEEE Netw 20:26–33
Wang T, Zhang G, Yang X, Vajdi A (2018) Genetic algorithm for energy-efficient clustering and routing in wireless sensor networks. J Syst Softw 146:196–214
Xiao G, Sun N, Lv L, Ma J, Chen Y (2015) An heed-based study of cell-clustered algorithm in wireless sensor network for energy efficiency. Wirel Pers Commun 81(1):373–386
Yarinezhad R, Hashemi SN (2019) Solving the load balanced clustering and routing problems in wsns with an fpt-approximation algorithm and a grid structure. Pervasive Mob Comput 58:101033
Yarinezhad R, Sabaei M (2021) An optimal cluster-based routing algorithm for lifetime maximization of internet of things. Journal of Parallel and Distributed Computing 156:7–24
Yao Y, Cao Q, Vasilakos AV (2015) Edal: an energy-efficient, delay-aware, and lifetime-balancing data collection protocol for heterogeneous wireless sensor networks. IEEE/ACM Transactions on Networking (TON) 23(3):810–823
Younis O, Fahmy S (2004) Heed: a hybrid, energy-efficient, distributed clustering approach for ad hoc sensor networks. IEEE Trans Mob Comput 3:366–379
Zhu X, Shen L, Yum TP (2009) Hausdorff clustering and minimum energy routing for wireless sensor networks. IEEE Trans Veh Technol 58:990–997
Zhou Y, Wang N, Xiang W (2017) Clustering hierarchy protocol in wireless sensor networks using an improved pso algorithm. IEEE Access 5:2241–2253
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Joshi, U., Kumar, R. Reinforcement learning based energy efficient protocol for wireless multimedia sensor networks. Multimed Tools Appl 81, 2827–2840 (2022). https://doi.org/10.1007/s11042-021-11387-w
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-021-11387-w