Abstract
A wireless sensor network is a potential technique which is most suitable for continuous monitoring applications where the human intervention is not possible. It employs large number of sensor nodes, which will perform various operations like data gathering, transmission and forwarding. An optimal Q-learning based clustering and load balancing technique using improved K-Means algorithm is proposed. It contains two phases namely clustering phase and node balancing phase. The proposed algorithm uses Q-learning technique for deploying sensor nodes in appropriate clusters and cluster head CH election. In the clustering phase, the node will be placed in appropriate clusters based on the computation of the mean values. Once the sensors are placed in an appropriate cluster, then the cluster will be divided into ‘k’ partitions. The node which is having maximum residual energy in each partition will be elected as the partition head PH. In node balancing phase, the number of sensors in each partition will be evenly distributed by considering the area of the cluster and the number of sensors inside the cluster. Among the PHs, the node which is having residual energy to the maximum and also having the minimal distance to the sink is elected as the CH. The residual energy of the CH is monitored periodically. If it falls below the threshold level, then another partition head PH which is having residual energy to the maximum level and possessing minimum distance to the sink node will be elected as CH. The proposed Q-Learning based clustering technique maximize the reward by considering the throughput, end-to-end delay, packet delivery ratio and energy consumption. Finally, the performance of the Q-learning based clustering algorithm is evaluated and compared existing k-means based clustering algorithms. Our results indicate that the proposed method reduces end to end delay by 8.23%, throughput is increased by 2.34%, network lifetime is increased by 3.34%, packet delivery ratio is improved by 1.56%.
Similar content being viewed by others
References
Jin, Y., Kwak, K. S., & Yoo, S.-J.J.I.S.J. (2020). A novel energy supply strategy for stable sensor data delivery in wireless sensor networks. IEEE Systems Journal, 14(3), 3418–3429.
Wang, S., Wan, J., Li, D., & Zhang, C. (2016). Implementing smart factory of industrie 4.0: an outlook. International Journal of Distributed Sensor Networks, 12(1), 3159805.
Rani, S., Maheswar, R., Kanagachidambaresan, G. R., & Jayarajan, P. (2020). Integration of WSN and IoT for Smart Cities. Berlin: Springer.
Huang, J., Hong, Y., Zhao, Z. & Yuan, Y. (2017). An energy-efficient multi-hop routing protocol based on grid clustering for wireless sensor networks. Cluster Computing, 20(4), 3071–3083.
Khan, Z. A., & Samad, A. J. I. J. C. N. A. (2017). A study of machine learning in wireless sensor network. International Journal of Computer Networks And Applications, 4(4), 105–112.
Kumar, D. P., Amgoth, T., & Annavarapu, C. S. R. J. I. F. (2019). Machine learning algorithms for wireless sensor networks: A survey. Information Fusion, 49, 1–25.
Wang, Z., Wang, Z., Liu, Y., Ma, Z., Liu, X., & Ma, J. (2020). LiPSG: lightweight privacy-preserving Q-learning-Based energy management for the IoT-enabled smart grid. IEEE Internet of Things Journal, 7(5), 3935–3947.
Hajjej, F., Hamdi, M., Ejbali, R., & Zaied, M. (2020). A distributed coverage hole recovery approach based on reinforcement learning for Wireless Sensor Networks. Ad Hoc Networks, 101, 102082.
Sharma, A., & Chauhan, S. J. W. N. (2020). A distributed reinforcement learning based sensor node scheduling algorithm for coverage and connectivity maintenance in wireless sensor network. Wireless Networks, 26(6), 4411–4429.
Kosunalp, S. J. I. A. (2016). A new energy prediction algorithm for energy-harvesting wireless sensor networks with Q-learning. IEEE Access, 4, 5755–5763.
Srivastava, V., Tripathi, S. Singh, K., & Son, L. H. (2020). Energy efficient optimized rate based congestion control routing in wireless sensor network. Journal of Ambient Intelligence and Humanized Computing, 11(3), 1325–1338.
Karimi-Bidhendi, S., Guo, J., & H.J.I.T.o.W.C. Jafarkhani, . (2020). Energy-efficient node deployment in heterogeneous two-tier wireless sensor networks with limited communication range. IEEE Transactions on Wireless Communications, 20, 40.
Feng, Y., Zhao, S., & Liu, H. J. I. A. (2020). Analysis of network coverage optimization based on feedback k-means clustering and artificial fish swarm algorithm. Electronics, 8, 42864–42876.
Ahmed, M., Seraj, R., & Islam, S. M. S. J. E. (2020). The k-means algorithm: A comprehensive survey and performance evaluation. Electronics, 9(8), 1295.
Hassan, A. A.-h., Md. Shah, W., Othman, M. F. I., & Hassan, H. A. H. (2020) Evaluate the performance of K-Means and the fuzzy C-Means algorithms to formation balanced clusters in wireless sensor networks. International Journal of Electrical and Computer Engineering 10(2).
Ezenugu, I. A. & D.H. Ugochi, K-means-based energy-aware cluster head selection in wireless sensor networks.
Benmahdi, M. B., & Lehsaini, , M. J. (2020). Computing, Performance evaluation of main approaches for determining optimal number of clusters in wireless sensor networks. IEEE Transactions on Wireless Communications, 33(3), 184–195.
El Khediri, S., et al. (2020). Improved node localization using K-means clustering for Wireless Sensor Networks 37: 100284.
Zhu, B., et al., Improved Soft-k-Means Clustering Algorithm for Balancing Energy Consumption in Wireless Sensor Networks. 2020.
Mostafavi, S., & Hakami, V. J. (2020). A new rank-order clustering algorithm for prolonging the lifetime of wireless sensor networks. International Journal of Communication Systems, 33(7), 4313.
Chandrawanshi, V.S., et al., An intelligent low power consumption routing protocol to extend the lifetime of wireless sensor networks based on fuzzy C-Means++ clustering algorithm. 2020(Preprint): p. 1–10.
Sinaga, K. P., & Yang, M.-S.J.I.A. (2020). Unsupervised K-means clustering algorithm. IEEE Access, 8, 80716–80727.
Mehta, D., (2020) Saxena, and Systems, MCH-EOR: Multi-objective cluster head based energy-aware optimized routing algorithm in wireless sensor networks. 28: 100406.
Augustine, S., & Ananth, J. J. W. N. (2020). Taylor kernel fuzzy C-means clustering algorithm for trust and energy-aware cluster head selection in wireless sensor networks., 26, 5113–5132.
Baradaran, A. A., & Navi, K. J. F. S. (2020). Systems, HQCA-WSN: High-quality clustering algorithm and optimal cluster head selection using fuzzy logic in wireless sensor networks. Fuzzy Sets and Systems, 389, 114–144.
Huang, H. Y., Kim, K. T., & H.Y.J.F.o.C.S. Youn, . (2021). Determining node duty cycle using Q-learning and linear regression for WSN. Frontiers of Computer Science, 15(1), 1–7.
Ahmad, T.J.J.o.I. and O. Sciences, . (2020). Energy EC: An artificial bee colony optimization based energy efficient cluster leader selection for wireless sensor networks. Journal of Information and Optimization Sciences, 41(2), 587–597.
Wang, N.-C., & Hsu, W.-J.J.I.A. (2020). Energy efficient two-tier data dissemination based on Q-learning for wireless sensor networks. IEEE Access, 8, 74129–74136.
Yun, W.-K., & Yoo, S.-J.J.I.A. (2021). Q-learning-based data-aggregation-aware energy-efficient routing protocol for wireless sensor networks. IEEE Access, 9, 10737–10750.
Kalaimani, D., Zah, Z., & Vashist, S. J. A. (2020). Energy-efficient density-based Fuzzy C-means clustering in WSN for smart grids. Australian Journal of Multi-Disciplinary Engineering, 17, 1–16.
Lu, Z., & Shen, H. (2021). Differentially private k-means clustering with convergence guarantee. IEEE Transactions on Dependable and Secure Computing, 18(4), 1541–1552. https://doi.org/10.1109/TDSC.2020.3043369
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Sathyamoorthy, M., Kuppusamy, S., Dhanaraj, R.K. et al. Improved K-Means Based Q Learning Algorithm for Optimal Clustering and Node Balancing in WSN. Wireless Pers Commun 122, 2745–2766 (2022). https://doi.org/10.1007/s11277-021-09028-4
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11277-021-09028-4