Skip to main content

Advertisement

SpringerLink
Log in

Improved K-Means Based Q Learning Algorithm for Optimal Clustering and Node Balancing in WSN

  • Published:
Wireless Personal Communications Aims and scope Submit manuscript

Abstract

A wireless sensor network is a potential technique which is most suitable for continuous monitoring applications where the human intervention is not possible. It employs large number of sensor nodes, which will perform various operations like data gathering, transmission and forwarding. An optimal Q-learning based clustering and load balancing technique using improved K-Means algorithm is proposed. It contains two phases namely clustering phase and node balancing phase. The proposed algorithm uses Q-learning technique for deploying sensor nodes in appropriate clusters and cluster head CH election. In the clustering phase, the node will be placed in appropriate clusters based on the computation of the mean values. Once the sensors are placed in an appropriate cluster, then the cluster will be divided into ‘k’ partitions. The node which is having maximum residual energy in each partition will be elected as the partition head PH. In node balancing phase, the number of sensors in each partition will be evenly distributed by considering the area of the cluster and the number of sensors inside the cluster. Among the PHs, the node which is having residual energy to the maximum and also having the minimal distance to the sink is elected as the CH. The residual energy of the CH is monitored periodically. If it falls below the threshold level, then another partition head PH which is having residual energy to the maximum level and possessing minimum distance to the sink node will be elected as CH. The proposed Q-Learning based clustering technique maximize the reward by considering the throughput, end-to-end delay, packet delivery ratio and energy consumption. Finally, the performance of the Q-learning based clustering algorithm is evaluated and compared existing k-means based clustering algorithms. Our results indicate that the proposed method reduces end to end delay by 8.23%, throughput is increased by 2.34%, network lifetime is increased by 3.34%, packet delivery ratio is improved by 1.56%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Jin, Y., Kwak, K. S., & Yoo, S.-J.J.I.S.J. (2020). A novel energy supply strategy for stable sensor data delivery in wireless sensor networks. IEEE Systems Journal, 14(3), 3418–3429.

    Article  Google Scholar 

  2. Wang, S., Wan, J., Li, D., & Zhang, C. (2016). Implementing smart factory of industrie 4.0: an outlook. International Journal of Distributed Sensor Networks, 12(1), 3159805.

    Article  Google Scholar 

  3. Rani, S., Maheswar, R., Kanagachidambaresan, G. R., & Jayarajan, P. (2020). Integration of WSN and IoT for Smart Cities. Berlin: Springer.

    Book  Google Scholar 

  4. Huang, J., Hong, Y., Zhao, Z. & Yuan, Y. (2017). An energy-efficient multi-hop routing protocol based on grid clustering for wireless sensor networks. Cluster Computing, 20(4), 3071–3083.

    Article  Google Scholar 

  5. Khan, Z. A., & Samad, A. J. I. J. C. N. A. (2017). A study of machine learning in wireless sensor network. International Journal of Computer Networks And Applications, 4(4), 105–112.

    Article  Google Scholar 

  6. Kumar, D. P., Amgoth, T., & Annavarapu, C. S. R. J. I. F. (2019). Machine learning algorithms for wireless sensor networks: A survey. Information Fusion, 49, 1–25.

    Article  Google Scholar 

  7. Wang, Z., Wang, Z., Liu, Y., Ma, Z., Liu, X., & Ma, J. (2020). LiPSG: lightweight privacy-preserving Q-learning-Based energy management for the IoT-enabled smart grid. IEEE Internet of Things Journal, 7(5), 3935–3947.

    Article  Google Scholar 

  8. Hajjej, F., Hamdi, M., Ejbali, R., & Zaied, M. (2020). A distributed coverage hole recovery approach based on reinforcement learning for Wireless Sensor Networks. Ad Hoc Networks, 101, 102082.

    Article  Google Scholar 

  9. Sharma, A., & Chauhan, S. J. W. N. (2020). A distributed reinforcement learning based sensor node scheduling algorithm for coverage and connectivity maintenance in wireless sensor network. Wireless Networks, 26(6), 4411–4429.

    Article  Google Scholar 

  10. Kosunalp, S. J. I. A. (2016). A new energy prediction algorithm for energy-harvesting wireless sensor networks with Q-learning. IEEE Access, 4, 5755–5763.

    Article  Google Scholar 

  11. Srivastava, V., Tripathi, S. Singh, K., & Son, L. H. (2020). Energy efficient optimized rate based congestion control routing in wireless sensor network. Journal of Ambient Intelligence and Humanized Computing, 11(3), 1325–1338.

    Article  Google Scholar 

  12. Karimi-Bidhendi, S., Guo, J., & H.J.I.T.o.W.C. Jafarkhani, . (2020). Energy-efficient node deployment in heterogeneous two-tier wireless sensor networks with limited communication range. IEEE Transactions on Wireless Communications, 20, 40.

    Article  Google Scholar 

  13. Feng, Y., Zhao, S., & Liu, H. J. I. A. (2020). Analysis of network coverage optimization based on feedback k-means clustering and artificial fish swarm algorithm. Electronics, 8, 42864–42876.

    Google Scholar 

  14. Ahmed, M., Seraj, R., & Islam, S. M. S. J. E. (2020). The k-means algorithm: A comprehensive survey and performance evaluation. Electronics, 9(8), 1295.

    Article  Google Scholar 

  15. Hassan, A. A.-h., Md. Shah, W., Othman, M. F. I., & Hassan, H. A. H. (2020) Evaluate the performance of K-Means and the fuzzy C-Means algorithms to formation balanced clusters in wireless sensor networks. International Journal of Electrical and Computer Engineering 10(2).

  16. Ezenugu, I. A. & D.H. Ugochi, K-means-based energy-aware cluster head selection in wireless sensor networks.

  17. Benmahdi, M. B., & Lehsaini, , M. J. (2020). Computing, Performance evaluation of main approaches for determining optimal number of clusters in wireless sensor networks. IEEE Transactions on Wireless Communications, 33(3), 184–195.

    Google Scholar 

  18. El Khediri, S., et al. (2020). Improved node localization using K-means clustering for Wireless Sensor Networks 37: 100284.

  19. Zhu, B., et al., Improved Soft-k-Means Clustering Algorithm for Balancing Energy Consumption in Wireless Sensor Networks. 2020.

  20. Mostafavi, S., & Hakami, V. J. (2020). A new rank-order clustering algorithm for prolonging the lifetime of wireless sensor networks. International Journal of Communication Systems, 33(7), 4313.

    Article  Google Scholar 

  21. Chandrawanshi, V.S., et al., An intelligent low power consumption routing protocol to extend the lifetime of wireless sensor networks based on fuzzy C-Means++ clustering algorithm. 2020(Preprint): p. 1–10.

  22. Sinaga, K. P., & Yang, M.-S.J.I.A. (2020). Unsupervised K-means clustering algorithm. IEEE Access, 8, 80716–80727.

    Article  Google Scholar 

  23. Mehta, D., (2020) Saxena, and Systems, MCH-EOR: Multi-objective cluster head based energy-aware optimized routing algorithm in wireless sensor networks. 28: 100406.

  24. Augustine, S., & Ananth, J. J. W. N. (2020). Taylor kernel fuzzy C-means clustering algorithm for trust and energy-aware cluster head selection in wireless sensor networks., 26, 5113–5132.

    Google Scholar 

  25. Baradaran, A. A., & Navi, K. J. F. S. (2020). Systems, HQCA-WSN: High-quality clustering algorithm and optimal cluster head selection using fuzzy logic in wireless sensor networks. Fuzzy Sets and Systems, 389, 114–144.

    Article  MathSciNet  Google Scholar 

  26. Huang, H. Y., Kim, K. T., & H.Y.J.F.o.C.S. Youn, . (2021). Determining node duty cycle using Q-learning and linear regression for WSN. Frontiers of Computer Science, 15(1), 1–7.

    Article  Google Scholar 

  27. Ahmad, T.J.J.o.I. and O. Sciences, . (2020). Energy EC: An artificial bee colony optimization based energy efficient cluster leader selection for wireless sensor networks. Journal of Information and Optimization Sciences, 41(2), 587–597.

    Article  Google Scholar 

  28. Wang, N.-C., & Hsu, W.-J.J.I.A. (2020). Energy efficient two-tier data dissemination based on Q-learning for wireless sensor networks. IEEE Access, 8, 74129–74136.

    Article  Google Scholar 

  29. Yun, W.-K., & Yoo, S.-J.J.I.A. (2021). Q-learning-based data-aggregation-aware energy-efficient routing protocol for wireless sensor networks. IEEE Access, 9, 10737–10750.

    Article  Google Scholar 

  30. Kalaimani, D., Zah, Z., & Vashist, S. J. A. (2020). Energy-efficient density-based Fuzzy C-means clustering in WSN for smart grids. Australian Journal of Multi-Disciplinary Engineering, 17, 1–16.

    Google Scholar 

  31. Lu, Z., & Shen, H. (2021). Differentially private k-means clustering with convergence guarantee. IEEE Transactions on Dependable and Secure Computing, 18(4), 1541–1552. https://doi.org/10.1109/TDSC.2020.3043369

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Kongu Engineering College, Erode, 638060, India

    Malathy Sathyamoorthy & Sangeetha Kuppusamy

  2. School of Computing Science and Engineering, Galgotias University, Greater Noida, Uttar Pradesh, India

    Rajesh Kumar Dhanaraj

  3. Center for Artificial Intelligence, Prince Mohammad Bin Fahd University, Khobar, Saudi Arabia

    Vinayakumar Ravi

Authors
  1. Malathy Sathyamoorthy
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sangeetha Kuppusamy
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rajesh Kumar Dhanaraj
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Vinayakumar Ravi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Malathy Sathyamoorthy.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sathyamoorthy, M., Kuppusamy, S., Dhanaraj, R.K. et al. Improved K-Means Based Q Learning Algorithm for Optimal Clustering and Node Balancing in WSN. Wireless Pers Commun 122, 2745–2766 (2022). https://doi.org/10.1007/s11277-021-09028-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11277-021-09028-4

Keywords

Search

Navigation