Skip to main content

Advertisement

Springer Nature Link
Log in

An intelligent fault detection approach based on reinforcement learning system in wireless sensor network

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

The Internet of Things (IoT) has developed a well-defined infrastructure due to commercializing novel technologies. IoT networks enable smart devices to compile environmental information and transmit it to demanding users through an IoT gateway. The explosive increase of IoT users and sensors causes network bottlenecks, leading to significant energy depletion in IoT devices. The wireless network is a robust, empirically significant, and IoT layer based on progressive characteristics. The development of energy-efficient routing protocols for learning purposes is critical due to environmental volatility, unpredictability, and randomness in the wireless network’s weight distribution. To achieve this critical need, learning-based routing systems are emerging as potential candidates due to their high degree of flexibility and accuracy. However, routing becomes more challenging in dynamic IoT networks due to the time-varying characteristics of link connections and access status. Hence, modern learning-based routing systems must be capable of adapting in real-time to network changes. This research presents an intelligent fault detection, energy-efficient, quality-of-service routing technique based on reinforcement learning to find the optimum route with the least amount of end-to-end latency. However, the cluster head selection is dependent on residual energy from the cluster nodes that reduce the entire network’s existence. Consequently, it extends the network’s lifetime, overcomes the data transmission’s energy usage, and improves network robustness. The experimental results indicate that network efficiency has been successfully enhanced by fault-tolerance strategies that include highly trusted computing capabilities, thus decreasing the risk of network failure.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Ko YB, Vaidya NH (2000) Location-aided routing (lar) in mobile ad hoc networks. Wirel Netw, 6(4):307–321

    Article  Google Scholar 

  2. Biswas S, Das R, Chatterjee P (2018) Energy-efficient connected target coverage in multi-hop wireless sensor networks. Ind Interactive Innov Sci Eng Technol, pp 411–421. Springer

  3. Sotheara S, Aso K, Aomi N, Shimamoto S (2014) Effective data gathering and energy efficient communication protocol in wireless sensor networks employing uav. In: 2014 IEEE wireless communications and networking Conference (WCNC), pages 2342–2347. IEEE

  4. Amutha J, Sharma S, Sharma SK (2021) Strategies based on various aspects of clustering in wireless sensor networks using classical, optimization and machine learning techniques: review, taxonomy, research findings, challenges and future directions. Comput Sci Revi, 40:1–45

    MathSciNet  Google Scholar 

  5. Diallo O, Rodrigues JJ, Sene M (2012) Real-time data management on wireless sensor networks: a survey. J Netw Comput Appl, 35(3), 1013–1021

    Article  Google Scholar 

  6. Kanoun O, Bradai S, Khriji S, Bouattour G, El Houssaini D, Ben Ammar M, Naifar S, Bouhamed A., Derbel F, Viehweger C, (2021) Energy-aware system design for autonomous wireless sensor nodes: a comprehensive review. Sensors, 21(2): 1–25

    Article  Google Scholar 

  7. Papadimitratos P and Haas Z (2002) Secure routing for mobile ad hoc networks. In: communication networks and distributed systems modeling and simulation Conference (CNDS 2002), number CONF, pp 1–13. SCS

  8. Heinzelman WR, Chandrakasan A, Balakrishnan H (2000) Energy-efficient communication protocol for wireless microsensor networks. In: Proceedings of the 33rd annual Hawaii international Conference on system sciences, pages 1–10. IEEE

  9. Chawla N, Jasuja A (2014) Algorithm for optimizing first node die (fnd) time in leach protocol. Int J Curr Eng Technol, 4(4): 2748–2750

    Google Scholar 

  10. Tiberti W, Cassioli D, Di Marco A, Pomante L, Santic M (2021) A model-based approach for adaptable middleware evolution in wsn platforms. J Sensor Actuator Netw, 10(1): 1–22

    Google Scholar 

  11. Qiu M, Ming Z, Li J, Liu J, Quan G, Zhu Y (2013) Informer homed routing fault tolerance mechanism for wireless sensor networks. J Syst Archit, 59(4–5): 260–270

    Article  Google Scholar 

  12. Mahmood T, Akhtar F, Ur Rehman K, Ali S, Mokbal FM, Daudpota S (2019) A comprehensive survey on the performance analysis of underwater wireless sensor networks (uwsn) routing protocols. IJACSA, 10(5):1–11

    Article  Google Scholar 

  13. Littman Michael L (2015) Reinforcement learning improves behaviour from evaluative feedback. Nature, 521(7553): 445–451

    Article  Google Scholar 

  14. Wang D, Liu J, Yao D, Member IEEE (2020) An energy-efficient distributed adaptive cooperative routing based on reinforcement learning in wireless multimedia sensor networks. Comput Netw, pp 1–12

  15. Banerjee A, Sufian A (2020) Reinforcement learning based transmission range control (rl-trc) in sd-wsn with moving sensors. arXiv preprint arXiv:2005.08215, pp 1–27

  16. Sivakumar P, Radhika M (2018) Performance analysis of leach-ga over leach and leach-c in wsn. Procedia Comput Sci, 125:248–256

    Article  Google Scholar 

  17. Azharuddin M, Jana Prasanta K (2015) A distributed algorithm for energy efficient and fault tolerant routing in wireless sensor networks. Wirel Netw, 21(1), 251–267

    Article  Google Scholar 

  18. Shi F, Tuo X, Yang SX, Lu J, Li H (2019) Rapid-flooding time synchronization for large-scale wireless sensor networks. IEEE Trans Ind Inf, 16(3): 1581–1590

    Article  Google Scholar 

  19. Mahmood T, Akhtar F, Rehman KU, Azeem M, Mudassir A, Daudpota SM (2020) Introducing robustness in dbr routing protocol. Int J Commun Netw Distrib Syst, 24(3): 316–338

    Google Scholar 

  20. Tripathi M, Gaur MS, Laxmi V, Battula RB (2013) Energy efficient leach-c protocol for wireless sensor network. pp 1–4

  21. Ali B, Mahmood T, Mirza MA, Memon S, Rashid M, Ajebesone EF (2019) Study and analysis of delay sensitive and energy efficient routing approach. IJACSA, 10(8):14–20

    Google Scholar 

  22. Moon SY, Cho TH (2009) Intrusion detection scheme against sinkhole attacks in directed diffusion based sensor networks. Int J Comput Sci Netw Secur, 9(7): 118–122

    Google Scholar 

  23. Loscri V, Morabito G, Marano S (2005) A two-levels hierarchy for low-energy adaptive clustering hierarchy (tl-leach). In: IEEE vehicular technology conference, 62, pp 1809–1813. IEEE

  24. Ko YB, Choi JM, and Kim JH (2004) A new directional flooding protocol for wireless sensor networks. In: international Conference on information networking, pp 93–102. Springer

  25. Zhe H, Maiko S (2009) New bounds on the minimum number of calls in failure-tolerant gossiping. Netw Int J, 53(1):35–38

    MathSciNet  MATH  Google Scholar 

  26. Dehkordi SA, Farajzadeh K, Rezazadeh J, Farahbakhsh R, Sandrasegaran K, Dehkordi MA (2020) A survey on data aggregation techniques in iot sensor networks. Wirel Netw, 26(2): 1243–1263

    Article  Google Scholar 

  27. Lindsey S and Raghavendra CS (2020) Pegasis: power-efficient gathering in sensor information systems. In: Proceedings, IEEE aerospace conference, 3: 1125–1130. IEEE

  28. Arati M and Agrawal Dharma P (2001) Teen: arouting protocol for enhanced efficiency in wireless sensor networks. In ipdps, 1: 1–7

    Google Scholar 

  29. Yao Y, Gehrke J, et al. (2003) Query processing in sensor networks. In Cidr, pp 233–244

  30. Younis O, Fahmy S (2004) Heed: a hybrid, energy-efficient, distributed clustering approach for ad hoc sensor networks. IEEE Trans Mobile Comput, 3(4):366–379

    Article  Google Scholar 

  31. Qiu M, Liu J, Li J, Fei Z, Ming Z, Edwin HM (2011) A novel energy-aware fault tolerance mechanism for wireless sensor networks. In: 2011 IEEE/ACM international Conference on green computing and communications, pp 56–61. IEEE

  32. Liu X, Cao J, Bhuiyan M, Lai S, Wu H, Wang G (2011) Fault tolerant wsn-based structural health monitoring. In: 2011 IEEE/IFIP 41st international Conference on dependable systems and networks (DSN), pp 37–48. IEEE

  33. Cheraghlou MN, Haghparast M (2014) A novel fault-tolerant leach clustering protocol for wireless sensor networks. J Circuits Syst Comput, 23(03): 1–17

    Article  Google Scholar 

  34. Mehdi M, A Al-Fuqaha, M Guizani, JS Oh (2017) Semisupervised deep reinforcement learning in support of iot and smart city services. IEEE Internet Things J, 5(2): 624–635

    Google Scholar 

  35. Tang F, Zhou, Y, Kato N (2020) Deep reinforcement learning for dynamic uplink/downlink resource allocation in high mobility 5g hetnet. IEEE J Sel Areas Commun, 38(12): 2773–2782

    Article  Google Scholar 

  36. Ajmi N, Helali A, Lorenz P, Mghaieth R (2021) Speech-mac: special purpose energy-efficient contention-based hybrid mac protocol for wsn and zigbee network. Int J Commun Syst, 34(1): 1–18

    Google Scholar 

  37. Zhao T, Xu XB, Wang SG (2020) Centralized q-learning based routing in eh-wsns with dual alternative batteries. J Phys Conf Series, 1544: 1–10. IOP Publishing

  38. Seah MWM, Tham CK, Srinivasan V, Xin A (2007) Achieving coverage through distributed reinforcement learning in wireless sensor networks. In: 2007 3rd international Conference on intelligent sensors, sensor networks and information, pp 425–430. IEEE

  39. Wang CW, Xia Q, Yao X, Wang W, Jornet JM (2018) Multi-hop deflection routing algorithm based on q-learning for energy-harvesting nanonetworks. In: 2018 IEEE 15th international Conference on mobile ad hoc and sensor systems (MASS), pp 362–370. IEEE

  40. Bouzid SE, Serrestou Y, Raoof K, Omri MN (2020) Efficient routing protocol for wireless sensor network based on reinforcement learning. In 2020 5th International Conference on Advanced Technologies for Signal and Image Processing (ATSIP), pp 1–5

  41. Sun P, Lan J, Guo Z, Xu Y, Hu Y (2020) Improving the scalability of deep reinforcement learning-based routing with control on partial nodes. In: ICASSP 2020-2020 IEEE international Conference on acoustics, speech and signal processing (ICASSP), pp 3557–3561. IEEE

  42. Raj RN, Nayak A, Kumar MS (2020) A survey and performance evaluation of reinforcement learning based spectrum aware routing in cognitive radio ad hoc networks. Int J Wirel Inf Netw, 27(1): 144–163

    Article  Google Scholar 

  43. Yau KLA., Komisarczuk P, Teal PD (2012) Reinforcement learning for context awareness and intelligence in wireless networks: review, new features and open issues. J Netw Comput Appl, 35(1), 253–267

    Article  Google Scholar 

  44. Sutton RS, Barto AG (2018) Reinforcement learning: an introduction. MIT press

  45. Junping H, Yuhui J and Liang D (2008) A time-based cluster-head selection algorithm for leach. In: 2008 IEEE symposium on computers and communications, pp. 1172–1176. IEEE

  46. Ali MS, Dey T, Biswas R (2008) Aleach: advanced leach routing protocol for wireless microsensor networks. In: 2008 international Conference on electrical and computer engineering, pp. 909–914. IEEE

  47. Batra PK, Kant K (2016) Leach-mac: a new cluster head selection algorithm for wireless sensor networks. Wirel Netw, 22(1): 49–60

    Article  Google Scholar 

  48. Subhashree VK, Tharini C, Swarna LM (2014) Modified leach: a qos-aware clustering algorithm for wireless sensor networks. In: 2014 international Conference on communication and network technologies, pages 119–123. IEEE

  49. Kumar GS, Vinu PMV, Jacob KP (2008) Mobility metric based leach-mobile protocol. In: 2008 16th international Conference on advanced computing and communications, pages 248–253. IEEE, 2008

  50. Sharma V, Alam B, Doja MN (2019) An improvement in dsr routing protocol of manets using anfis. In applications of artificial intelligence techniques in engineering, pages 569–576. Springer

Download references

Author information

Authors and Affiliations

  1. Faculty of Information Technology, Beijing University of Technology, Beijing, 100124, China

    Tariq Mahmood, Jianqiang Li & Sirajuddin Qureshi

  2. Department of Information Sciences, Division of Science and Technology, University of Education, Lahore, 54000, Pakistan

    Tariq Mahmood, Suhail Ashfaq Butt & Allah Ditta

  3. Beijing Engineering Research Center for IoT Software and Systems, Beijing, 100124, China

    Jianqiang Li

  4. Computer Science Division, University of Aizu, Aizuwakamatsu, Fukushima, 965-8580, Japan

    Yan Pei

  5. Department of Computer Science, Sukkur IBA University, Sukkur, 65200, Pakistan

    Faheem Akhtar

Authors
  1. Tariq Mahmood
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Jianqiang Li
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Yan Pei
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Faheem Akhtar
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Suhail Ashfaq Butt
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Allah Ditta
    View author publications

    You can also search for this author inPubMed Google Scholar

  7. Sirajuddin Qureshi
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Yan Pei.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mahmood, T., Li, J., Pei, Y. et al. An intelligent fault detection approach based on reinforcement learning system in wireless sensor network. J Supercomput 78, 3646–3675 (2022). https://doi.org/10.1007/s11227-021-04001-1

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-021-04001-1

Keywords