Skip to main content
Springer Nature Link
Log in

MRL-SCSO: Multi-agent Reinforcement Learning-Based Self-Configuration and Self-Optimization Protocol for Unattended Wireless Sensor Networks

  • Published:
Wireless Personal Communications Aims and scope Submit manuscript

An Erratum to this article was published on 25 October 2016

This article has been updated

Abstract

Resource-constrained nodes in unattended wireless sensor network (UWSN) operate in a hostile environment with less human intervention. Achieving the optimal quality of service (QoS) in terms of packet delivery ratio, delay, energy, and throughput is crucial. In this paper, we propose a topology control and data dissemination protocol that uses multi-agent reinforcement learning (MRL) and energy-aware convex-hull algorithm, for effective self-configuration and self-optimization (SCSO) in UWSN, called MRL-SCSO. MRL-SCSO maintains a reliable topology in which the effective active neighbor nodes are selected using MRL. The network boundary is determined using convex-hull algorithm to maintain the connectivity and coverage of the network. The boundary nodes transmit data under high traffic load conditions. The performance of MRL-SCSO is evaluated for various nodes count and under different load conditions by using the Contiki’s Cooja simulator. The results showed that MRL-SCSO stabilizes the performance and improves QoS.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Change history

  • 25 October 2016

    An erratum to this article has been published.

References

  1. Misra, S., & Jain, A. (2011). Policy controlled self-configuration in unattended wireless sensor networks. Journal of Network and Computer Applications, 34(5), 1530–1544.

    Article  Google Scholar 

  2. Song, W., Huang, R., Xu, M., Ma, A., Shirazi, B., & LaHusen, R. (2009). Air-dropped sensor network for real-time high-fidelity volcano monitoring. In Proceedings of the 7th international conference on mobile systems, applications, and services (pp. 305–318). ACM.

  3. Atzori, L., Iera, A., & Morabito, G. (2010). The internet of things: A survey. Computer Networks, 54(15), 2787–2805.

    Article  MATH  Google Scholar 

  4. Geng, W., Talwar, S., Johnsson, K., Himayat, N., & Johnson, K. D. (2011). M2 M: From mobile to embedded internet. IEEE Communications Magazine, 49(4), 36–43.

    Article  Google Scholar 

  5. He, T., Krishnamurthy, S., Stankovic, J. A., Abdelzaher, T., Luo, L., Stoleru, R., Yan, T., Gu, L., Hui, J., & Krogh, B.(2004). Energy-efficient surveillance system using wireless sensor networks. In Proceedings of the 2nd international conference on mobile systems, applications, and services (pp. 270–283). ACM.

  6. Huang, R., Song, W. Z., Xu, M., Peterson, N., Shirazi, B. A., & LaHusen, R. (2012). Real-world sensor network for long-term volcano monitoring: Design and findings. IEEE Transactions on Parallel and Distributed Systems, 23(2), 321–329.

    Article  Google Scholar 

  7. Hariri, S., Khargharia, B., Chen, H., Yang, J., Zhang, Y., Parashar, M., et al. (2006). The autonomic computing paradigm. Cluster Computing, 9(1), 5–17.

    Article  Google Scholar 

  8. Cerpa, A., & Estrin, D. (2004). ASCENT: Adaptive self-configuring sensor networks topologies. IEEE Transactions on Mobile Computing, 3(3), 272–285.

    Article  Google Scholar 

  9. Winkler, M., Street, M., Tuchs, K. D., & Wrona, K. (2013). Wireless sensor networks for military purposes. In D. Filippini (Ed.), Autonomous sensor networks: Collective sensing strategies for analytical purposes (pp. 365–394). Berlin: Springer.

    Google Scholar 

  10. Li. M., Li, Z., & Vasilako, A.V. (2013). A survey on topology control in wireless sensor networks: Taxonomy, comparative study, and open issues, In Proceedings of the IEEE (pp. 2538–2557).

  11. Aziz, A., Sekercioglu, Y., Fitzpatrick, P., & Ivanovich, M. (2013). A Survey on distributed topology control techniques for extending the lifetime of battery powered wireless sensor networks. IEEE Communications Surveys and Tutorials, 15(1), 121–144.

    Article  Google Scholar 

  12. Renold, A. P., & Chandrakala, S. (2016). Survey on state scheduling-based topology control in unattended wireless sensor networks. Computers & Electrical Engineering,. doi:10.1016/j.compeleceng.2015.12.024.

    Google Scholar 

  13. Ye, F., Zhong, G., Cheng, J., Lu, S., & Zhang, L. (2003). PEAS: A robust energy conserving protocol for long-lived sensor networks. In Proceedings of 23rd IEEE international conference on distributed computing systems (pp. 28–37).

  14. Xu, Y., Heidemann, J., & Estrin, D. (2001). Geography-informed energy conservation for ad hoc routing. In Proceedings of the 7th annual international conference on mobile computing and networking (pp.70–84).

  15. Luo, X., Yan, Y., Li, S., & Guan, X. (2013). Topology control based on optimally rigid graph in wireless sensor networks. Computer Networks, 57(4), 1037–1047.

    Article  Google Scholar 

  16. Buettner, M., Yee, G., Anderson, E., & Han, R. (2006). X-MAC: A short preamble MAC protocol for duty-cycled wireless sensor networks. In Proceedings of the 4th international conference on embedded networked sensor systems (pp. 307–320). ACM.

  17. Lin, P., Qiao, C., & Wang, X. (2004). Medium access control with a dynamic duty cycle for sensor networks. In Proceedings of wireless communications and networking conference (IEEE WCNC’04) (pp. 1534–1539).

  18. Rajendran, V., Obraczka, K., & Garcia-Luna-Aceves, J. J. (2006). Energy-efficient, collision-free medium access control for wireless sensor networks. Wireless Networks, 12(1), 63–78.

    Article  Google Scholar 

  19. Valera, A. C., Soh, W.-S., & Tan, H.-P. (2014). Survey on wakeup scheduling for environmentally-powered wireless sensor networks. Computer Communications, 52(1), 21–36.

    Article  Google Scholar 

  20. Ye, W., Heidemann, J., & Estrin, D. (2002). An energy-efficient MAC protocol for wireless sensor networks. In Proceedings of 21st annual joint conference of the IEEE computer and communications societies (pp. 1567–1576).

  21. Van Dam, T., & Langendoen, K. (2003). An adaptive energy-efficient MAC protocol for wireless sensor networks. In Proceedings of the 1st international conference on embedded networked sensor systems (pp. 171–180). ACM.

  22. Polastre, J., Hill, J., & Culler, D. (2004). Versatile low power media access for wireless sensor networks. In Proceedings of the 2nd international conference on embedded networked sensor systems (pp. 95–107). ACM.

  23. Du, D. Z., & Pardalos, P. (2004). Handbook of combinatorial optimization. Boston: Kluwer Academic Publishers.

    MATH  Google Scholar 

  24. Jie, W., Ming, G., & Stojmenovic, I. (2002). On calculating power-aware connected dominating sets for efficient routing in ad hoc wireless networks. Journal of Communications and Networks, 4(1), 59–70.

    Article  Google Scholar 

  25. Yuanyuan, Z., Jia, X., & Yanxiang, H. (2006). Energy efficient distributed connected dominating sets construction in wireless sensor networks. In Proceedings of the international conference on wireless communications and mobile computing (pp. 797–802). ACM.

  26. Aziz, A. A., & Sekercioglu, Y. A. (2012). A distributed topology control method for improving energy efficiency of wireless sensor networks. In Proceeding of 4th international conference on intelligent and advanced systems (ICIAS) (pp. 247–252). IEEE.

  27. Cormen, T. H., Leiserson, C. E., Rivest, R. L., & Stein, C. (2001). Introduction to algorithms (2nd ed.). Cambridge, MA: MIT Press.

    MATH  Google Scholar 

  28. Xu, R., Dai, H., Wang, F., & Jia, Z. (2013). A convex hull based optimization to reduce the data delivery latency of the mobile elements in wireless sensor networks. In Proceedings of 10th IEEE international conference on embedded and ubiquitous computing (pp. 2245–2252).

  29. Guo, P., Cao, J., & Zhang, K. (2015). Distributed topological convex hull estimation of event region in wireless sensor networks without location information. IEEE Transactions on Parallel and Distributed Systems, 26(1), 85–94.

    Article  Google Scholar 

  30. Senel, F., Younis, M., & Akkaya, K. (2011). Bio-inspired relay node placement heuristics for repairing damaged wireless sensor networks. IEEE Transactions on Vehicular Technology, 60(4), 1835–1848.

    Article  Google Scholar 

  31. Lee, S., Younis, M., & Lee, M. (2015). Connectivity restoration in a partitioned wireless sensor network with assured fault tolerance. Ad Hoc Networks, 24, 1–19.

    Article  Google Scholar 

  32. Xia, B., Wahab, M. H., Yang, Y., Fan, Z., & Sooriyabandara, M. (2009). Reinforcement learning based spectrum-aware routing in multi-hop cognitive radio networks. In Proceeding of 4th international conference on cognitive radio oriented wireless networks and communications (CROWNCOM’09) (pp. 1–5).

  33. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction. Cambridge: MIT Press.

    Google Scholar 

  34. Forster, A. (2007). Machine learning techniques applied to wireless ad-hoc networks: Guide and survey. In Proceeding of 3rd international conference on intelligent sensors, sensor networks and information (ISSNIP 2007) (pp. 365–370). IEEE.

  35. Kennedy, J., & Eberhart, R. (1995). Particle swarm optimization. In Proceedings of IEEE international conference on neural networks (pp. 1942–1948).

  36. Gen, M., & Cheng, R. (1999). Genetic algorithms and engineering optimization. New York, NY: Wiley.

    Book  Google Scholar 

  37. Baccour, N., Koubâa, A., Mottola, L., Zúñiga, M. A., Youssef, H., Boano, C. A., et al. (2012). Radio link quality estimation in wireless sensor networks: A survey. ACM Transactions on Sensor Networks, 8(4), 1–33.

    Article  Google Scholar 

  38. Cerpa, A., Wong, J. L., Kuang, L., Potkonjak, M., & Estrin, D. (2005). Statistical model of lossy links in wireless sensor networks. In Proceedings of the 4th international symposium on information processing in sensor networks (IPSN’05) (pp. 81–88).

  39. De Couto, D. S., Aguayo, D., Bicket, J., & Morris, R. (2005). A high-throughput path metric for multihop wireless routing. Wireless Networks, 11(4), 419–434.

    Article  Google Scholar 

  40. Kim, K. H., & Shin, K. G. (2006). On accurate measurement of link quality in multihop wireless mesh networks. In Proceedings of the 12th annual international conference on mobile computing and networking (pp. 38–49). ACM.

  41. Gnawali, O., Fonseca, R., Jamieson, K., Moss, D., & Levis, P. (2009). Collection tree protocol. In Proceedings of the 7th ACM conference on embedded networked sensor systems (SenSys’09) (pp. 90–100).

  42. Lal, D., Manjeshwar, A., & Herrmann, F. (2003). Measurement and characterization of link quality metrics in energy constrained wireless sensor networks. In Proceedings of the IEEE global telecommunications conference (Globecom’03) (pp. 446–452).

  43. Jiang, P., Huang, Q., Wang, J., Dai, X., & Lin, R. (2006). Research on wireless sensor networks routing protocol for wetland water environment monitoring. In Proceedings of the 1st international conference on innovative computing, information and control (ICICIC’06) (pp. 251–254). IEEE.

  44. Dunkels, A., Osterlind, F., Tsiftes, N., & He, Z. (2007). Software-based on-line energy estimation for sensor nodes, In Proceedings of the 4th workshop on embedded networked sensors (pp. 28-32). ACM.

  45. Schurgers, C., & Srivastava, M. B. (2001). Energy efficient routing in wireless sensor networks. In MILCOM on communications for network-centric operations: Creating the information force (pp. 357–361). IEEE.

  46. Dunkels, A., Österlind, F., & He, Z. (2007). An adaptive communication architecture for wireless sensor networks, In Proceedings of the 5th international conference on embedded networked sensor systems (pp. 335–349). ACM.

  47. Bourke, P. (1988). Calculating the area and centroid of polygon. http://www.seas.upenn.edu/~sys502/extra_materials/Polygon%20Area%20and%20Centroid.pdf.

  48. Eriksson, J., Osterlind, F., Finne, N., Dunkels, A., Tsiftes, N., & Voigt, T. (2009). Accurate network-scale power profiling for sensor network simulators. In Wireless sensor networks, lecture notes in computer science (pp. 312–326). Berlin/Heidelberg: Springer.

  49. Polastre, J., Szewczyk, R., & Culler, D. (2005). Telos: Enabling ultra-low power wireless research. In Proceedings of the 4th international symposium on information processing in sensor networks (pp. 364–369). IEEE.

  50. Grep. Access Aug. 12, 2015. http://www.gmu.org/software/grep/.

  51. Wireshark Tool. Access Aug. 22, 2015. http://www.wireshark.org/.

Download references

Author information

Authors and Affiliations

  1. Mobile Ad-hoc and Sensor Network Lab, TIFAC-CORE in Pervasive Computing Technologies, Velammal Engineering College, Chennai, India

    A. Pravin Renold

  2. Department of Computer Science and Engineering, Sri Krishna College of Engineering and Technology, Coimbatore, India

    S. Chandrakala

Authors
  1. A. Pravin Renold
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. S. Chandrakala
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to A. Pravin Renold.

Additional information

An erratum to this article is available at https://doi.org/10.1007/s11277-016-3832-5.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Renold, A.P., Chandrakala, S. MRL-SCSO: Multi-agent Reinforcement Learning-Based Self-Configuration and Self-Optimization Protocol for Unattended Wireless Sensor Networks. Wireless Pers Commun 96, 5061–5079 (2017). https://doi.org/10.1007/s11277-016-3729-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11277-016-3729-3

Keywords