Skip to main content
SpringerLink
Log in

A Survey of Reinforcement Learning Based Routing Protocols for Mobile Ad-Hoc Networks

  • Conference paper
Recent Trends in Wireless and Mobile Networks (CoNeCo 2011, WiMo 2011)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 162))

Abstract

Designing mobility and power aware routing protocols have made the main focus of the early contributions to the field of Mobile Ad-hoc NETworks (MANETs). However, almost all conventional routing protocols for MANETs suffer from their lack of adaptivity leading to their performance degradation under varying network conditions. In fact, this is due to both simplistic conception hypotheses they made about the network and to the use of some prefixed parameters in protocols implementations. Currently, artificial intelligence methods like Reinforcement Learning (RL) are widely used to design adaptive routing strategies for MANETs. In this paper, we present a comprehensive survey of RL-based routing protocols for MANETs. Besides, we propose some future research directions in this area.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Sutton, R., Barto, A.: Reinforcement learning. MIT Press, Cambridge (1998)

    Google Scholar 

  2. Togelius, J., Schaul, T., Wierstra, D., Igel, C., Gomez, F., Schmidhuber, J.: Ontogenetic and phylogenetic reinforcement learning. ZeitschriftK unstlicheIntelligenz 3, 30–33 (2009)

    Google Scholar 

  3. Watkins, C.J.: Learning with Delayed Rewards. PhD thesis, Psychology Department, University of Cambridge, UK (1989)

    Google Scholar 

  4. Dowling, J., Cunningham, R., Harrington, A., Curran, E., Cahill, V.: Emergent consensus in decentralised systems using collaborative reinforcement learning. In: Babaoğlu, Ö., Jelasity, M., Montresor, A., Fetzer, C., Leonardi, S., van Moorsel, A., van Steen, M. (eds.) SELF-STAR 2004. LNCS, vol. 3460, pp. 63–80. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  5. Williams, R.J.: Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning 8(3), 229–256 (1992)

    MATH  Google Scholar 

  6. Peshkin, L.: Reinforcement Learning by Policy Search. PhD thesis, Brown University (2001)

    Google Scholar 

  7. Johnson, D.B., Maltz, D.A.: Dynamic source routing in ad hoc wireless networks. In: Mobile Computing, ch. 5, pp. 153–181. Kluwer Academic Publishers, Dordrecht (1996)

    Chapter  Google Scholar 

  8. Perkins, C.E., Royer, E.M.: Ad-hoc on-demand distance vector routing. In: WMCSA 1999, New Orleans, pp. 90–100 (1999)

    Google Scholar 

  9. Perkins, C.E., Watson, T.J.: Highly dynamic destination sequenced distance vector routing (DSDV) for mobile computers. In: ACM SIGCOMM 1994 Conf. on Communications Architectures, London (1994)

    Google Scholar 

  10. Jacquet, P., Muhlethaler, P., Clausen, T., Laouiti, A., Qayyum, A., Viennot, L.: Optimized link state routing protocol for ad hoc networks. In: IEEE INMIC, Pakistan (2001)

    Google Scholar 

  11. Toh, C.: A novel distributed routing protocol to support ad-hoc mobile computing. In: IEEE 15th Annual Int. Phoenix Conf., pp. 480–486 (1996)

    Google Scholar 

  12. Dube, R., Rais, C., Wang, K., Tripathi, S.: Signal stability based adaptive routing (SSA) for ad hoc mobile networks. IEEE Personal Communication 4(1), 36–45 (1997)

    Article  Google Scholar 

  13. Boyan, J.A., Littman, M.L.: Packet routing in dynamically changing networks: A reinforcement learning approach. Advances In Neural Information Processing Systems 6, 671–678 (1994)

    Google Scholar 

  14. Sun, R., Tatsumi, S., Zhao, G.: Q-map: A novel multicast routing method in wireless ad hoc networks with multiagent reinforcement learning. In: Proc. of the IEEE Conf. on Comp., Comm., Control and Power Engineering, vol. 1, pp. 667–670 (2002)

    Google Scholar 

  15. Förster, A.: Machine learning techniques applied to wireless ad hoc networks: Guide and survey. In: Proc. 3rd Int. Conf. on Intelligent Sensors, Sensor Networks and Information Processing (2007)

    Google Scholar 

  16. Chang, Y.-H., Ho, T.: Mobilized ad-hoc networks: A reinforcement learning approach. In: ICAC 2004: Proceedings of the First International Conference on Autonomic Computing, pp. 240–247. IEEE Computer Society, USA (2004)

    Chapter  Google Scholar 

  17. Tao, T., Tagashira, S., Fujita, S.: LQ-Routing Protocol for Mobile Ad-Hoc Networks. In: Proceedings of the Fourth Annual ACIS International Conference on Computer and Information Science (2005)

    Google Scholar 

  18. Chen, S., Nahrstedt, K.: Distributed quality-of-service routing in ad-hoc networks. IEEE Journal on Selected Areas in Communications 17(8), 1488–1505 (1999)

    Article  Google Scholar 

  19. Usaha, W., Barria, J.A.: A reinforcement learning Ticket-Based Probing path discovery scheme for MANETs. Ad Hoc Networks Journal 2, 319–334 (2004)

    Article  Google Scholar 

  20. Maneenil, K., Usaha, W.: Preventing malicious nodes in ad hoc networks using reinforcement learning. In: The 2nd International Symposium on Wireless Communication Systems, Italy, pp. 289–292 (2005)

    Google Scholar 

  21. Dewan, P., Dasgupta, P., Bhattacharya, A.: On using reputations in ad hoc networks to counter malicious nodes. In: Proceedings of Tenth International Conference on Parallel and Distributed Systems, pp. 665–672 (2004)

    Google Scholar 

  22. Dowling, J., Curran, E., Cunningham, R., Cahill, V.: Using feedback in collaborative reinforcement learning to adaptively optimize MANET routing. IEEE Trans. Syst. Man, Cybern. 35, 360–372 (2005)

    Article  Google Scholar 

  23. Binbin, Z., Quan, L., Shouling, Z.: Using statistical network link model for routing in ad hoc networks with multi-agent reinforcement learning. In: International Conference on Advanced Computer Control, pp. 462–466 (2010)

    Google Scholar 

  24. Nurmi, P.: Reinforcement Learning for Routing in Ad Hoc Networks. In: Proc. 5th Intl. Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks. IEEE Computer Society, Los Alamitos (2007)

    Google Scholar 

  25. Bowling, M., Veloso, M.: Rational and convergent learning in stochastic games. In: Proceedings of the 17th International Joint Conference on Artificial Intelligence, pp. 1021–1026. Morgan Kaufmann, San Francisco (2001)

    Google Scholar 

  26. Naruephiphat, W., Usaha, W.: Balancing tradeoffs for energy-efficient routing in MANETs based on reinforcement learning. In: The IEEE 67th Vehicular Technology Conference, Singapore (2008)

    Google Scholar 

  27. Chang, J.H., Tassiulas, L.: Maximum lifetime routing in wireless sensor networks. IEEE/ACM Transactions on Networking 12(4), 609–619 (2004)

    Article  Google Scholar 

  28. Toh, C.K.: Maximum battery life routing to support ubiquitous mobile computing in wireless ad hoc networks. IEEE Communications Magazine 39, 138–147 (2001)

    Article  Google Scholar 

  29. Petrowski, A., Aissanou, F., Benyahia, I., Houcke, S.: Multicriteria reinforcement learning based on a Russian Doll method for network routing. In: 5th IEEE International Conference on Intelligent Systems, United Kingdom (2010)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. SCAL Group, MISC Laboratory, Mentouri University, Constantine, Algeria

    Saloua Chettibi & Salim Chikhi

Authors
  1. Saloua Chettibi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Salim Chikhi
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Girne American University, Girne, TRNC, Turkey

    Abdulkadir Özcan

  2. Mendel University, Brno, Czech Republic

    Jan Zizka

  3. Wireilla Net Solutions PTY Ltd, Melbourne, Victoria, Australia

    Dhinaharan Nagamalai

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Chettibi, S., Chikhi, S. (2011). A Survey of Reinforcement Learning Based Routing Protocols for Mobile Ad-Hoc Networks. In: Özcan, A., Zizka, J., Nagamalai, D. (eds) Recent Trends in Wireless and Mobile Networks. CoNeCo WiMo 2011 2011. Communications in Computer and Information Science, vol 162. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-21937-5_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-21937-5_1

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-21936-8

  • Online ISBN: 978-3-642-21937-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation