skip to main content
10.1145/1298091.1298115acmconferencesArticle/Chapter ViewAbstractPublication PagesmobicomConference Proceedingsconference-collections
Article

Reinforcement learning in multi-agent environment and ant colony for packet scheduling in routers

Published: 22 October 2007 Publication History

Abstract

The packet scheduling in router plays an important role in the sense to achieve QoS differentiation and to optimize the queuing delay, in particular when this optimization is accomplished on all routers of a path between source and destination. In a dynamically changing environment a good scheduling discipline should be also adaptive to the new traffic conditions. To solve this problem we use a multi-agent system in which each agent tries to optimize its own behaviour and communicate with other agents to make global coordination possible. This communication is done by mobile agents. In this paper, we adopt the framework of Markov decision processes applied to multi-agent system and present a pheromone-Q learning approach which combines the standard Q-learning technique with a synthetic pheromone that acts as a communication medium speeding up the learning process of cooperating agents.

References

[1]
Mellouk, A. Quality of service mechanisms in multimedia and real time applications, HERMES Science Publications, Marsh 2007.
[2]
Nichols, K., Blake S., Baker F., and Black D., Definition of the differentiated services field (DS field) in the IPv4 and IPv6 headers, RFC 2474, 1998.
[3]
Sutton, R. S., Barto, A. G. Reinforcement Learning: An Introduction, MIT Press/Bradford Books, March 1998.
[4]
Bonabeau, E., Dorigo, M., Theraulaz, G. From Natural to Artificial Swarm Intelligence, Oxford University Press, 1999.
[5]
Mellouk A., "How to Integrate Quality of Service using Dynamic Routing Approaches for Irregular Traffic's Network", In International Transactions on Computer Science and Engineering (GESTS), ISSN: 1738--6438, vol. 10, pp 85--96, 2005.
[6]
Nouyan, S., Ghizzioli, R., Birattari, M., Dorigo, M. An insect-based algorithm for the dynamic task allocation problem, Künstliche Intelligenz, vol.4/05, pp.25--31, 2005.
[7]
Mellouk A., Lorenz P., Boukerche A., Lee M. H., Impact of Adaptive Quality of Service Based Routing Algorithms in the next generation heterogeneous networks, In IEEE Communication Magazine, Vol. 45, n°2, pp 65--66, 2007.
[8]
Mellouk A., Hoceini S., A Reinforcement Learning Approach for QoS Based Routing Packets in Integrated Service Web Based Systems, In Lecture Notes in Artificial Intelligence, LNAI 3528, Springer-Heidelberg GmbH, vol. 3528, pp 299--305, 2005.
[9]
Hoceini S., Mellouk A., Amirat Y., A New QoS Routing Algorithm in Dynamic Traffic's Network : N-Best Routing Policy based on Reinforcement Learning, In International Transactions on Computer Science and Engineering (GESTS), vol. 8, pp 25--36, 2005.
[10]
Puterman, M., Markov Decision Processes: Discrete Stochastic Dynamic Programming, Wiley-Interscience, 2005.
[11]
Mellouk A., Hoceini S., Amirat Y., Adaptive Quality of Service Based Routing Approaches: Development of a Neuro-Dynamic State-Dependent Reinforcement Learning Algorithm, In International Journal of Communication Systems, Ed. Wiley-InterSciences, on-line september 2006.
[12]
Watkins, J.C.H. Learning from delayed rewards, PhD thesis, King's College of Cambridge, UK., 1989.
[13]
Christopher, Watkins, J.C.H., Dayan, P. Q-learning. Machine Learning, vol. 3, pp. 279--292, 1992.
[14]
Boutilier C. Sequential Optimality and Coordination in Multiagent Systems, IJCAI, pp. 478--485, 1999.
[15]
Hadeli, Valckenaers, P., Kollingbaum, M., Van Brussel, H., Multi-Agent Coordination and Control Using Stigmergy. Computers in Industry vol. 53, pp. 75--96, 2004.
[16]
Mellouk A., Hoceini S., Larynouna S. Self-Optimization Quality of Service by Adaptive Routing in Dynamic Communication Networks, In International Transactions on Systems Science and Applications, Xiaglow UK Ed., vol. 2, n°3, pp 265--273, 2006.
[17]
Monekosso, N., Remagnino P., Analysis and performance evaluation of the pheromone-Q-learning algorithm, Expert Systems 21 (2), pp 80--91, 2004.
[18]
Kapetanakis S., Kudenko, D. Reinforcement learning of coordination in cooperative multi-agent systems. In AAAI 2002, pp 326--331, 2002.
[19]
Hoceini S., Mellouk A., Amirat Y., K-Shortest Paths Q-Routing: A New QoS Routing Algorithm in Telecommunication Networks, In Lecture Notes in Computer Science, LNCS 3421, Springer-Heidelberg GmbH, vol. 3421, pp 164--172, 2005.
[20]
Mellouk A., Hoceini S., Cheurfa M., Reinforcing Probabilistic Selective Quality of service Routes in Dynamic Heterogeneous Networks, In Journal of Computer Communication, Elsevier Ed., on line March 2007.
[21]
Bourenane MM, Mellouk A., Benhamamouche D., A QoS-based scheduling by Neurodynamic Learning, In System and Information Sciences Journal, Vol. 2, n° 2, pp 138--144, 2007.

Cited By

View all
  • (2022)Stigmergic Independent Reinforcement Learning for Multiagent CollaborationIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2021.305641833:9(4285-4299)Online publication date: Sep-2022
  • (2022)Pheromone-inspired Communication Framework for Large-scale Multi-agent Reinforcement LearningArtificial Neural Networks and Machine Learning – ICANN 202210.1007/978-3-031-15931-2_7(75-86)Online publication date: 7-Sep-2022

Index Terms

  1. Reinforcement learning in multi-agent environment and ant colony for packet scheduling in routers

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      MobiWac '07: Proceedings of the 5th ACM international workshop on Mobility management and wireless access
      October 2007
      196 pages
      ISBN:9781595938091
      DOI:10.1145/1298091
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 22 October 2007

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. QoS routing
      2. ant colony
      3. mobile agent
      4. multi-agent system
      5. packet scheduling
      6. reinforcement learning

      Qualifiers

      • Article

      Conference

      MSWiM07
      Sponsor:

      Acceptance Rates

      Overall Acceptance Rate 83 of 272 submissions, 31%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)3
      • Downloads (Last 6 weeks)0
      Reflects downloads up to 15 Jan 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2022)Stigmergic Independent Reinforcement Learning for Multiagent CollaborationIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2021.305641833:9(4285-4299)Online publication date: Sep-2022
      • (2022)Pheromone-inspired Communication Framework for Large-scale Multi-agent Reinforcement LearningArtificial Neural Networks and Machine Learning – ICANN 202210.1007/978-3-031-15931-2_7(75-86)Online publication date: 7-Sep-2022

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media