Skip to main content
SpringerLink
Log in

Coordinating Agents in Dynamic Environment

  • Conference paper
  • First Online:
Enterprise Information Systems (ICEIS 2013)

Part of the book series: Lecture Notes in Business Information Processing ((LNBIP,volume 190))

Included in the following conference series:

  • 997 Accesses

Abstract

This paper presents strategies for speeding up the convergence of agents on swarm. Speeding up the learning of an agent is a complex task since the choice of inadequate updating techniques may cause delays in the learning process or even induce an unexpected acceleration that causes the agent to converge to a non-satisfactory policy. We have developed strategies for updating policies which combines local and global search using past policies. Experimental results in dynamic environments of different dimensions have shown that the proposed strategies are able to speed up the convergence of the agents while achieving optimal action policies, improving the coordination of agents in the swarm while deliberating.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    www.iwr.uni-heidelberg.de/groups/comopt/software/TSPLIB95/

References

  1. Wooldridge, M.J.: An Introduction to MultiAgent Systems. Wiley, Chichester (2002)

    Google Scholar 

  2. Ribeiro, R., Favarim F., Barbosa, M.A.C., Borges, A.P., Dordal, B.O., Koerich, A.L., Enembreck, F.: Unified algorithm to improve reinforcement learning in dynamic environments: an instance-based approach. In: 14th International Conference on Enterprise Information Systems (ICEIS’12), Wroclaw, Poland, pp. 229–238 (2012)

    Google Scholar 

  3. Mihaylov, M., Tuyls, K., Nowé, A.: Decentralized learning in wireless sensor networks. In: Taylor, M.E., Tuyls, K. (eds.) ALA 2009. LNCS, vol. 5924, pp. 60–73. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  4. Chaharsooghi, S.K., Heydari, J., Zegordi, S.H.: A reinforcement learning model for supply chain ordering management: an application to the beer game. J. Decision Support Syst. 45(4), 949–959 (2008)

    Article  Google Scholar 

  5. Dorigo, M.: Optimization, Learning and Natural Algorithms. Ph.D. thesis, Politecnico di Milano, Itália (1992)

    Google Scholar 

  6. Ribeiro, R., Enembreck, F.: A sociologically inspired heuristic for optimization algorithms: a case study on ant systems. expert systems with applications. Expert Syst. Appl. 40(5), 1814–1826 (2012)

    Article  Google Scholar 

  7. Sudholt, D.: Theory of swarm intelligence. In: Proceedings of the 13th Annual Conference Companion on Genetic and Evolutionary Computation (GECCO ‘11), pp. 1381–1410. ACM, New York (2011)

    Google Scholar 

  8. Dorigo, M., Gambardella, L.M.: A study of some properties of Ant-Q. In: Proceedings of PPSN Fourth International Conference on Parallel Problem Solving From Nature, pp. 656–665 (1996)

    Google Scholar 

  9. Ribeiro, R., Borges, A.P., Enembreck, F.: Interaction models for multiagent reinforcement learning. In: International Conference on Computational Intelligence for Modelling Control and Automation - CIMCA08, Vienna, Austria, pp. 1–6 (2008)

    Google Scholar 

  10. Gambardella, L.M., Dorigo, M.: Ant-Q: a reinforcement learning approach to the TSP. In: Proceedings of ML-95, Twelfth International Conference on Machine Learning, pp. 252–260 (1995)

    Google Scholar 

  11. Reinelt, G.: TSPLIB - a traveling salesman problem library. ORSA J. Comput. 3, 376–384 (1991)

    Article  MATH  Google Scholar 

  12. Dorigo, M., Maniezzo, V., Colorni, A.: Ant system: optimization by a colony of cooperting agents. IEEE Trans. Syst., Man, Cybern.-Part B 26(1), 29–41 (1996)

    Article  Google Scholar 

  13. Watkins, C.J.C.H., Dayan, P.: Q-Learning. Mach. Learn. 8(3), 279–292 (1992)

    MATH  Google Scholar 

  14. Guntsch, M., Middendorf, M.: Applying population based ACO to dynamic optimization problems. In: Proceedings of Third International Workshop ANTS, pp. 111–122 (2003)

    Google Scholar 

  15. Sim, K.M., Sun, W.H.: Multiple ant-colony optimization for network routing. In: Proceedings of the First International Symposium on Cyber Worlds, pp. 277–281 (2002)

    Google Scholar 

  16. Li, Y., Gong, S.: Dynamic ant colony optimization for TSP. Int. J. Adv. Manuf. Technol. 22(7–8), 528–533 (2003)

    Article  Google Scholar 

  17. Lee, S.G., Jung, T.U., Chung, T.C.: Improved ant agents system by the dynamic parameter decision. In Proceedings of the IEEE International Conference on Fuzzy Systems, pp. 666–669 (2001)

    Google Scholar 

  18. Gambardella, L.M., Taillard, E.D., Dorigo, M.: Ant colonies for the QAP. Technical report, IDSIA, Lugano, Switzerland (1997)

    Google Scholar 

  19. Stutzle, T., Hoos, H.: MAX-MIN Ant system and local search for the traveling salesman problem. In: Proceedings of the IEEE International Conference on Evolutionary Computation, pp. 309–314 (1997)

    Google Scholar 

  20. Guntsch, M., Middendorf, M.: Pheromone modification strategies for ant algorithms applied to dynamic TSP. In: Proceedings of the Workshop on Applications of Evolutionary Computing, pp. 213–222 (2001)

    Google Scholar 

  21. Christofides, N., Eilon, S.: Expected distances in distribution problems. Oper. Res. Q. 20, 437–443 (1969)

    Article  Google Scholar 

  22. Tesauro, G.: Temporal difference learning and TD-Gammon. Commun. ACM 38(3), 58–68 (1995)

    Article  Google Scholar 

  23. Enembreck, F., Ávila, B.C., Scalabrin, E.E., Barthes, J.P.: Distributed constraint optimization for scheduling in CSCWD. In: International Conference on Computer Supported Cooperative Work in Design, Santiago, vol. 1, pp. 252–257 (2009)

    Google Scholar 

  24. Hao, J., Leung, H.-F.: The dynamics of reinforcement social learning in cooperative multiagent systems. In: Proceedings of the 23rd. International Joint Conference on Artificial Intelligence (IJCAI’13), Beijing, China, pp. 184–190 (2013)

    Google Scholar 

  25. Kötzing, T., Frank, N., Röglin, H., Witt, C.: Theoretical analysis of two ACO approaches for the traveling salesman problem. Swarm Intell. 6(1), 1–21 (2012)

    Article  Google Scholar 

  26. Brambilla, M., Ferrante, E., Birattari, M., Dorigo, M.: Swarm robotics: a review from the swarm engineering perspective. Swarm Intell. 7(1), 1–41 (2013)

    Article  Google Scholar 

Download references

Acknowledgements

This research is supported by the Program for Research Support of UTFPR - campus Pato Branco, DIRPPG (Directorate of Research and Post-Graduation) and Fundação Araucária (Araucaria Foundation of Parana State).

Author information

Authors and Affiliations

  1. Department of Informatic, Federal University of Technology, Pato Branco, Parana, Brazil

    Richardson Ribeiro & Marco A. C. Barbosa

  2. Graduate School in Electrical Engineering & Industrial Computer Science (CPGEI), Federal University of Technology, Curitiba, Parana, Brazil

    Adriano F. Ronszcka

  3. Post-Graduate Program in Computer Science, Pontificial Catholical University, Curitiba, Parana, Brazil

    Fabrício Enembreck

Authors
  1. Richardson Ribeiro
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Adriano F. Ronszcka
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marco A. C. Barbosa
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fabrício Enembreck
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Richardson Ribeiro .

Editor information

Editors and Affiliations

  1. Groupe ESEO, Angers, France

    Slimane Hammoudi

  2. Polytechnic Institute of Setúbal, Setúbal, Portugal & INSTICC, Setúbal, Portugal

    José Cordeiro

  3. Wroclaw University of Economics, Wroclaw, Poland & Macquarie University, Sydney, NSW, Australia

    Leszek A. Maciaszek

  4. Polytechnic Institute of Setúbal, Setúbal, Portugal & INSTICC, Setúbal, Poland

    Joaquim Filipe

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this paper

Cite this paper

Ribeiro, R., Ronszcka, A.F., Barbosa, M.A.C., Enembreck, F. (2014). Coordinating Agents in Dynamic Environment. In: Hammoudi, S., Cordeiro, J., Maciaszek, L., Filipe, J. (eds) Enterprise Information Systems. ICEIS 2013. Lecture Notes in Business Information Processing, vol 190. Springer, Cham. https://doi.org/10.1007/978-3-319-09492-2_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-09492-2_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-09491-5

  • Online ISBN: 978-3-319-09492-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation