Skip to main content
SpringerLink
Log in

A novel approach to accelerate the convergence speed of a stochastic multi-agent system using recurrent neural nets

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

One problem in the design of multi-agent systems is the difficulty of predicting the occurrences that one agent might face, also to recognize and to predict their optimum behavior in these situations. Therefore, one of the most important characteristic of the agent is their ability during adoption, to learn, and correct their behavior. With consideration of the continuously changing environment, the back and forth learning of the agents, the inability to see the agent’s action first hand, and their chosen strategies, learning in a multi-agent environment can be very complex. On the one hand, with recognition to the current learning models that are used in deterministic environment that behaves linearly, which contain weaknesses; therefore, the current learning models are unproductive in complex environments that the actions of agents are stochastic. Therefore, it is necessary for the creation of learning models that are effective in stochastic environments. Purpose of this research is the creation of such a learning model. For this reason, the Hopfield and Boltzmann learning algorithms are used. In order to demonstrate the performance of their algorithms, first, an unlearned multi-agent model is created. During the interactions of the agents, they try to increase their knowledge to reach a specific value. The predicated index is the number of changed states needed to reach the convergence. Then, the learned multi-agent model is created with the Hopfield learning algorithm, and in the end, the learned multi-agent model is created with the Boltzmann learning algorithm. After analyzing the obtained figures, a conclusion can be made that when learning impose to multi-agent environment the average number of changed states needed to reach the convergence decreased and the use of Boltzmann learning algorithm decreased the average number of changed states even further in comparison with Hopfield learning algorithm due to the increase in the number of choices in each situation. Therefore, it is possible to say that the multi-agent systems behave stochastically, the more closer they behave to their true character, the speed of reaching the global solution increases.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Ren Z, Anumba CJ (2003) Learning in multi-agent systems: a case study of construction claims negotiation. Advanc Eng Inf 16:265–275

    Article  Google Scholar 

  2. Conitzer V, Sandholm T (2006) AWESOME: a general multiagent learning algorithm that converges in self—platy and learns a best response against stationary opponents. Mach Learn 67:23–43

    Article  Google Scholar 

  3. Miller J (1993) The educational spectrum: orientation to curriculum. Longman, New York

    Google Scholar 

  4. Vidal JM (2007) Fundamentals of multiagent systems with netlogo examples, pp 19–38, 61–78

  5. Alonso E, d’ Inverno M, kudenko D, Luck M, Noble J (2001) Learning in multi-agent systems. Knowl Eng Rev 16(3):277–284

  6. Banerjee B, Peng J (2004). Performance bounded reinforcement learning in strategic interactions. American Association for Artificial Intelligence, pp 2–7

  7. Carmel D, Markovitch S (1999) Exploration strategies for model-based learning in multi-agent systems. Auton Agent Multi Agent Syst 2:141–172

    Article  Google Scholar 

  8. Busoniu L, Babuska R, Schutter BD (2008) A comprehensive survey of multiagent reinforcement learning. IEEE Trans Syst Man Cybern c Appl Rev 38(2):156–172

    Google Scholar 

  9. Dam N Discussion of learning algorithms in stochastic games. Computer Science Department, Iowa State University, Ames, IA. p 50010

  10. Shoham Y, Powers R, Grenager T (2004) On the agenda(s) of research on multi-agent learning. Stanford University, Stanford

    Google Scholar 

  11. Chang Y, Kaelbling LP (2001) Playing is believing: the role of beliefs in multi-agent learning. In: Proceedings of NIPS. MIT Press, Cambridge

  12. Powers R, Shoham Y, Vu T (2007) A general criterion and an algorithmic framework for learning in multi-agent systems. Mach Learn 67:45–76

    Article  Google Scholar 

  13. Powers R, Shoham Y (2005) New criteria and a new algorithm for learning in multi-agent systems. Computer Science Department, Stanford University, Stanford

  14. Russell SJ, Norvig P (2003) Artificial intelligence a modern approach. Prentice Hall, New Jersey

    Google Scholar 

  15. Wooldridge M (1996) An introduction to multiagent systems. Wiley, England, pp 105–125, 190–220

  16. Stone P, Veloso M (2000) Multiagent systems: a survey from a machine learning perspective, Auton Robotics 8(3):345–383

    Google Scholar 

  17. Nourafza N, Setayeshi S (2008) Learning based on genetic algorithm and fuzzy genetic. In: National conference of information Technology, Islamic Azad University, Najafabad, Iran

  18. Nourafza N, Setayeshi S, Rahman A (2009) Identifying models and algorithms learning chalenges in multi-agent systems. In: Proceeding of the first Iranian national conference on software engineering, Islamic Azad University, Roudehen, Iran

  19. Ben-David S, Kushilevitz E, Mansour Y (1997) Online learning versus offline learning. Mach learn 29:45–63

    Article  MATH  Google Scholar 

  20. Freeman JA, Skapura DM (1991) Neural networks algorithms, applications, and programming techniques. Addison-Wesley Publication Company, California

  21. Beale R, Jackson T (1990) Introduction to neural networks. Institute of Physics Publishing, Bristol and Philadelphia

  22. Haykin S (1999) Neural networks a comprehensive foundation. Prentice Hall, London

  23. Zhang P (2000) Neural networks in optimization. Springer, New York

  24. Hopfield JJ (1984) Neurons with graded response have collective computational properties like those of two-state neurons. Proc Natl Acad Sci USA 81:3088–3092

    Article  Google Scholar 

  25. Hopfield JJ (1982) Neural networks and physical systems with emergent collective computational abilities. Proc Natl Acad Sci USA 79:2554–2558

    Article  MathSciNet  Google Scholar 

  26. Ackley DH, Hinton GE, Sejnowski TJ (1985) A learning algorithm for Boltzmann machine. Cogn Sci 9:147–169

    Article  Google Scholar 

  27. Kirkpatrick S, Gelatt CD, Vecchi MP (1983) Optimization by simulated annealing. Science 220:671–680

    Article  MathSciNet  MATH  Google Scholar 

  28. Krose B, Smagt P (1996) An introduction to neural networks

  29. Fyfe C (1996) Artificial neural networks. Lecture notes

  30. Anderson D, McNeill G (1992) Artificial neural networks technology. DACS report

  31. Fausett L (1994) Fundamentals of neural networks, architectures, algorithms, and applications. Prentice-Hall, London

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Faculty of Engineering, Science and Research Branch, Islamic Azad University (IAU), Tehran, Iran

    Nasim Nourafza

  2. Department of Medical Radiation Engineering, Amirkabir University of Technology (Tehran Polytechnic), Tehran, Iran

    Saeed Setayeshi

  3. Iran Telecom Research Center, North Kargar St., Tehran, Iran

    Ahmad Khadem-Zadeh

  4. No: 81766-55951, Shahid Majid Javadi 4 alley, Vahid St., Isfahan, Iran

    Nasim Nourafza

Authors
  1. Nasim Nourafza
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Saeed Setayeshi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ahmad Khadem-Zadeh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nasim Nourafza.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Nourafza, N., Setayeshi, S. & Khadem-Zadeh, A. A novel approach to accelerate the convergence speed of a stochastic multi-agent system using recurrent neural nets. Neural Comput & Applic 21, 2015–2021 (2012). https://doi.org/10.1007/s00521-011-0624-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-011-0624-4

Keywords

Search

Navigation