Skip to main content
SpringerLink
Log in

Multiagent Learning through Neuroevolution

  • Chapter
Advances in Computational Intelligence (WCCI 2012)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 7311))

Included in the following conference series:

Abstract

Neuroevolution is a promising approach for constructing intelligent agents in many complex tasks such as games, robotics, and decision making. It is also well suited for evolving team behavior for many multiagent tasks. However, new challenges and opportunities emerge in such tasks, including facilitating cooperation through reward sharing and communication, accelerating evolution through social learning, and measuring how good the resulting solutions are. This paper reviews recent progress in these three areas, and suggests avenues for future work.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 16.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Acerbi, A., Nolfi, S.: Social learning and cultural evolution in embodied and situated agents. In: IEEE Symposium on Artificial Life, ALIFE 2007, pp. 333–340. IEEE (2007)

    Google Scholar 

  2. Acerbi, A., Parisi, D.: Cultural transmission between and within generations. Journal of Artificial Societies and Social Simulation 9(1) (2006)

    Google Scholar 

  3. Arneson, R.: Egalitarianism. In: Zalta, E.N. (ed.) The Stanford Encyclopedia of Philosophy. Stanford University (2009)

    Google Scholar 

  4. Boehm, C.: Hierarchy in the forest: The evolution of egalitarian behavior. Harvard Univ. Pr. (2001)

    Google Scholar 

  5. Byrne, R., Whiten, A.: Machiavellian Intelligence: Social Expertise and the Evolution of Intellect in Monkeys, Apes, and Humans. Oxford University Press, USA (1989)

    Google Scholar 

  6. Czirok, A., Vicsek, T.: Collective behavior of interacting self-propelled particles. Physica A 281, 17–29 (2000)

    Article  Google Scholar 

  7. de Oca, M., Stutzle, T., Van den Enden, K., Dorigo, M.: Incremental social learning in particle swarms. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 41(2), 368–384 (2011)

    Article  Google Scholar 

  8. Denaro, D., Parisi, D.: Cultural evolution in a population of neural networks. In: Marinaro, M., Tagliaferri, R. (eds.) Neural Nets Wirn 1996, pp. 100–111. Springer, Newyork (1996)

    Google Scholar 

  9. Dorigo, M., Maniezzo, V., Colorni, A.: Ant system: optimization by a colony of cooperating agents. IEEE Transactions on Systems, Man, and Cybernetics–Part B 26(1), 29–41 (1996)

    Article  Google Scholar 

  10. Erda, L., Whiten, A., Mellars, P., Gibson, K.: Egalitarianism and Machiavellian intelligence in human evolution. McDonald Institute for Archaeological Research (1996)

    Google Scholar 

  11. Floreano, D., Dürr, P., Mattiussi, C.: Neuroevolution: From architectures to learning. Evolutionary Intelligence 1, 47–62 (2008)

    Article  Google Scholar 

  12. Floreano, D., Urzelai, J.: Evolutionary robots with on-line self-organization and behavioral fitness. Neural Networks 13, 431–4434 (2000)

    Article  Google Scholar 

  13. Fogel, D.B.: Blondie24: Playing at the Edge of AI. Morgan Kaufmann, San Francisco (2001)

    Google Scholar 

  14. Fogel, L., Owens, A., Walsh, M.: Artificial intelligence through simulated evolution. John Wiley (1966)

    Google Scholar 

  15. Gomez, F., Miikkulainen, R.: Incremental evolution of complex general behavior. Adaptive Behavior, 317–342 (1997)

    Google Scholar 

  16. Gomez, F., Miikkulainen, R.: Active guidance for a finless rocket using neuroevolution. In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 2084–2095. Morgan Kaufmann, San Francisco (2003)

    Google Scholar 

  17. Haasdijk, E., Vogt, P., Eiben, A.: Social learning in population-based adaptive systems. In: IEEE Congress on Evolutionary Computation, CEC 2008 (IEEE World Congress on Computational Intelligence), pp. 1386–1392. IEEE (2008)

    Google Scholar 

  18. Herrmann, E., Call, J., Hernández-Lloreda, M., Hare, B., Tomasello, M.: Humans have evolved specialized skills of social cognition: The cultural intelligence hypothesis. Science 317(5843), 1360 (2007)

    Article  Google Scholar 

  19. Humphrey, N.: The social function of intellect. Growing Points in Ethology, 303–317 (1976)

    Google Scholar 

  20. Karpov, I.V., Sheblak, J., Miikkulainen, R.: OpenNERO: A game platform for AI research and education. In: Proceedings of the Fourth Artificial Intelligence and Interactive Digital Entertainment Conference (2008)

    Google Scholar 

  21. Kohl, N., Stanley, K.O., Miikkulainen, R., Samples, M., Sherony, R.: Evolving a real-world vehicle warning system. In: Proceedings of the Genetic and Evolutionary Computation Conference (2006)

    Google Scholar 

  22. Kubota, N., Shimojima, K., Fukuda, T.: Virus-evolutionary genetic algorithm -coevolution of planar grid model. In: Proceedings of the Fifth IEEE International Conference on Fuzzy Systems (FUZZ IEEE 1996), pp. 8–11 (1996)

    Google Scholar 

  23. Lipson, H., Pollack, J.B.: Automatic design and manufacture of robotic lifeforms. Nature 406, 974–978 (2000)

    Article  Google Scholar 

  24. Miikkulainen, R.: Neuroevolution. In: Encyclopedia of Machine Learning. Springer, Berlin (2010)

    Google Scholar 

  25. Mitchell, M., Thomure, M.D., Williams, N.L.: The role of space in the success of coevolutionary learning. In: Artificial Life X: Proceedings of the Tenth International Conference on the Simulation and Synthesis of Living Systems, pp. 118–124 (2006)

    Google Scholar 

  26. Moriarty, D.E., Miikkulainen, R.: Forming neural networks through efficient and adaptive coevolution. Evolutionary Computation 5(4), 373–399 (1997)

    Article  Google Scholar 

  27. Nowak, M.A.: Five rules for the evolution of cooperation. Science 314, 1560–1563 (2006)

    Article  Google Scholar 

  28. Parisi, D.: Cultural evolution in neural networks. IEEE Expert 12(4), 9–14 (1997)

    Article  Google Scholar 

  29. Pérez-Uribe, A., Floreano, D., Keller, L.: Effects of Group Composition and Level of Selection in the Evolution of Cooperation in Artificial Ants. In: Banzhaf, W., Ziegler, J., Christaller, T., Dittrich, P., Kim, J.T. (eds.) ECAL 2003. LNCS (LNAI), vol. 2801, pp. 128–137. Springer, Heidelberg (2003)

    Chapter  Google Scholar 

  30. Rawal, A., Rajagopalan, P., Miikkulainen, R.: Constructing competitive and cooperative agent behavior using coevolution. In: IEEE Conference on Computational Intelligence and Games (CIG 2010) (August 2010)

    Google Scholar 

  31. Reynolds, C.W.: Flocks, herds, and schools: A distributed behavioral model. In: Computer Graphics (SIGGRAPH 1987 Conference Proceedings), vol. 21(4), pp. 25–34 (1987)

    Google Scholar 

  32. Reynolds, R.: An introduction to cultural algorithms. In: Proceedings of the Third Annual Conference on Evolutionary Programming, pp. 131–139. World Scientific (1994)

    Google Scholar 

  33. Roeva, O., Pencheva, T., Tzonkov, S., Arndt, M., Hitzmann, B., Kleist, S., Miksch, G., Friehs, K., Flaschel, E.: Multiple model approach to modelling of escherichia coli fed-batch cultivation extracellular production of bacterial phytase. Electronic Journal of Biotechnology 10(4), 592–603 (2007)

    Article  Google Scholar 

  34. Rumelhart, D., Hinton, G., Williams, R.: Learning representations by back-propagating errors. Nature 323(6088), 533–536 (1986)

    Article  Google Scholar 

  35. Schrum, J., Miikkulainen, R.: Evolving agent behavior in multiobjective domains using fitness-based shaping. In: Proceedings of the Genetic and Evolutionary Computation Conference (2010)

    Google Scholar 

  36. Schrum, J., Miikkulainen, R.: Evolving multimodal networks for multitask games. In: Proceedings of the IEEE Conference on Computational Intelligence and Games (CIG 2011), pp. 102–109. IEEE, Seoul (2011)

    Chapter  Google Scholar 

  37. Seno, H.: A density-dependent diffusion model of shoaling of nesting fish. Ecol. Modell. 51, 217–226 (1990)

    Article  Google Scholar 

  38. Simpson, G.: The baldwin effect. Evolution 7(2), 110–117 (1953)

    Article  Google Scholar 

  39. Stanley, K., Miikkulainen, R.: Evolving neural networks through augmenting topologies. Evolutionary Computation 10(2), 99–127 (2002)

    Article  Google Scholar 

  40. Stanley, K.O., Bryant, B.D., Miikkulainen, R.: Real-time neuroevolution in the NERO video game. IEEE Transactions on Evolutionary Computation 9(6), 653–668 (2005)

    Article  Google Scholar 

  41. Stanley, K.O., Bryant, B.D., Miikkulainen, R.: Real-time neuroevolution in the NERO video game. IEEE Transactions on Evolutionary Computation 9(6), 653–668 (2005)

    Article  Google Scholar 

  42. Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)

    Google Scholar 

  43. Thain, D., Tannenbaum, T., Livny, M.: Distributed computing in practice: the condor experience. Concurrency - Practice and Experience 17(2-4), 323–356 (2005)

    Article  Google Scholar 

  44. Tomasello, M.: The cultural origins of human cognition. Harvard Univ. Pr. (1999)

    Google Scholar 

  45. Valsalam, V., Miikkulainen, R.: Evolving symmetry for modular system design. IEEE Transactions on Evolutionary Computation 15, 368–386 (2011)

    Article  Google Scholar 

  46. Vogt, P., Haasdijk, E.: Modeling social learning of language and skills. Artificial Life 16(4), 289–309 (2010)

    Article  Google Scholar 

  47. Waibel, M., Floreano, D., Magnenat, S., Keller, L.: Division of labour and colony efficiency in social insects: effects of interactions between genetic architecture, colony kin structure and rate of perturbations. In: Proceedings of the Royal Society B, vol. 273, pp. 1815–1823 (2006)

    Google Scholar 

  48. Watkins, C., Dayan, P.: Q-learning. Machine Learning 8(3), 279–292 (1992)

    MATH  Google Scholar 

  49. Werbos, P.: Backpropagation through time: what it does and how to do it. Proceedings of the IEEE 78(10), 1550–1560 (1990)

    Article  Google Scholar 

  50. Whiteson, S., Stone, P.: Evolutionary function approximation for reinforcement learning. The Journal of Machine Learning Research 7, 877–917 (2006)

    MathSciNet  MATH  Google Scholar 

  51. Yong, C., Miikkulainen, R.: Coevolution of role-based cooperation in multi-agent systems. IEEE Transactions on Autonomous Mental Development (2010)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, The University of Texas at Austin, Austin, TX, 78712, USA

    Risto Miikkulainen, Eliana Feasley, Leif Johnson, Igor Karpov, Padmini Rajagopalan, Aditya Rawal & Wesley Tansey

Authors
  1. Risto Miikkulainen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eliana Feasley
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Leif Johnson
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Igor Karpov
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Padmini Rajagopalan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Aditya Rawal
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Wesley Tansey
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Key Lab of Intelligent Perception and Image Understanding of Ministry of Education of China, Xidian University, 710071, Xi’an, Shaanxi, China

    Jing Liu

  2. Dipartimento di Elettronica e Informazione, Politecnico di Milano, 20133, Milano, Italy

    Cesare Alippi

  3. Université Pierre et Marie Curie - Paris 6, LIP6, 4 place Jussieu, 75005, Paris, France

    Bernadette Bouchon-Meunier

  4. Department of Electrical & Computer Engineering, Portland State University, 97207, Portland, OR, USA

    Garrison W. Greenwood

  5. School of Engineering and Information Technology, University of New South Wales, 2600, Canberra, ACT, Australia

    Hussein A. Abbass

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Miikkulainen, R. et al. (2012). Multiagent Learning through Neuroevolution. In: Liu, J., Alippi, C., Bouchon-Meunier, B., Greenwood, G.W., Abbass, H.A. (eds) Advances in Computational Intelligence. WCCI 2012. Lecture Notes in Computer Science, vol 7311. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-30687-7_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-30687-7_2

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-30686-0

  • Online ISBN: 978-3-642-30687-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation