David B. D'Ambrosio
ABSTRACT
This paper argues that multiagent learning is a potential "killer application" for generative and developmental systems (GDS) because key challenges in learning to coordinate a team of agents are naturally addressed through indirect encodings and information reuse. For example, a significant problem for multiagent learning is that policies learned separately for different agent roles may nevertheless need to share a basic skill set, forcing the learning algorithm to reinvent the wheel for each agent. GDS is a good match for this kind of problem because it specializes in ways to encode patterns of related yet varying motifs. In this paper, to establish the promise of this capability, the Hypercube-based NeuroEvolution of Augmenting Topologies (HyperNEAT) generative approach to evolving neurocontrollers learns a set of coordinated policies encoded by a single genome representing a team of predator agents that work together to capture prey. Experimental results show that it is not only possible, but beneficial to encode a heterogeneous team of agents with an indirect encoding. The main contribution is thus to open up a significant new application domain for GDS.
- P. J. Bentley and S. Kumar. The ways to grow designs: A comparison of embryogenies for an evolutionary design problem. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-1999), pages 35--43, 1999.Google Scholar
Digital Library
- J. Bongard. Reducing Collective Behavioural Complexity through Heterogeneity. Arti.cial Life VII: Proceedings of the Seventh International Conference on Artificial Life, 2000.Google Scholar
- J. Bongard. Reducing Collective Behavioural Complexity through Heterogeneity. Arti.cial Life VII: Proceedings of the Seventh International Conference on Artificial Life, 2000.Google Scholar
- G. Cybenko. Approximation by superpositions of a sigmoidal function. Mathematics of Control, Signals, and Systems, 2(4):303--314, 1989.Google Scholar
- D. B. D'Ambrosio and K. O. Stanley. A novel generative encoding for exploiting neural network sensor and output geometry. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2007), New York, NY, 2007. ACM Press. Google ScholarDigital Library
- T. N. Dupuy. The Evolution of Weapons and Warfare. Da Capo, New York, NY, USA, 1990.Google Scholar
- P. Eggenberger. Evolving Morphologies of Simulated 3d Organisms Based on Differential Gene Expression. Fourth European Conference on Artificial Life, 1997.Google Scholar
- J. Gauci and K. O. Stanley. Generating large-scale neural networks through discovering geometric regularities. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2007), New York, NY, 2007. ACM Press. Google ScholarDigital Library
- C. Green. SharpNEAT homepage. http://sharpneat.sourceforge.net/, 2003--2008.Google Scholar
- F. Gruau, D. Whitley, and L. Pyeatt. A comparison between cellular encoding and direct encoding for genetic neural networks. In J. R. Koza, D. E. Goldberg, D. B. Fogel, and R. L. Riolo, editors, Genetic Programming 1996: Proceedings of the First Annual Conference, pages 81--89. MIT Press, 1996. Google ScholarDigital Library
- F. Gruau, D. Whitley, and L. Pyeatt. A comparison between cellular encoding and direct encoding for genetic neural networks. In J. R. Koza, D. E. Goldberg, D. B. Fogel, and R. L. Riolo, editors, Genetic Programming 1996: Proceedings of the First Annual Conference, pages 81--89. MIT Press, 1996. Google ScholarDigital Library
- A. Lindenmayer. Adding continuous components to L-systems. In G. Rozenberg and A. Salomaa, editors, L Systems, Lecture Notes in Computer Science 15, pages 53--68. Springer-Verlag, Heidelberg, Germany, 1974. Google ScholarDigital Library
- S. Luke and L. Spector. Evolving graphs and networks with edge encoding: Preliminary report. In J. R. Koza, editor, Late-Breaking Papers of Genetic Programming 1996. Stanford Bookstore, 1996.Google Scholar
- A. P. Martin. Increasing genomic complexity by gene duplication and the origin of vertebrates. The American Naturalist, 154(2):111--128, 1999.Google ScholarCross Ref
- J. F. Miller. Evolving a self-repairing, self-regulating, French flag organism. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2004), Berlin, 2004. Springer Verlag.Google ScholarCross Ref
- L. Panait and S. Luke. Cooperative multi-agent learning: The state of the art. Autonomous Agents and Multi-Agent Systems, 3(11):383--434, November 2005. Google ScholarDigital Library
- L. Panait, R. Wiegand, and S. Luke. Improving coevolutionary search for optimal multiagent behaviors. Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence (IJCAI), pages 653--658, 2003. Google ScholarDigital Library
- M. A. Potter, K. A. De Jong, and J. J. Grefenstette. A coevolutionary approach to learning sequential decision rules. In L. J. Eshelman, editor, Proceedings of the Sixth International Conference on Genetic Algorithms, 1995. Google ScholarDigital Library
- K. Sims. Evolving 3D morphology and behavior by competition. pages 28--39. MIT Press, Cambridge, MA, 1994.Google Scholar
- K. O. Stanley. Compositional pattern producing networks: A novel abstraction of development. Genetic Programming and Evolvable Machines Special Issue on Developmental Systems, 8(2):131--162, 2007. Google ScholarDigital Library
- K. O. Stanley. Compositional pattern producing networks: A novel abstraction of development. Genetic Programming and Evolvable Machines Special Issue on Developmental Systems, 8(2):131--162, 2007. Google ScholarDigital Library
- K. O. Stanley, D. B. D'Ambrosio, and J. Gauci. A hypercube-based indirect encoding for evolving large-scale neural networks. Artificial Life, 2008. To appear. Google ScholarDigital Library
- K. O. Stanley, D. B. D'Ambrosio, and J. Gauci. A hypercube-based indirect encoding for evolving large-scale neural networks. Artificial Life, 2008. To appear.Google Scholar
- K. O. Stanley and R. Miikkulainen. A taxonomy for artificial embryogeny. Artificial Life, 9(2):93--130, 2003. Google ScholarDigital Library
- K. O. Stanley and R. Miikkulainen. Competitive coevolution through evolutionary complexification. Journal of Artifical Intelligence Research, 21:63--100, 2004. Google ScholarDigital Library
- P. Stone and M. Veloso. Multiagent Systems: A Survey from a Machine Learning Perspective. Autonomous Robots, 8(3):345--383, 2000. Google ScholarDigital Library
- R. P. Wiegand. An analysis of cooperative coevolutionary algorithms. PhD thesis, George Mason University, Fairfax, VA, USA, 2004. Director-Kenneth A. Jong. Google ScholarDigital Library
- C. Yong and R. Miikkulainen. Coevolution of role-based cooperation in multi-agent systems. Technical Report AI07-338, The University of Texas at Austin Department of Computer Sciences, 2007.Google Scholar
Index Terms
- Generative encoding for multiagent learning
Recommendations
Evolving the placement and density of neurons in the hyperneat substrate
GECCO '10: Proceedings of the 12th annual conference on Genetic and evolutionary computationThe Hypercube-based NeuroEvolution of Augmenting Topologies (HyperNEAT) approach demonstrated that the pattern of weights across the connectivity of an artificial neural network (ANN) can be generated as a function of its geometry, thereby allowing ...
Evolving policy geometry for scalable multiagent learning
AAMAS '10: Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems: volume 1 - Volume 1A major challenge for traditional approaches to multiagent learning is to train teams that easily scale to include additional agents. The problem is that such approaches typically encode each agent's policy separately. Such separation means that ...
A hypercube-based encoding for evolving large-scale neural networks
Research in neuroevolution---that is, evolving artificial neural networks (ANNs) through evolutionary algorithms---is inspired by the evolution of biological brains, which can contain trillions of connections. Yet while neuroevolution has produced ...
Comments