Abstract
This paper focuses on TWEANN (Topology and Weight Evolving Artificial Neural Network) methods based on indirect developmental encodings. TWEANNs are Evolutionary Algorithms (EAs) which evolve both topology and parameters (weights) of neural networks. Indirect developmental encoding is an approach inspired by multi-cellular organisms’ development from a single cell (zygote) known from Nature. The possible benefits of such encoding can be seen in Nature: for example, human genome consists of roughly 30 000 genes, which describe more than 20 billion neurons, each linked to as many as 10 000 others. In this work we examine properties of known tree-based indirect developmental encodings: Cellular Encoding and Edge Encoding. Well known Genetic Programming is usualy used to evolve tree structures. We have employed its successors: Gene Expression Programming (GEP) and Grammatical Evolution (GE) to optimize the trees. The combination of well designed developmental encoding and proper optimization method should bring compact genomes able to describe large-scale, modular neural networks. We have compared GE and GEP using a benchmark and found that GE was able to find solution about 7 times faster then GEP. On the other hand GEP solutions were more compact.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Michalewicz, Z.: Genetic algorithms + data structures = evolution programs, 2nd edn. Springer, New York (1994)
Tsoukalas, L.H., Uhrig, R.E.: Fuzzy and Neural Approaches in Engineering. John Wiley & Sons, Inc., New York (1996)
Kumar, S., Bentley, P.J. (eds.): On Growth, Form, and Computers. Elsevier Academic Press, London (2003)
Reisinger, J., Miikkulainen, R.: Acquiring evolvability through adaptive representations. In: GECCO 2007: Proceedings of the 9th annual conference on Genetic and evolutionary computation, pp. 1045–1052. ACM, New York (2007)
Mattiussi, C.: Evolutionary synthesis of analog networks. PhD thesis, Ecole polytechnique fédérale de Lausanne EPFL, Lausanne (2005)
Eggenberger-Hotz, P.: Creation of neural networks based on developmental and evolutionary principles. In: Gerstner, W., Hasler, M., Germond, A., Nicoud, J.-D. (eds.) ICANN 1997. LNCS, vol. 1327, pp. 337–342. Springer, Heidelberg (1997)
Eggenberger-Hotz, P., Gómez, G., Pfeifer, R.: Evolving the morphology of a neural network for controlling a foveating retina and its test on a real robot. In: Artificial Life VIII. The 8th International Conference on the Simulation and Synthesis of Living Systems (2003)
D’Ambrosio, D.B., Stanley, K.O.: A novel generative encoding for exploiting neural network sensor and output geometry. In: GECCO 2007: Proceedings of the 9th annual conference on Genetic and evolutionary computation, pp. 974–981. ACM, New York (2007)
Gauci, J., Stanley, K.: Generating large-scale neural networks through discovering geometric regularities. In: GECCO 2007: Proceedings of the 9th annual conference on Genetic and evolutionary computation, pp. 997–1004. ACM, New York (2007)
Stanley, K.O., Miikkulainen, R.: Evolving neural networks through augmenting topologies. Technical report, University of Texas at Austin, Austin, TX, USA (2001)
Gruau, F.: Neural Network Synthesis using Cellular Encoding and the Genetic Algorithm. PhD thesis, France (1994)
Koza, J.R.: Genetic programming: on the programming of computers by means of natural selection. MIT Press, Cambridge (1992)
Luke, S., Spector, L.: Evolving graphs and networks with edge encoding: Preliminary report. In: Koza, J.R. (ed.) Late Breaking Papers at the Genetic Programming 1996 Conference, July 1996, pp. 117–124. Stanford University, USA (1996)
Koza, J.R., Rice, J.P.: Genetic generation of both the weights and architecture for a neural network. In: International Joint Conference on Neural Networks, IJCNN 1991, Washington State Convention and Trade Center, Seattle, WA, USA, 8-12, 1991, vol. II, pp. 397–404. IEEE Computer Society Press, Los Alamitos (1991)
Ferreira, C.: Gene Expression Programming: Mathematical Modeling by an Artificial Intelligence (Studies in Computational Intelligence), 2nd edn. Springer, New York (2006)
Ryan, C., Collins, J.J., O’Neill, M.: Grammatical evolution: Evolving programs for an arbitrary language. In: EuroGP, pp. 83–96 (1998)
Gruau, F., Whitley, D., Pyeatt, L.: A comparison between cellular encoding and direct encoding for genetic neural networks. In: Koza, J.R., Goldberg, D.E., Fogel, D.B., Riolo, R.L. (eds.) Proceedings of the First Annual Conference on Genetic Programming 1996, Stanford University, CA, USA, July 1996, pp. 81–89. MIT Press, Cambridge (1996)
Hornby, G.S.: Shortcomings with using edge encodings to represent graph structures. Genetic Programming and Evolvable Machines 7(3), 231–252 (2006)
Stanley, K.O., Miikkulainen, R.: Competitive coevolution through evolutionary complexification. Journal of Artificial Intelligence Research (21), 63–100 (2004)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Drchal, J., Šnorek, M. (2008). Tree-Based Indirect Encodings for Evolutionary Development of Neural Networks. In: Kůrková, V., Neruda, R., Koutník, J. (eds) Artificial Neural Networks - ICANN 2008. ICANN 2008. Lecture Notes in Computer Science, vol 5164. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-87559-8_87
Download citation
DOI: https://doi.org/10.1007/978-3-540-87559-8_87
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-87558-1
Online ISBN: 978-3-540-87559-8
eBook Packages: Computer ScienceComputer Science (R0)