Abstract
This paper explains the optimisation of neuralnetwork topology using Incremental Evolution;that is, by allowing the network to expand byadding to its structure. This method allows anetwork to grow from a simple to a complexstructure until it is capable of fulfilling itsintended function. The approach is somewhatanalogous to the growth of an embryo or theevolution of a fossil line through time, it istherefore sometimes referred to as anembryology or embryological algorithm. Thepaper begins with a general introduction,comparing this method to other competingtechniques such as The Genetic Algorithm, otherEvolutionary Algorithms and SimulatedAnnealing. A literature survey of previous workis included, followed by an extensive newframework for application of the technique.Finally, examples of applications and a generaldiscussion are presented.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Alpaydin, E. (1994). GAL: Networks that Grow When They Learn and Shrink When They Forget. International Journal of Pattern Recognition 8(1): 391-414.
Alpaydin, R., Unluakin, U., Gurgen, F. & Alpaydin, E. (1993). Comparing Distributed and Local Neural Classifiers for the Recognition of Japanese Phonemes. '93), 239-242.
Anderle, M., Schweng, H., Kurten, K. E. & Kratky, K. W. (1995). Pattern Specific Neural Network Design. Journal of statistical physics 81(3/4): 843-849.
Ash, T. (1989). Dynamic Node Creation in Backpropagation Networks. Connection Science 1: 365-375.
Baum, E. B. (1989). A proposal forMore Powerful Learning Algorithms. Neural Computation 1: 295-311.
Brouwer, R. F. (1995). Automatic Growing of a Hopfield Style Neural Network for Classification of Patterns. Image processing and its applications, IEE, 637-641.
Carpenter, G. A. & Grossberg, S. (1986). Neural Dynamics of Category Learning and Recognition. Brain Structure, Learning and Memory (AAA Symposium).
Carpenter, G. A. & Grossberg, S. (1987). A Massively Parallel structure for a Self Organised Neural Pattern Recognition Machine. Computer vision, graphics and image processing 37: 54-115.
Carpenter, G. A. & Grossberg, S. (1987). ART-2: Self Organisation of Stable Category Recognition Codes for Analog Input Patterns. Applied optics 26: 4919-4930.
Chakraborty, G. (1995). A Growing Network which Optimises between Undertraining and Overtraining. IEEE Conference on Neural Networks 2: 1116-1120.
Demuth, H. & Beale, M. (1994). Neural Network TOOLBOX, The math works inc. 11.32-11.39.
Fang, J. & Xi, Y. (1997). Neural Network Design Based on Evolutionary Programming. Artificial Intelligence in engineering 11: 155-161.
Ferran, E. & Perazzo, R. (1991). Asymptotic Inferential Capabilities of Feed-forward Neural Networks. Europhysics letters 14(2): 175-180.
Kozma, R. & Kitamura, M. (1995). Dynamic Structure Adaptation in Feedforward Neural Networks-An Example of Plant Monitoring. IEEE conference on Neural Networks 2: 622-627.
Lin, C. T. (1994). FALCON: A Fuzzy Adaptive Learning Control Network. NAFIPS / IFIS / NASA 94, 228-232.
MacLeod, C. (1999). The Synthesis of Artificial Neural Networks using Single String Evolutionary Techniques. PhD Thesis, The Robert Gordon University, Aberdeen UK.
McDonnall, J. R. (1994). Neural Network Construction Using Evolutionary Search. Conference On Evolutionary Programming, 9-16.
McMinn, D. (2001). The use of Evolutionary Artificial Neural Networks to Synthesise Artifi-cial Nervous Systems for Animals. Ph.D. thesis, The Robert Gordon University, Aberdeen, UK.
Palmer-Brown, D. (1992). High Speed Learning in a Supervised, Self-growing Net. Artificial Neural Networks 2: 1159-1162.
Reignier, P., Hansen, V. & Crowley, S. (1995). Incremental Supervised Learning for Mobile Robot Reactive Control. Intelligence autonomous systems international conference, 287-294. IOS Press.
Rudolph, G. (1997). The Handbook of Evolutionary Computation, B1.3:1-B1.3:6. Oxford: IOP Press.
Schaffer, J. D., Whitley, D. & Eshelman, L. J. (1992). Combinations of Genetic Algorithms and Neural Networks: A Survey of the State of the Art. COGANN-92, IEEE, 1-37.
Simpson, P. K. (1990). Artificial Neural Networks: Foundations, Paradigms, Applications and Implementations, 36-46. Pergamon Press.
Sriram, M. & Kang, S. M. (1990). A Modified Hopfield Network for Two-dimensionalModule Placement. IEEE international symposium on circuits and systems 3: 1664-1667.
Thompson, A. (1996). Silicon Evolution. Genetic Programming: 1996 roceedings of the First International Conference, 444-452.
Vinod, V. V. & Ghoso, S. (1996). Growing Non-uniform Feed-forward Networks for Continuous Mappings. Neurocomputing 10: 55-69.
Wasserman, P. D. (1989). Neural Computing: Theory and Practice, 127. New York: Van Nostrand Reinhold.
Yip, P. P. C. & Pao, Y. (1994). Growing Neural Networks Using Guided Evolutionary Simulated Annealing. Conference On Evolutionary Programming, 17-20.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
MacLeod, C., Maxwell, G.M. Incremental Evolution in ANNs: Neural Nets which Grow. Artificial Intelligence Review 16, 201–224 (2001). https://doi.org/10.1023/A:1011951731821
Issue Date:
DOI: https://doi.org/10.1023/A:1011951731821