Skip to main content
SpringerLink
Log in

Mutation operators for structure evolution of neural networks

  • Conference paper
  • First Online:
Parallel Problem Solving from Nature — PPSN III (PPSN 1994)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 866))

Included in the following conference series:

Abstract

The architecture of a neural network can be optimized by structure evolution. The structure evolution is based upon a two-stage evolution strategy (multipopulation strategy): On the population level, the structure is optimized, on the individual level, the parameters are adapted. For the variation of the (discrete) architecture, a mutation operator must be defined. To attain successful optimization, a mutation operator must satisfy two main conditions in the space of structures: First the principle of strong causality must be obeyed (smoothness of the fitness landscape in the space of structure), and second, a transition path between the structures must be guaranteed. In this paper different heuristic mutation operators will be defined and examined on their behavior with respect to strong causality and to neighborhood relation in the space of structures.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Angeline, P. J., Saunders, G. M., Pollack, J. B.: An Evolutionary Algorithm that Constructs Recurrent Neural Networks; IEEE Transactions on Neural Networks, vol. 5, no. 1, pp. 54–65; 1994

    Article  Google Scholar 

  • Born, J., Santibáñez-Koref, I., Voigt, H.-M.: Designing Neural networks by Adaptively Building Blocks in Cascades; Proc. of the 3. Conf. on Parallel Problem Solving from Nature; Jerusalem; 1994

    Google Scholar 

  • Fogel, D. B.: Using Evolutionary Programming to Create Neural Networks that are Capable of Playing Tic-Tac-Toe; IEEE Int. Conf. on Neural Networks, vol. 2, pp. 875–880; San Francisco, CA; 1993

    Article  Google Scholar 

  • Fogel, D. B.: An Introduction to Simulated Evolutionary Optimization; IEEE Transactions on Neural Networks, vol. 5, no. 1, pp. 3–14; 1994

    Article  Google Scholar 

  • Fogel, L. J., Owens, A. J., Walsh, M. J.: Artificial Intelligence Through Simulated Evolution; John Wiley & Sons; 1966

    Google Scholar 

  • Goldberg, D. E.: Genetic Algorithms in Search Optimization and Machine Learning; Addison Wesley; Reading, MA; 1989

    Google Scholar 

  • Gruau, F.: Genetic Synthesis of Modular Neural Networks; Proc. of the 5. Int. Conf. on Genetic Algorithms; Morgan Kaufmann Publishers; San Mateo, CA; 1993

    Google Scholar 

  • Gruau, F., Whitley, D.: Adding Learning to the Cellular Development of Neural Networks: Evolution and the Baldwin Effect; Evolutionary Computation, vol. 1, no. 3; The Massachusetts Institute of Technology; 1993

    Google Scholar 

  • Holland, J. H.: Adaptation in Natural and Artificial Systems; The University of Michigan Press; Ann Arbor, MI; 1975

    Google Scholar 

  • Jacob, C., Rehder, J.: Evolution of Neural Net Architectures by a hierarchical grammar-based Genetic System; Artificial Neural Nets and Genetic Algorithms; Proc. of the Int. Conf. in Innsbruck; Springer-Verlag; Wien; 1993

    Google Scholar 

  • Koza, J. R., Rice, J. P.: Genetic Generation of both the Weights and Architecture for a Neural Network; Proc. of the Int. Joint Conf. on Neural Networks, Seattle; IEEE Press vol. 2, pp. 47–56; 1991

    Google Scholar 

  • Kitano, H.: Designing neural networks using genetic algorithm with graph generating system; Complex Systems, vol. 4, pp. 461–476; 1992

    Google Scholar 

  • Lohmann, R.: Selforganisation by Evolution Strategy in Visual Systems; in: Voigt, Mühlenbein, Schwefel (ed.): Evolution and Optimization '89; pp. 61–68; Akademie Verlag; Berlin; 1990

    Google Scholar 

  • Lohmann, R.: Bionische Verfahren zur Entwicklung visueller Systeme; Master thesis, Technical University Berlin, Bionics and Evolution Techniques; 1991

    Google Scholar 

  • Lohmann, R.: Structure Evolution and Incomplete Induction; Proc. of the 2. Conf. on Parallel Problem Solving from Nature, pp. 175–185; Brussels; 1992

    Google Scholar 

  • Maniezzo, V.: Genetic Evolution of the Topology and Weight Distribution of Neural Networks; IEEE Transactions on Neural Networks, vol. 5, no. 1, pp. 39–53; 1994

    Article  Google Scholar 

  • McDonnell, J. R., Waagen, D.: Evolving Neural Network Connectivity; IEEE Int. Conf. on Neural Networks, vol. 2, pp. 863–868; San Francisco, California; 1993

    Article  Google Scholar 

  • McDonnell, J. R., Waagen, D.: Evolving Recurrent Perceptrons for Time-Series Modeling; IEEE Transactions on Neural Networks, vol. 5, no. 1, pp. 24–38; 1994

    Article  Google Scholar 

  • Miller, G., Todd, P., Hedge, S.: Designing Neural networks using genetic Algorithms; Proc. of the 3. Int. Conf. on Genetic Algorithms, pp. 379–384; 1989

    Google Scholar 

  • Rechenberg, I.: Evolutionsstrategie — Optimierung technischer Systeme nach Prinzipien der biologischen Evolution; Frommann-Holzboog; Stuttgart; 1973

    Google Scholar 

  • Rechenberg, I.: Evolutionsstrategie — Optimierung nach Prinzipien der biologischen Evolution; in: Albertz, J. (ed.): Evolution und Evolutionsstrategien in Biologie, Technik und Gesellschaft; Freie Akademie; Wiesbaden; 1989

    Google Scholar 

  • Rechenberg, I., Gawelczyk, A., Görne, T., Hansen, N., Herdy, M., Kost, B., Lohmann, R., Ostermeier, A., Trint, K., Utecht, U.: Evolutionsstrategische Strukturbildung in neuronalen Systemen; Statusseminar des BMFT Neuroinformatik, Maurach; ed: Projektträger Informationstechnik BMFT, DLR, pp. 25–34; 1992

    Google Scholar 

  • Rechenberg, I.: Evolutionsstrategie '94; Frommann-Holzboog; Stuttgart; 1994

    Google Scholar 

  • Rumelhart, D. E., McClelland, J. L.: Parallel Distributed Processing, Vol. 1 & 2; MIT-Press; Cambridge, MA; 1986

    Google Scholar 

  • Schwefel, H.-P.: Numerische Optimierung von Computer-Modellen mittels der Evolutionsstrategie; Birkhäuser; Basel, Stuttgart; 1977

    Google Scholar 

  • Trint, K., Utecht, U.: Methodik der Strukturevolution; Technical Report; Technical University Berlin, Bionics and Evolution Techniques; 1992

    Google Scholar 

  • Trint, K., Utecht, U.: Selection Mechanisms in Two-Stage Evolution Strategies for Noisy Evaluations; Technical Report TR-94-04; Technical University Berlin, Bionics and Evolution Techniques; 1994

    Google Scholar 

  • Whitley, D., Starkweather, T., Bogart, C.: Genetic algorithms and neural networks: optimizing connections and connectivity; Parallel Computing, vol. 14, pp. 347–361; 1990

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department Bionics and Evolution Techniques, Technical University Berlin, Ackerstr. 71 - 76 Sekr. ACK 1, D - 13355, Berlin

    Uwe Utecht & Karsten Trint

Authors
  1. Uwe Utecht
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Karsten Trint
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Yuval Davidor Hans-Paul Schwefel Reinhard Männer

Rights and permissions

Reprints and permissions

Copyright information

© 1994 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Utecht, U., Trint, K. (1994). Mutation operators for structure evolution of neural networks. In: Davidor, Y., Schwefel, HP., Männer, R. (eds) Parallel Problem Solving from Nature — PPSN III. PPSN 1994. Lecture Notes in Computer Science, vol 866. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-58484-6_292

Download citation

  • DOI: https://doi.org/10.1007/3-540-58484-6_292

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-58484-1

  • Online ISBN: 978-3-540-49001-2

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation