Skip to main content
SpringerLink
Log in

Genetic set recombination and its application to neural network topology optimisation

  • Articles
  • Published:
Neural Computing & Applications Aims and scope Submit manuscript

Abstract

Forma analysis is applied to the task of optimising the connectivity of a feed- forward neural network with a single layer of hidden units. This problem is reformulated as a multiset optimisation problem, and techniques are developed to allow principled genetic search over fixed- and variable-size sets and multisets. These techniques require a further generalisation of the notion of gene, which is presented. The result is a non-redundant representation of the neural network topology optimisation problem, together with recombination operators which have carefully designed and well-understood properties. The techniques developed have relevance to the application of genetic algorithms to constrained optimisation problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Radcliffe NJ. Equivalence class analysis of genetic algorithms. Complex Syst 1991; 5(2): 183–205

    Google Scholar 

  2. Radcliffe NJ. Forma analysis and random respectful recombination. In: Proceedings of the Fourth International Conference on Genetic Algorithms. San Mateo (CA): Morgan Kaufmann, 1991: 222–229

    Google Scholar 

  3. Rudnick M. A bibliography of the intersection of genetic search and neural networks. Oregon (OR): Oregon Graduate Centre, Technical Report CS/E 90-001, 1990

    Google Scholar 

  4. Weiss G. Combining neural and evolutionary learning: Aspects and approaches. Technical Report FKI-132-90, Forschungsberichte Künstliche Intelligenz, 1990

  5. Rumelhart DE, Hinton GE, Williams RJ. Learning representations by back-propagating errors. Nature 1986: 323

  6. Miller GF, Todd PM, Hegde SU. Designing neural networks using genetic algorithms. In: Proceedings of the Third International Conference on Genetic Algorithms. San Mateo, Morgan Kaufmann, 1989

    Google Scholar 

  7. Harp SA, Samad T, Guha A. Towards the genetic synthesis of neural networks. In: Proceedings of the Third International Conference on Genetic Algorithms. San Mateo (CA): Morgan Kaufmann, 1989

    Google Scholar 

  8. Harp SA, Samad T, Guha A. The genetic synthesis of neural networks. Technical Report CSDD-89-I4852-2, Honeywell, 1989

  9. Whitley D, Starkweather T, Bogart C. Genetic algorithms and neural networks: Optimising connections and connectivity. Colorado (CO): Colorado State University, Technical Report CS-89-117, 1989

    Google Scholar 

  10. Mülenbein H, Kindermann J. The dynamics of evolution and learning — Towards genetic neural networks. In Pfiefer P, Schreter Z, Fogelman-Soulié F, Steels L, editors. Connectionism in perspective. Amsterdam: North-Holland, 1989

    Google Scholar 

  11. Hancock PJB. GANNET: Design of a neral net for face recognition by a genetic algorithm. Stirling (UK): University of Stirling, Technical report, 1990

    Google Scholar 

  12. Whitley D. Applying genetic algorithms to neural network problems: A preliminary report. (Unpublished manuscript)

  13. Whitley D, Hanson T. The Genitor algorithm: Using genetic algorithms to optimise neural networks. Colorado (CO): Colorado State University, Technical Report CS-89-107, 1989

    Google Scholar 

  14. Montana DJ, Davis L. Training feedforward neural networks using genetic algorithms. In: Proceedings of the Eleventh International Joint Conference on Artificial Intelligence, New York: AAAI, 1989: 762–767

    Google Scholar 

  15. Radcliffe NJ. Genetic neural networks on MIMD computers [dissertation]. Edinburgh (UK): Univ of Edinburgh, 1990

    Google Scholar 

  16. Belew RK, McInerny J, Schraudolph NN. Evolving networks: Using the genetic algorithm with connectionist learning. San Diego (CA): UCSD (La Jolla), Technical Report CS90-174, 1990

    Google Scholar 

  17. Radcliffe NJ. The permutation problem (unpublished manuscript), 1988

  18. Whitley D, Dominic S, Das R. Genetic reinforcement learning with multilayer neural networks. In: Proceedings of the Fourth International Conference on Genetic Algorithms. San Mateo (CA): Morgan Kaufmann, 1991

    Google Scholar 

  19. Holland JH. Adaptation in natural and artificial systems. Ann Arbor (MI): University of Michigan Press, 1975

    Google Scholar 

  20. Goldberg DE, Lingle Jr R. Alleles, loci and the traveling salesman problem. In: Proceedings of an International Conference on Genetic Algorithms. Hillsdale: Lawrence Erlbaum Associates, 1985

    Google Scholar 

  21. Goldberg DE. Genetic algorithms in search, optimization & machine learning. Reading (MA): Addison-Wesley, 1989

    Google Scholar 

  22. Vose MD. Generalizing the notion of schema in genetic algorithms. Artif. Intell. (in press)

  23. Vose MD, Liepins GE. Schema disruption. In: Proceedings of the Fourth International Conference on Genetic Algorithms. San Mateo (CA): Morgan Kaufmann, 1991: 237–243

    Google Scholar 

  24. Norman M. A genetic approach to topology optimisation for multiprocessor architectures. Edinburgh (UK): University of Edinburgh, Technical Report, 1988

    Google Scholar 

  25. Mühlenbein H. Parallel genetic algorithms, population genetics and combinatorial optimization. In: Proceedings of the Third International Conference on Genetic Algorithms. San Mateo (CA): Morgan Kaufmann, 1989

    Google Scholar 

  26. Seitsma J, Dow RJF. Neural net pruning — Why and how. In: Proceedings of the IEEE Conference on Neural Networks, vol II. New York: IEEE Press, 1988

    Google Scholar 

  27. Burkitt AN. Optimisation of the architecture of feed-forward neural networks with hidden layers by unit elimination. Complex Syst 1991; 5(4): 371–380

    Google Scholar 

  28. Whitley D. Using reproductive evaluation to improve genetic search and heuristic discovery. In: Proceedings of the Second International Conference on Genetic Algorithms. Hillsdale (NJ): Lawrence Erlbaum Associates, 1987

    Google Scholar 

  29. Goldberg DE. Genetic algorithms and Walsh functions: Part I, A gentle introduction. Complex Syst 1990; 3: 129–152

    Google Scholar 

  30. Syswerda G. Uniform crossover in genetic algorithms. In: Proceedings of the Third International Conference on Genetic Algorithms. San Mateo (CA): Morgan Kaufmann, 1989

    Google Scholar 

  31. Koza JR. Genetic programming: A paradigm for genetically breeding populations of computer programs to solve problems. Stanford (EA): Stanford University, Technical Report STAN-CS-90-1314, 1990

    Google Scholar 

  32. Koza JR. Evolving a computer to generate random numbers using the genetic programming paradigm. In: Proceedings of the Fourth International Conference on Genetic Algorithms, San Mateo (CA): Morgan Kaufmann, 1991, 37–44

    Google Scholar 

  33. Schaffer JD, Eshelman LJ. On crossover as an evolutionary viable strategy: In Proceedings of the Fourth International Conference on Genetic Algorithms, San Mateo (CA): Morgan Kaufmann, 1991: 61–68

    Google Scholar 

  34. Davis L. Bit-climbing, representational bias, and test suite design. In: Proceedings of the Fourth International Conference on Genetic Algorithms. San Mateo: Morgan Kaufmann, 1991

    Google Scholar 

  35. Whitley D, Starkweather T, Fuquay D. Scheduling problems and traveling salesmen: The genetic edge recombination operator In: Proceedings of the Third International Conference on Genetic Algorithms. San Mateo (CA): Morgan Kaufmann, 1989

    Google Scholar 

  36. Vose MD. Personal communication, 1991

  37. Whitley D, Starkweather T, Shaner D. The traveling salesman and sequence sheduling: Quality solutions using genetic edge recombination. In: Davis L, editor. Handbook of genetic algorithms. New York (NY): Van Nostrand Reinhold, 1991

    Google Scholar 

  38. Davis L. Handbook of genetic algorithms. New York: Van Nostrand Reinhold, 1991

    Google Scholar 

  39. Goldberg DE. Real-coded genetic algorithms, virtual alphabets, and blocking. Urbana-Champaign (IL): University of Illinois at Urbana-Champaign, Technical Report IlliGAL Report No. 90001, 1990

    Google Scholar 

  40. Booker L. Improving search in genetic algorithms. In: Davis L, editor. Genetic algorithms and simulated annealing. London: Pitman, 1987

    Google Scholar 

  41. Schaffer JD, Caruna RA, Eshelman LJ, Das R. A study of the control parameters affecting online performance of genetic algorithms for function optimisation. In: Proceedings of the Third International Conference on Genetic Algorithms. San Mateo (CA): Morgan Kaufmann, 1989

    Google Scholar 

  42. Spears WM, De Jong KA. On the virtues of parameterised uniform crossover. In: Proceedings of the Fourth International Conference on Genetic Algorithms. San Mateo (CA): Morgan Kaufmann, 1991: 230–236

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Edinburgh Parallel Computing Centre, University of Edinburgh, King's Buildings, EH9 3JZ, Edinburgh, UK

    Nicholas J. Radcliffe

Authors
  1. Nicholas J. Radcliffe
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Radcliffe, N.J. Genetic set recombination and its application to neural network topology optimisation. Neural Comput & Applic 1, 67–90 (1993). https://doi.org/10.1007/BF01411376

Download citation

  • Received:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF01411376

Keywords

Search

Navigation