Skip to main content

Advertisement

SpringerLink
Log in

A review of genetic algorithms applied to training radial basis function networks

  • Original Article
  • Published:
Neural Computing & Applications Aims and scope Submit manuscript

Abstract

The problems associated with training feedforward artificial neural networks (ANNs) such as the multilayer perceptron (MLP) network and radial basis function (RBF) network have been well documented. The solutions to these problems have inspired a considerable amount of research, one particular area being the application of evolutionary search algorithms such as the genetic algorithm (GA). To date, the vast majority of GA solutions have been aimed at the MLP network. This paper begins with a brief overview of feedforward ANNs and GAs followed by a review of the current state of research in applying evolutionary techniques to training RBF networks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1.

Similar content being viewed by others

References

  1. Park J, Sandberg IW (1991) Universal approximation using radial basis function networks. Neur Comput 3(2):246–257

    Google Scholar 

  2. Hornik K, Stinchcombe M, White H (1989) Multilayer feedforward networks are universal approximators. Neur Netw 2:359–366

    Article  Google Scholar 

  3. Niranjan M, Fallside F (1998) Neural networks and radial basis functions in classifying static speech patterns. Report CUED/FINFENG/TR22, University Engineering Department, Cambridge University, UK

  4. Minsky ML, Papert SA (1990) Perceptrons. MIT Press, Cambridge, MA

  5. Rumelhart DE, McClelland JL (eds) (1986) Parallel distributed processing: explorations in the microstructure of cognition, vol 1. MIT Press, Cambridge, MA

    Google Scholar 

  6. Broomhead DS, Lowe D (1988) Multivariable function interpolation and adaptive networks. Comp Sys 2:321–355

    MATH  Google Scholar 

  7. Moody JE, Darken CJ (1989) Fast learning in networks of locally-tuned processing units. Neur Comput 1(2):281–294

    Google Scholar 

  8. Powell MJD (1987) Radial basis functions for multivariable interpolation: a review. In: Mason JC, Cox MG (eds) Algorithms for approximation, Clarendon Press, Oxford, UK

  9. Batchelor BG, Wilkins BR (1969) Method for location of clusters of patterns to initialise a learning machine. Elect Lett 5:481–483

    Google Scholar 

  10. Kohonen T (1982) Self-organised formation of topologically correct feature maps. Biol Cybern 43:59–69

    MATH  Google Scholar 

  11. Holland J (1975) Adaptation in natural and artificial systems. University of Michigan Press, Ann Arbor, MI

  12. Goldberg DE (1989) Genetic algorithms in search, optimization and machine learning. Addison-Wesley, Reading, MA

  13. De Jong KA (1975) An analysis of the behaviour of a class of genetic adaptive systems. Dissertation, University of Michigan

  14. Schaffer JD, Caruana RA, Eshelman LJ, Das R (1989) A study of control parameters affecting online performance of genetic algorithms for function optimisation. In: Schaffer JD (ed) Proceedings of the 11th International Joint Conference on Artificial Intelligence, Morgan Kaufmann, San Mateo, CA

  15. Michalewicz Z (1996) Genetic algorithms + data structures = evolution programs, 3rd edition. Springer, Berlin Heidelberg New York

  16. Coley DA (1999) An introduction to genetic algorithms for engineers and scientists. World Scientific, New York

  17. Back T (1996) Evolutionary algorithms in theory and practice. Oxford University Press, New York

  18. Reed R (1993) Pruning algorithms—a survey. IEEE Trans Neur Netw 4:740–746

    Article  Google Scholar 

  19. Kwok TY, Yeung Y (1997) Constructive algorithms for structure learning in feedforward neural networks for regression problems. IEEE Trans Neur Netw 8(3):630–645

    Article  Google Scholar 

  20. Fahlman SE, Lebiere C (1990) The cascade-correlation learning architecture. In: Touretzky DS (ed) Advances in neural information processing systems, vol 2, Morgan Kaufmann, Los Altos, CA

  21. Chudova DI, Dolenko SA, Orlov YuV, Pavlov DYu, Persiantsev IG (1998) Benchmarking of different modifications of the cascade correlation algorithm. In: Parmee I (ed) Proceedings of the 3rd International Conference on Adaptive Computing in Design and Manufacture. Springer, Berlin Heidelberg New York

  22. Chen S, Billings SA, Cowan CFN, Grant PW (1990) Practical identification of narmax models using radial basis functions. Int J Contr 52(6):1327–1350

    MATH  Google Scholar 

  23. Akaike H (1973) Information theory and an extension of the maximum likelihood principle. In: Petrov BN, Csaki F (eds) Proceedings of the 2nd International Symposium on Information Theory, Tsahkadsov, Armenia

  24. Holcomb T, Morari M (1991) Local training for radial basis function networks: towards solving the hidden unit problem. In: Proceedings of the American Control Conference, Boston, MA, June 1991

  25. Lee S, Kil RM (1991) A Gaussian potential function network with hierarchically self-organising learning. Neur Netw 4:207–224

    Article  Google Scholar 

  26. Musavi MT, Ahmed W, Chan KH, Faris KB, Hummels DM (1992) On the training of radial basis function classifiers. Neur Netw 5:595–603

    Google Scholar 

  27. Sundararajan N, Saratchandran P, Lu Ying Wei (1999) Radial basis function neural networks with sequential learning: MRAN and its applications. World Scientific, New York

    Google Scholar 

  28. Kirkpatrick S, Gelatt CD, Vecchi MP (1983) Optimisation by simulated annealing. Science 220(4598):671–680

    MathSciNet  Google Scholar 

  29. Schaffer JD, Whitley D, Eschleman LJ (1992) Combinations of genetic algorithms and neural networks: a survey of the state of the art. In: Proceedings of the International Workshop on Combinations of Genetic Algorithms and neural Networks (COGANN-92), IEEE Computer Society Press, Los Alamitos, CA

  30. Yao X (1993) A review of evolutionary artificial neural networks. Int J Intellig Sys 8(4):539–567

    Google Scholar 

  31. Yao X (1999) Evolving artificial neural networks. Proc IEEE 87(9):1423–1447

    Article  Google Scholar 

  32. Balakrishnan K, Honavar V (1995) Evolutionary design of neural architecture—a preliminary taxonomy and guide to literature. Technical report, AI Research Group, CS-TR 95–01

  33. Whitley D (1995) Genetic algorithms and neural networks. In: Winter G, Periaux J, Galan M, Cuesta P (eds) Genetic algorithms in engineering and computer science, Wiley, New York

  34. Masters T (1993) Practical neural network recipes in C++. Academic Press, New York

  35. Rooiji AJF, Jain LC, Johnson RP (1997) Neural network training using genetic algorithms. Series in machine perception and artificial intelligence, vol. 26, World Scientific Publishing, New York

  36. Reeves CR, Taylor SJ (1998) Selection of training data for neural networks by a genetic algorithm. Lecture Notes in Computer Science, Springer, Berlin Heidelberg New York

  37. Hancock PJB (1992) Genetic algorithms and permutation problems: a comparison of recombination operators for neural net structure specification. In: Schaffer JD, Whitley D. and Eschleman LJ (eds) Proceedings of the International Workshop on Combinations of Genetic Algorithms and Neural Networks, (COGANN-92), IEEE Computer Society Press, Los Alamitos, CA

  38. Whitehead BA, Choate DC (1994) Evolving space-filling curves to distribute radial basis functions over an input space. IEEE Trans Neur Netw 5(1):15–23

    Article  MATH  Google Scholar 

  39. Goldberg DE (1991) Real coded genetic algorithms, virtual alphabets and blocking. Compl Sys 5:139–167

    MathSciNet  MATH  Google Scholar 

  40. Mackey MC, Glass L (1977) Oscillation and chaos in physiological control systems. Science 197:287–289

    Google Scholar 

  41. Whitehead BA, Choate DC (1996) Cooperative-competitive genetic evolution of radial basis function centres and widths for time series prediction. IEEE Trans Neur Netw 7:869–880

    Article  Google Scholar 

  42. Deb K, Goldberg DE (1989) An investigation of niche and species formation in genetic function optimization. In: Proceedings of the Third International Conference on Genetic Algorithms, Morgan Kaufmann, San Francisco

  43. Whitehead BA (1996) Genetic evolution of radial basis function coverage using orthogonal niches. IEEE Trans Neur Netw 7(6):1525–1528

    Article  Google Scholar 

  44. Billings SA, Zheng GL (1995) Radial basis function network configuration using genetic algorithms. Neur Netw 8(6):877–890

    Article  Google Scholar 

  45. Lucasius CB, Kateman G (1992) Towards solving subset selection problems with the aid of the genetic algorithm. In: Manner R, Manderick B (eds) Parallel problem solving from nature, vol 2, Elsevier, Amsterdam

  46. Neruda R (1995) Functional equivalence and genetic learning of RBF networks. In: Pearson DW, Steele NC, Albrecht RF (eds) Artificial neural networks and genetic algorithms, Springer, Berlin Heidelberg New York

  47. Hecht-Nielsen R (1990) On the algebraic structure of feed-forward network weight spaces. In: Advanced neural computers, Elsevier, Amsterdam

  48. Carse B, Pipe AG, Fogarty TC, Hill T (1995) Evolving radial basis function neural networks using a genetic algorithm. In: Proceedings of the 2nd IEEE International Conference on Evolutionary Computation, Perth, Australia, 29 November-1 December 1995

  49. Carse B, Fogarty TC, Munro A (1996) Evolving fuzzy rule based controllers using genetic algorithms. Int J Fuzz Set Sys 80(3):273–293

    Article  Google Scholar 

  50. Jang JR, Sun T (1993) Functional equivalence between radial basis function networks and fuzzy inference systems. IEEE Trans Neur Netw 4:156–159

    Article  Google Scholar 

  51. Carse B, Fogarty TC (1996) Tackling the “curse of dimensionality” of radial basis function networks using a genetic algorithm. Lecture Notes in Computer Science, Springer, Berlin Heidelberg New York

  52. Carse B, Fogarty TC (1996) Fast evolutionary learning of minimal radial basis function neural networks using a genetic algorithm. Lecture Notes in Computer Science, Springer, Berlin Heidelberg New York

  53. Burdsall B, Giraud-Carrier C (1997) GA-RBF: a self optimising RBF network. In: Proceedings of the Third International Conference on Artificial Neural Networks and Genetic Algorithms (ICANNGA’97), Springer, Berlin Heidelberg New York

  54. Burdsall B, Giraud-Carrier C (1997) Evolving fuzzy prototypes for efficient data clustering. In: Proceedings of the Second International ICSC Symposium on Fuzzy Logic and Applications (ISFL’97), ICSC Academic Press, New York

  55. Kuncheva LI (1997) Initializing of an RBF network by a genetic algorithm. Neurocomput 14(3):273–288

    Article  Google Scholar 

  56. Beasley D, Bull DR, Martin RR (1993) An overview of genetic algorithms: Part 1, fundamentals. Univer Comput 15:59–69

    Google Scholar 

  57. Beasley D, Bull DR, Martin RR (1993) An overview of genetic algorithms: Part 2, research topics. Univer Comput 15:170–181

    Google Scholar 

  58. Chaiyaratana N, Zalzala AMS (1998) Evolving hybrid RBF-MLP networks using combined genetic/unsupervised/supervised learning. In: Proceedings of the UKACC International Conference on CONTROL ‘98, Swansea, UK, September 1998

  59. Sergeev SA, Mahotilo KV, Voronovsky GK, Petrashev SN (1998) Genetic algorithm for training dynamical object emulator based on RBF neural network. Int J Appl Electro Mech 9(1): 65–74

    Google Scholar 

  60. Xue Y, Watton J (1998) Dynamics modelling of fluid power systems applying a global error descent algorithm to a self-organising radial basis function network. Mechatronics 8(7): 727–745

    Article  Google Scholar 

  61. Vesin JM, Gruter R (1999) Model selection using a simplex reproduction genetic algorithm. Sig Process (78):321–327

    MATH  Google Scholar 

  62. Rissanen J (1989) Stochastic complexity in statistical enquiry, World Scientific, Singapore

  63. Tong H (1990) Non-linear time series, Oxford University Press, Oxford, UK

  64. Chen S, Cowan CFN, Grant PM (1991) Orthogonal least squares learning algorithm for radial basis function networks. IEEE Trans Neur Netw 2(2):302–309

    Article  Google Scholar 

  65. Moechtar M, Farag AS, Hu L, Cheng TC (1999) Combined genetic algorithms and neural network approach for power-system transient stability evaluation. Europ Trans Elect Power 9(2):115–122

    Google Scholar 

  66. Dawson CW, Wilby RL, Harpham C, Brown MR, Cranston, E, Darby EJ (2000) Modelling Ranunculus presence in the Rivers test and Itchen using artificial neural networks. The Fifth International Conference on GeoComputation, Greenwich, UK, August 2000

  67. Sumathi S, Sivanandam SN, Ravindran R (2001) Design of a soft computing hybrid model classifier for data mining applications. Engin Intell Sys Electr Engin Comm 9(1):33–56

    Google Scholar 

  68. Leung H, Dubash N. Xie N (2002) Detection of small objects in clutter using a GA-RBF neural network. IEEE Trans Aero Electr Sys 38(1): 98–118

    Article  Google Scholar 

  69. Leung H, Lo T (1993) Chaotic radar signal processing over the sea. IEEE J Ocean Eng 18:287–295

    Google Scholar 

  70. Sheta AF, De Jong K (2001) Time-series forecasting using GA-tuned radial basis functions. Info Sci 133(3–4):221–228

    Google Scholar 

  71. Chen S, Wu Y, Luk BL (1999) Combined genetic algorithm optimization and regularized orthogonal least squares learning for radial basis function networks. IEEE Trans Neur Netw 10(5):1239–1243

    Article  MATH  Google Scholar 

  72. Chen S, Wu Y, Alkadhimi K (1995) A two-layer learning method for radial basis function networks using combined genetic and regularised OLS algorithms. In: Proceedings of the 1st IEE/IEEE International Conference on Genetic Algorithms in Engineering Systems: Innovations and Applications, Piscataway, NJ, September 1995

  73. Jiang N, Zhao ZY, Ren LQ (2003) Design of structural modular neural networks with genetic algorithm. Adv Eng Soft (1):17–24

    Article  Google Scholar 

  74. Liu Y, Yao X (1996) Evolutionary design of artificial neural networks with different nodes, evolutionary computation. Proc IEEE Int Conf 1996, 670–675

  75. Mor JJ (1977) The Levenberg-Marquardt algorithm: implementation and theory. In: Watson, GA (ed) Numerical analysis, Lecture Notes in Mathematics, Springer, Berlin Heidelberg New York

  76. Aiguo S, Jiren L (1998) Evolving Gaussian RBF network for nonlinear time series modelling and prediction. Electr Lett 34(12):1241–1243

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Geography, King’s College London, Strand, London, WC2R 2LS, UK

    C. Harpham

  2. Department of Computer Science, Loughborough University, Leicestershire, LE11 3TU, UK

    C. W. Dawson

  3. Department of Computing, University of Central Lancashire, Preston, Lancashire, PR1 2HE, UK

    M. R. Brown

Authors
  1. C. Harpham
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. C. W. Dawson
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. M. R. Brown
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to C. Harpham.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Harpham, C., Dawson, C.W. & Brown, M.R. A review of genetic algorithms applied to training radial basis function networks. Neural Comput & Applic 13, 193–201 (2004). https://doi.org/10.1007/s00521-004-0404-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-004-0404-5

Keywords

Search

Navigation