Skip to main content
SpringerLink
Log in

A Group Search Optimizer for Neural Network Training

  • Conference paper
Computational Science and Its Applications - ICCSA 2006 (ICCSA 2006)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 3982))

Included in the following conference series:

Abstract

A novel optimization algorithm: Group Search Optimizer (GSO) [1] has been successfully developed, which is inspired by animal behavioural ecology. The algorithm is based on a Producer-Scrounger model of animal behaviour, which assumes group members search either for ‘finding’ (producer) or for ‘joining’ (scrounger) opportunities. Animal scanning mechanisms (e.g., vision) are incorporated to develop the algorithm. In this paper, we apply the GSO to Artificial Neural Network (ANN) training to further investigate its applicability to real-world problems. The parameters of a 3-layer feed-forward ANN, including connection weights and bias are tuned by the GSO algorithm. Two real-world classification problems have been employed as benchmark problems trained by the ANN, to assess the performance of the GSO-trained ANN (GSOANN). In comparison with other sophisticated machine learning techniques proposed for ANN training in recent years, including some ANN ensembles, GSOANN has a better convergence and generalization performances on the two benchmark problems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 139.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. He, S., Wu, Q.H., Saunders, J.R.: Group search optimizer - an optimization algorithm inspired by animal behavioral ecology (Subimtted to IEEE Trans. on Evolutionary Computation)

    Google Scholar 

  2. Thrun, S.B., et al.: The MONK’s problems: A performance comparison of different learning algorithms. Technical Report CS-91-197, Pittsburgh, PA (1991)

    Google Scholar 

  3. Wu, Q.H., Hogg, B.W., Irwin, G.W.: A neural network regulator for turbogenerators. IEEE Trans. on Neural Networks 3(1), 95–100 (1992)

    Article  Google Scholar 

  4. Yao, X.: Evolving artificial neural networks. Proceeding of the IEEE 87(9), 1423–1447 (1999)

    Article  Google Scholar 

  5. Fogel, D.B., Fogel, L.J., Porto, V.W.: Evolving neural networks. Biol. Cybern. 63, 487–493 (1990)

    Article  Google Scholar 

  6. Yao, X., Liu, Y.: A new evolutionary system for evolving artificial neural networks. IEEE Trans. on Neural Networks 8(3), 694–713 (1997)

    Article  MathSciNet  Google Scholar 

  7. Leung, F.H.F., Lam, H.K., Ling, S.H., Tam, P.K.S.: Tuning of the structure and parameters of a neural network using an improved genetic algorithm. IEEE Trans. on Neural Networks 14(1), 79–88 (2003)

    Article  Google Scholar 

  8. Palmes, P.P., Hayasaka, T., Usui, S.: Mutation-based genetic neural network. IEEE Trans. on Neural Networks 16(3), 587–600 (2005)

    Article  Google Scholar 

  9. Cantu-Paz, E., Kamath, C.: An empirical comparison of combinations of evolutionary algorithms and neural networks for classification problems. IEEE Transactions on Systems, Man, and Cybernetics-Part B: Cybernetics 35(5), 915–927 (2005)

    Article  Google Scholar 

  10. Wolpert, D.H.: A mathematical theory of generalization. Complex Systems 4(2), 151–249 (1990)

    MATH  MathSciNet  Google Scholar 

  11. Barnard, C.J., Sibly, R.M.: Producers and scroungers: a general model and its application to captive flocks of house sparrows. Animal Behaviour 29, 543–550 (1981)

    Article  Google Scholar 

  12. Couzin, I., Krause, J., Franks, N., Levin, S.: Effective leadership and decision-making in animal groups on the move. Nature 434, 513–516 (2005)

    Article  Google Scholar 

  13. Bell, J.W.: Searching Behaviour - The Behavioural Ecology of Finding Resources. Chapman and Hall Animal Behaviour Series. Chapman and Hall, Boca Raton (1990)

    Google Scholar 

  14. O’Brien, W.J., Evans, B.I., Howick, G.L.: A new view of the predation cycle of a planktivorous fish, white crappie (pomoxis annularis). Can. J. Fish. Aquat. Sci. 43, 1894–1899 (1986)

    Article  Google Scholar 

  15. Harper, D.G.C.: Competitive foraging in mallards: ideal free ducks. Animal Behaviour 30, 575–584 (1988)

    Article  Google Scholar 

  16. Dusenbery, D.B.: Ranging strategies. Journal of Theoretical Biology 136, 309–316 (1989)

    Article  MathSciNet  Google Scholar 

  17. Higgins, C.L., Strauss, R.E.: Discrimination and classfication of foraging paths produced by search-tactic models. Behavioral Ecology 15(2), 248–254 (2003)

    Article  Google Scholar 

  18. Viswanathan, G.M., Buldyrev, S.V., Havlin, S., da Luz, M.G., Raposo, E., Stanley, H.E.: Optimizing the success of random searches. Nature 401, 911–914 (1999)

    Article  Google Scholar 

  19. Dixon, A.F.G.: An experimental study of the searching behaviour of the predatory coccinellid beetle adalia decempunctata. J. Anim. Ecol. 28, 259–281 (1959)

    Article  Google Scholar 

  20. Haykin, S.: Neural Networks. A Comprehensive Foundation. Prentice Hall, New Jersey (1999)

    Google Scholar 

  21. Prechelt, L.: Problem1 - a set of neural network benchmark problems and benchmarking rules. Technical report, Fakultat fur Infromatik Universitat Karlsruhe, 76128 Karlsruhe, Germany (1995)

    Google Scholar 

  22. Garcia-Pedrajas, N., Hervas-Martinez, C., Ortiz-Boyer, D.: Cooperative coevolution of artificial neural network ensembles for pattern classification. IEEE Trans. on Evolutionary Computation 9(3), 271–302 (2005)

    Article  Google Scholar 

  23. Islam, M., Yao, X., Murase, K.: A constructive algorithm for training cooperative neural network ensembles. IEEE Trans. on Neural Networks 14(4), 820–834 (2003)

    Article  Google Scholar 

  24. Dzeroski, S., Zenko, B.: Is combining classifiers with stacking better than selecting the best one? Machine Learning 54(3), 255–273 (2004)

    Article  MATH  Google Scholar 

  25. Cantu-Paz, E., Kamath, C.: Inducing oblique decision trees with evolutionary algorithms. IEEE Trans. on Evolutionary Computation 7(1), 54–68 (2003)

    Article  Google Scholar 

  26. Dirtterich, T.G.: An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting, and randomization. Machine Learning 40(12), 139–157 (2000)

    Article  Google Scholar 

  27. Deb, K., Anand, A., Joshi, D.: A computationally efficient evolutionary algorithm for real-parameter optimization. Evolutionary Computation 10(4), 371–395 (2002)

    Article  Google Scholar 

  28. Gestel, T.V., et al.: Benchmarking least squares support vector machine classifiers. Machine Learning 54(1), 5–32 (2004)

    Article  MATH  Google Scholar 

  29. Garcia-Pedrajas, N., Hervas-Martinez, C., Munoz-Perez, J.: Covnet: a cooperative coevolutionary model for evolving artificial neural networks. IEEE Trans. on Neural Networks 14(3), 575–596 (2003)

    Article  Google Scholar 

  30. Liu, Y., Yao, X.: Evolutionary ensembles with negative correlation learning. IEEE Trans. on Evolutionary Computation 4(4), 380–387 (2000)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical Engineering and Electronics,  

    S. He & Q. H. Wu

  2. School of Biological Sciences, The University of Liverpool, Liverpool, L69 3GJ, UK

    J. R. Saunders

Authors
  1. S. He
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Q. H. Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. J. R. Saunders
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Computer Science, University of Calgary, 2500 University Drive N.W., T2N 1N4, Calgary, AB, Canada

    Marina Gavrilova

  2. Department of Mathematics and Computer Science, University of Perugia, via Vanvitelli, 1, I-06123, Perugia, Italy

    Osvaldo Gervasi

  3. William Norris Professor, Head of the Computer Science and Engineering Department, University of Minnesota, USA

    Vipin Kumar

  4. OptimaNumerics Ltd., Cathedral House, 23-31 Waring Street, BT1 2DX, Belfast, UK

    C. J. Kenneth Tan

  5. Clayton School of IT, Monash University, 3800, Clayton, Australia

    David Taniar

  6. Department of Chemistry, University of Perugia, Via Elce di Sotto, 8, I-06123, Perugia, Italy

    Antonio Laganá

  7. School of Computing, Soongsil University, Seoul, Korea

    Youngsong Mun

  8. School of Information and Communication Engineering, Sungkyunkwan University, Korea

    Hyunseung Choo

Rights and permissions

Reprints and permissions

Copyright information

© 2006 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

He, S., Wu, Q.H., Saunders, J.R. (2006). A Group Search Optimizer for Neural Network Training. In: Gavrilova, M., et al. Computational Science and Its Applications - ICCSA 2006. ICCSA 2006. Lecture Notes in Computer Science, vol 3982. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11751595_98

Download citation

  • DOI: https://doi.org/10.1007/11751595_98

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-34075-1

  • Online ISBN: 978-3-540-34076-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation