Skip to main content

Advertisement

SpringerLink
Log in

A new selective neural network ensemble with negative correlation

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

An ensemble of neural networks exhibits higher generalization performance compared to a single neural network. In this paper, a new design method for a neural network ensemble is proposed. The hierarchical pair competition-based parallel genetic algorithm (HFC-PGA) is employed to train the neural networks forming the ensemble. The aim of the HFC-PGA is to achieve not only the best neural network, but also a diversity of potential neural networks. A set of component neural networks is selected to build an ensemble such that the generalization error is minimized and the negative correlation is maximized. Finally, some experiments are carried out using several data sets to illustrate and quantify the performance of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Hassan YF (2011) Rough sets for adapting wavelet neural networks as a new classifier system. Appl Intell 35(2):260–268

    Article  Google Scholar 

  2. Bruzzone L, Prieto DF, Serpico SB (1999) A neural-statistical approach to multitemporal and multisource remote-sensing image classification. IEEE Trans Geosci Remote Sens 37:1350–1359

    Article  Google Scholar 

  3. Dong G, Xie M (2005) Color clustering and learning for image segmentation based on neural networks. IEEE Trans Neural Netw 16:925–936

    Article  Google Scholar 

  4. Tumar K, Ghosh J (1996) Error correlation and error reduction in ensemble classifiers. Connect Sci 8:385–404

    Article  Google Scholar 

  5. Optiz D, Maclin R (1999) Popular ensemble methods: an empirical study. J Artif Intell Res 11:169–198

    Google Scholar 

  6. Hansen L, Salamon P (1992) Neural network ensembles. IEEE Trans Pattern Anal Mach Intell 12:333–342

    Google Scholar 

  7. Huang F, Zhou Z, Zhang H, Chen T (2000) Pose invariant face recognition. In: Proc 4th IEEE int conf automatic face and gesture recognition, pp 245–250

    Chapter  Google Scholar 

  8. Hansen L, Liisberg L, Salamon P (1992) Ensemble methods for handwritten digit recognition. In: Proc IEEE workshop on neural networks for signal processing, pp 333–342

    Chapter  Google Scholar 

  9. Cherkauer K (1996) Human expert level performance on a scientific image analysis task by a system using combined artificial neural networks. In: Proc workshop on integrating multiple learned models for improving and scaling machine learning algorithms, pp 15–21

    Google Scholar 

  10. Zhou Z, Jiang Y, Yang Y, Chen S (2002) Lung cancer cell identification based on artificial neural network ensembles. Artif Intell Med 24, 25–36

    Article  MATH  Google Scholar 

  11. Lee H, Kim E, Park M (2007) A genetic feature weighting scheme for pattern recognition. Integr Comput-Aided Eng 14(2):161–171

    Google Scholar 

  12. Korkmaz EE (2010) Multi-objective genetic algorithms for grouping problems. Appl Intell 33(2):179–192

    Article  Google Scholar 

  13. Kim K-J, Cho S-B (2008) Evolutionary ensemble of diverse artificial neural networks using speciation. Neurocomputing 71(7–9):1604–1608

    Article  Google Scholar 

  14. Ishigami H, Fukuda T, Shibata T, Arai F (1995) Structure optimization of fuzzy neural network by genetic algorithm. Fuzzy Sets Syst 71:257–264

    Article  Google Scholar 

  15. Leung F, Lam H, Tam K (2003) Tuning of the structure and parameters of a neural network using an improved genetic algorithm. IEEE Trans Neural Netw 14(1):79–88

    Article  Google Scholar 

  16. Tsai J, Chou J, Liu T (2006) Tuning the structure and parameters of a neural network by using hybrid Taguchi-genetic algorithm. IEEE Trans Neural Netw 17(1):69–80

    Article  Google Scholar 

  17. Kang J-G, Kim S, An S-Y, Oh S-Y (2012) A new approach to simultaneous localization and map building with implicit model learning using neuro evolutionary optimization. Appl Intell 36(1):242–269

    Article  Google Scholar 

  18. Yao X (1999) Evolving artificial neural networks. Proc IEEE 87(9):1423–1447

    Article  Google Scholar 

  19. Optitz D, Shavlik J (1999) Generating accurate and diverse members of a neural-network ensemble. Adv Neural Inf Process Syst 8:535–541

    Google Scholar 

  20. Rogers A, Prugel-Bennett A (1999) Genetic drift in genetic algorithm selection schemes. IEEE Trans Evol Comput 3(4):298–303

    Article  Google Scholar 

  21. Choi J, Oh S, Pedrycz W (2008) Structural and parametric design of fuzzy inference systems using hierarchical fair competition-based parallel genetic algorithms and information granulation. Int J Approx Reason 49:631–648

    Article  Google Scholar 

  22. Michalewicz Z (1996) Genetic algorithms + data structures = evolution programs. Springer, Berlin

    MATH  Google Scholar 

  23. Wang H, Chen S, Zheng W (2008) Locality-preserved maximum information projection. IEEE Trans Neural Netw 19(4):571–585

    Article  Google Scholar 

  24. Hong J, Cho S (2006) Efficient huge-scale feature selection with speciated genetic algorithm. Pattern Recognit Lett 27:143–150

    Article  Google Scholar 

  25. Lee H, Hong S, Nizami IF, Kim E (2010) An efficient design of nearest neighbor classifier for various-scale problems. Pattern Recognit Lett 31(9):1020–1027

    Article  Google Scholar 

  26. Juo H, Chang H (2004) A new symbiotic evolution-based fuzzy-neural approach to fault diagnosis of marine propulsion systems. Eng Appl Artif Intell 17:919–930

    Article  Google Scholar 

  27. Lee H, Hong S, Kim E (2009) A new genetic feature selection with neural network ensemble. Int J Comput Math 86(7):1105–1117

    Article  MathSciNet  MATH  Google Scholar 

  28. Houck CR, Joines JA, Kay MG (1995) A genetic algorithm for function optimization: a Matlab implementation. NCSU-IE Technical Report 95-09, North Carolina State University

  29. Canuto A, Santos A, Vergas R (2011) Ensembles of ARTMAP-based neural networks: an experimental study. Appl Intell 35(1):1–17

    Article  Google Scholar 

  30. Hu JJ, Goodman ED (2002) The hierarchical fair competition (HFC) model for parallel evolutionary algorithms. In: Proc congress on evolutionary computation, pp 45–94

    Google Scholar 

  31. Choi J, Oh S, Pedrycz W (2008) Identification of fuzzy models using a successive tuning method with a variant identification ratio. Fuzzy Sets Syst 159:2873–2889

    Article  MathSciNet  MATH  Google Scholar 

  32. Liu Y, Yao X, Higuchi T (2000) Evolutionary ensembles with negative correlation learning. IEEE Trans Evol Comput 4(4):380–387

    Article  Google Scholar 

  33. Lee H, Hong S, Kim E (2008) An efficient gait recognition based on a selective neural network ensemble. Int J Imaging Syst Technol 18(4):237–241

    Article  Google Scholar 

  34. Kim K-J, Cho S-B (2007) Ensemble classifiers based on correlation analysis for DNA microarray classification. Appl Soft Comput 7(1):398–410

    Article  Google Scholar 

  35. Murphy PM, Aha DW (1994) UCI repository for machine learning databases. Technical report, Dept of Information and Computer Science, Univ of California, Irvine, Calif

  36. Lossos L et al (2000) Ongoing immunoglobulin somatic mutation in germinal center B cell-like but not in activated B cell-like diffuse large cell lymphomas. Proc Natl Acad Sci USA 97:10209–10213

    Article  Google Scholar 

  37. Tan H et al (2002) Prediction of central nervous system embryonal tumour outcome based on gene expression. Nature 415:436–442

    Article  Google Scholar 

  38. Hwang JP, Rou K, Park S, Kim E, Kang H (2006) PCA based vehicle detection for ACC. In: Proc 8th international conference on electronics, information, and communication, pp 98–101

    Google Scholar 

  39. Lee S-I, Ahn J-H, Cho S-B (2001) Exploiting diversity of neural ensembles with speciation evolution. In: Proc international joint conference on neural networks, vol 2, pp 808–813

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electrical and Electronic Engineering, Yonsei University, C613, Sinchon-dong, Seodaemun-gu, Seoul, 120-749, Korea

    Heesung Lee & Euntai Kim

  2. Department of Electrical and Computer Engineering, University of Alberta, Edmonton, Canada, T6G 2V4

    Witold Pedrycz

  3. Systems Research Institute, Polish Academy of Sciences, Warsaw, Poland

    Witold Pedrycz

Authors
  1. Heesung Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Euntai Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Witold Pedrycz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Euntai Kim.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lee, H., Kim, E. & Pedrycz, W. A new selective neural network ensemble with negative correlation. Appl Intell 37, 488–498 (2012). https://doi.org/10.1007/s10489-012-0342-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-012-0342-3

Keywords

Search

Navigation