Skip to main content
SpringerLink
Log in

ARTIE and MUSCLE models: building ensemble classifiers from fuzzy ART and SOM networks

  • Cont. Dev. of Neural Compt. & Appln.
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Ensemble Learning has proven to be an efficient method to improve the performance of single classifiers. In this context, the present article introduces ARTIE (ART networks in Ensembles) and MUSCLE (Multiple SOM Classifiers in Ensembles), two novel ensemble models that use Fuzzy ART and SOM networks as base classifiers, respectively. In addition, a hybrid metaheuristic solution based on Particle Swarm Optimization and Simulated Annealing is used for parameter tuning of the base classifiers. A comprehensive performance comparison using 10 benchmarking data sets indicates that the ARTIE and MUSCLE architectures consistently outperform ensembles built from standard supervised neural networks, such as the Fuzzy ARTMAP, Learning Vector Quantization, and the Extreme Learning Machine.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. Bagging is carried out by sampling (with replacement) training examples, forming new training sets, usually with the same size of the original one. For a training set of N samples and N being large enough, this procedure causes each sample to have a probability of \(\left(\frac{N-1}{N}\right)^N \approx 0.368\) of not being chosen.

References

  1. Barreto GA, Aguayo L (2009) Time series clustering for anomaly detection using competitive neural networks. In: Principe JC, Miikkulainen R (eds) Proceedings of the 7th international workshop on advances in self-organizing maps (WSOM’09), Springer, vol LNCS-5629, pp 28–36

  2. Baruque B, Corchado E (2010) A weighted voting summarization of SOM ensembles. Data Min Knowl Discov 21(3):398–426

    Article  MathSciNet  Google Scholar 

  3. Bermejo S, Cabestany J (2004) Local averaging of ensembles of LVQ-based nearest neighbor classifiers. Appl Intell 20(1):47–58

    Article  MATH  Google Scholar 

  4. Biebelmann E, Köppen M, Nickolay B (1996) Pratical aplications of neural networks in texture analysis. Neurocomputing 13(2–4):261–279

    Article  Google Scholar 

  5. Biehl M, Ghosh A, Hammer B (2007) Dynamics and generalization ability of LVQ algorithms. J Mach Learn Res 8(Feb):323–360

    MathSciNet  MATH  Google Scholar 

  6. Bratton D, Kennedy J (2007) Defining a standard for particle swarm optimization. In: Proceedings of the IEEE Swarm Intelligence Symposium, Honolulu, Hawaii, pp 120–127

  7. Breiman L (1996) Bagging predictors. Mach Learn 24(2):123–140

    MathSciNet  MATH  Google Scholar 

  8. Carpenter G (2003) Default ARTMAP. CAS/CNS Technical Report Series (008)

  9. Carpenter G, Grossberg S (1987) Stable self-organization of pattern recognition codes for analog input patterns. Appl Opt 26:4919–4930

    Article  Google Scholar 

  10. Carpenter G, Grossberg S, Reynolds JH (1991) ARTMAP: supervised real-time learning and classification of nonstationary data by a self-organizing neural network. Neural Netws 4(5):565–588

    Article  Google Scholar 

  11. Carpenter GA, Grossberg S (1987) A massively parallel architecture for a self-organizing neural pattern recognition machine. Comput Vis Graph Image Process 37(1):54–115

    Article  MATH  Google Scholar 

  12. Carpenter GA, Grossberg S, Rosen DB (1991) Fuzzy ART: fast stable learning, categorization of analog patterns by an adaptive resonance system. Neural Netw 4(6):759–771

    Article  Google Scholar 

  13. Carpenter GA, Grossberg S, Markuzon N, Reynolds JH, Rosen DB (2002) Fuzzy ARTMAP: a neural network architecture for incremental supervised learning of analog multidimensional maps. IEEE Trans Neural Netw 3(5):698–713

    Article  Google Scholar 

  14. Chang Y, Lee DJ, Hong Y, Archibald J (2008) Unsupervised video shot detection using clustering ensemble with a color global scale-invariant feature transform descriptor. EURASIP J Image Video Process 2008(Article ID 860743):10 pages. doi:10.1155/2008/860743

  15. Cho SB (1997) Self-organizing map with dynamical node splitting: application to handwritten digit recognition. Neural Comput 9(6):1345–1355

    Article  Google Scholar 

  16. Christodoulou CI, Michaelides SC, Pattichis CS (2003) Multifeature texture analysis for the classification of clouds in satellite imagery. IEEE Trans Geosci Remote Sens 41(11):2662–2668

    Article  Google Scholar 

  17. Corchado E, Baruque B, Yin H (2007) Boosting unsupervised competitive learning ensembles. In: de Sá JM, Alexandre LA, Duch W, Mandic DP (eds) Proceedings of the 17th international conference on artificial neural networks (ICANN’07), part I, Springer, vol LNCS 4668, pp 339–348

  18. del-Hoyo R, Buldain D, Marco A (2003) Supervised classification with associative SOM. In: Proceedings of the 7th international work-conference on artificial and neural networks, (IWANN)’03, pp 334–341

  19. Dietterich TG (1998) Approximate statistical tests for comparing supervised classification learning algorithms. Neural Comput 10(7):1895–1923

    Article  Google Scholar 

  20. Dietterich TG (2000) An experimental comparison of three methods for constructing ensembles of decision trees: bagging, boosting, and randomization. Mach Learn 40(2):139–157

    Article  Google Scholar 

  21. Dietterich TG (2003) Ensemble learning. In: Arbib MA (ed) The handbook of brain theory, neural networks, 2nd edn. MIT Press, Cambridge, pp 405–408

    Google Scholar 

  22. Everitt B (1977) The analysis of contingency tables. Chapman & Hall, CRC, London

    Google Scholar 

  23. Frank A, Asuncion A (2010) UCI machine learning repository. http://www.archive.ics.uci.edu/ml

  24. Georgakis A, Li H, Gordan M (2005) An ensemble of SOM networks for document organization and retrieval. In: International conference on adaptive knowledge representation and reasoning (AKRR’05), pp 141–147

  25. Gorgonio FA, Costa JAF (2008) Parallel self-organizing maps with application in clustering distributed data. In: Proceedings of the 2008 IEEE international joint conference on neural networks (IJCNN’2008), pp 3276–3283

  26. Hansen LK, Salamon P (2002) Neural network ensembles. IEEE Trans Pattern Anal Mach Intell 12(10):993–1001

    Article  Google Scholar 

  27. Haykin S (2008) Neural networks and learning machines, 3rd edn. Prentice Hall, NJ

    Google Scholar 

  28. He Q, Wang L (2007) A hybrid particle swarm optimization with a feasibility-based rule for constrained optimization. Appl Math Comput 186(2):1407–1422

    Article  MathSciNet  MATH  Google Scholar 

  29. Huang CL, Dun J (2008) A distributed PSO-SVM hybrid system with feature selection and parameter optimization. Appl Soft Comput 8(4):1381–1391

    Article  Google Scholar 

  30. Huang GB, Zhu QY, Ziew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1–3):489–501

    Article  Google Scholar 

  31. Huynh H, Yonggwan W, Kim J (2008) An improvement of extreme learning machine for compact single-hidden-layer feedforward neural networks. Int J Neural Syst 18(5):433–441

    Article  Google Scholar 

  32. Jiang Y, Zhou Z (2004) SOM ensemble-based image segmentation. Neural Process Lett 20(3):171–178

    Article  Google Scholar 

  33. Kangas JA, Kohonen TK, Laaksonen JT (1990) Variants of self-organizing maps. IEEE Trans Neural Netw 1(1):93–99

    Article  Google Scholar 

  34. Kennedy J, Eberhart RC (1995) Particle swarm optimization. In: Proceedings of the IEEE international conference on neural networks, Piscataway, NJ, USA, vol 4, pp. 1942–1948

  35. Keskin GA, Özkan C (2009) An alternative evaluation of FMEA: fuzzy art algorithm. Qual Reliabil Eng Int 25(6):647–661

    Article  Google Scholar 

  36. Kohonen T (1982) Self-organized formation of topologically correct feature maps. Biol Cybernet 43(1):59–69

    Article  MathSciNet  MATH  Google Scholar 

  37. Kohonen T (1988) An introduction to neural computing. Neural Netw 1(1):3–16

    Article  Google Scholar 

  38. Kohonen T (1988) The ’neural’ phonetic typewriter. Computer 21(3):11–22

    Article  Google Scholar 

  39. Kohonen T (1990) The self-organizing map. Proc of the IEEE 78(9):1464–1480

    Article  Google Scholar 

  40. Kohonen T (2003) Learning vector quantization. In: Arbib MA (ed) The handbook of brain theory, neural networks, 2nd edn. MIT Press, Cambridge, pp 631–635

    Google Scholar 

  41. Krogh A, Vedelsby J (1995) Neural network ensembles, cross validation, active learning. Advances in neural information processing systems, pp. 231–238

  42. Laha A, Pal NR (2001) Some novel classifiers designed using prototypes extracted by a new scheme based on self-organizing feature map. IEEE Trans Syst Man Cybern B 31(6):881–890

    Article  Google Scholar 

  43. Liu N, Wang H (2010) Ensemble based extreme learning machine. IEEE Signal Process Lett 17(8):754–757

    Article  Google Scholar 

  44. Loo CK, Law A, Lim WS, Rao MVC (2006) Probabilistic ensemble simplified fuzzy ARTMAP for sonar target differentiation. Neural Comput Appl 15(1):79–90

    Article  Google Scholar 

  45. Madeo R, Peres SM, Bíscaro HH, Dias DB, Boscarioli C (2010) A committee machine implementing the pattern recognition module for fingerspelling applications. In: Proceedings of the 2010 ACM symposium on applied computing (SAC’2010), pp 954–958

  46. Martinetz TM, Schulten KJ (1991) A “neural-gas” network learns topologies. In: Kohonen T, Makisara K, Simula O, Kangas J (eds) Artificial neural networks. North-Holland, Amsterdam, pp 397–402

    Google Scholar 

  47. Monteiro IQ, Queiroz SA, Carneiro AT, Souza LGM, Barreto GA (2006) Face recognition independent of facial expression through SOM-based classifiers. In: Proceedings of the IEEE/SBrT international telecommunications symposium (ITS’06), Fortaleza, Ceará, Brazil, pp 263–268

  48. Pedersen MEH, Chipperfield AJ (2010) Simplifying particle swarm optimization. Appl Soft Comput 10(2):618–628

    Article  Google Scholar 

  49. Pedreira CE (2006) Learning vector quantization with training data selection. IEEE Trans Pattern Anal Mach Intell 28(1):157–162

    Article  Google Scholar 

  50. Petrikieva L, Fyfe C (2002) Bagging and bumping self-organising maps. Comput Inform Syst 9(2):69

    Google Scholar 

  51. Platt JC (1998) Fast training of support vector machines using sequential minimal optimization. In: Advances in Kernel methods: support vector learning. MIT Press, Cambridge, pp 185–208

  52. Raafat HM, Tolba AS, Aly AM (2011) A novel training weighted ensemble (TWE) with application to face recognition. Appl Soft Comp 3608–3617

  53. Rocha M, Cortez P, Neves J (2007) Evolution of neural networks for classification and regression. Neurocomputing 70(16–18):1054–1060

    Google Scholar 

  54. Rocha Neto AR, Barreto GA (2009) On the application of ensembles of classifiers to the diagnosis of pathologies of the vertebral column: a comparative analysis. IEEE Latin Am Trans 7(4):487–496

    Article  Google Scholar 

  55. Rokach L (2010) Ensemble-based classifiers. Artif Intell Rev 33(1-2):1–39

    Article  Google Scholar 

  56. Santos AM, Canuto AMP (2008) Using ARTMAP-based ensemble systems designed by three variants of boosting. In: Proceedings of the international conference on artificial neural networks (ICANN’08), pp 562–571

  57. Sartain P, Hopkins ABT, McDonald-Mair KD, Howells WGJ (2008) A framework for self-diagnosis, condition monitoring for embedded systems using a SOM-based classifier. In: NASA/ESA conference on adaptive hardware, systems 2008, IEEE, pp 417–423

  58. Scherbart A, Nattkemper TW (2011) Looking inside self-organizing map ensembles with resampling and negative correlation learning. Neural Netw 24(1):130–141

    Article  MATH  Google Scholar 

  59. Souza~Júnior AH, Barreto GA, Varela AT (2011) A speech recognition system for embedded applications using the SOM and TS-SOM networks. In: Mwasiagi JI (ed) Self-organizing maps—applications and novel algorithm design. InTech Open, Croatia, pp 97–108

    Google Scholar 

  60. Suganthan PN (1999) Hierarchical overlapped SOM’s for pattern classification. IEEE Trans Neural Netw 10(1):193–196

    Article  Google Scholar 

  61. Turky AM, Ahmad MS (2010) The use of SOM for fingerprint classification. In: IEEE international conference on information retrieval & knowledge management (CAMP’2010), pp 287–290

  62. Umer MF, Khiyal MSH (2007) Classification of textual documents using learning vector quantization. Inform Technol J 6:154–159

    Article  Google Scholar 

  63. Wyns B, Sette S, Boullart L, Baeten D, Hoffman IEA, De Keyser F (2004) Prediction of diagnosis in patients with early arthritis using a combined Kohonen mapping and instance-based evaluation criterion. Artif Intell Med 31(1):45–55

    Article  Google Scholar 

  64. Wyns B, Sette S, Boullart L, Baeten D, Hoffman IEA, Keyser FD (2004) Prediction of diagnosis in patients with early arthritis using a combined Kohonen mapping, instance-based evaluation criterion. Artif Intell Med 31(1):45–55

    Article  Google Scholar 

  65. Xiao YD, Clauset A, Harris R, Bayram E, Santago P, Schmitt JD (2005) Supervised self-organizing maps in drug discovery. 1. Robust behavior with overdetermined data sets. J Chem Inform Model 45(6):1749–1758

    Article  Google Scholar 

Download references

Acknowledgments

The authors thank CAPES for the financial support. We also thank Prof. Ajalmar R. R. Neto for running the experiments with the SVM classifiers on the VCP data set.

Author information

Authors and Affiliations

  1. Department of Teleinformatics Engineering, Federal University of Ceará, Av. Mister Hull, S/N, Campus of Pici, Center of Technology CP 6005, Fortaleza, Ceará, 60455-970, Brazil

    César L. C. Mattos & Guilherme A. Barreto

Authors
  1. César L. C. Mattos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guilherme A. Barreto
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to César L. C. Mattos.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Mattos, C.L.C., Barreto, G.A. ARTIE and MUSCLE models: building ensemble classifiers from fuzzy ART and SOM networks. Neural Comput & Applic 22, 49–61 (2013). https://doi.org/10.1007/s00521-011-0747-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-011-0747-7

Keywords

Search

Navigation