Skip to main content
SpringerLink
Log in

Controlling Relations between the Individuality and Collectivity of Neurons and its Application to Self-Organizing Maps

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

In this paper, we consider a society of neurons where different types of neurons interact with each other. For the first approximation to this society, we suppose two types of neurons, namely, individually and collectively treated neurons. Just as individuality must be in harmony with collectivity in actual societies, individually treated neurons must cooperate with collectively treated neurons as much as possible. We here realize this cooperation by making individually treated neurons as similar to collectively treated neurons as possible. The difference between individually and collectively treated neurons is represented by the Kullback–Leibler divergence. This divergence is minimized using free energy minimization. We applied the method to three problems from the well-known machine learning database, namely wine and protein classification, and the image segmentation problem. In all three problems, we succeeded in producing clearer class structures than those obtainable using the conventional SOM. However, we observed that the fidelity to input patterns deteriorated. For this problem, we found that careful treatment of learning processes were needed to keep fidelity to input patterns at an acceptable level.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Comon P (1994) Independent component analysis: a new concept. Signal Process 36: 287–314

    Article  MATH  Google Scholar 

  2. Kohonen T (1995) Self-organizing maps. Springer, New York

    Book  Google Scholar 

  3. Kohonen T (1990) The self-organization map. Proc IEEE 78(9): 1464–1480

    Article  Google Scholar 

  4. Kohonen T (1982) Self-organized formation of topological correct feature maps. Biol Cybernet 43: 59–69

    Article  MathSciNet  MATH  Google Scholar 

  5. Sammon JW (1969) A nonlinear mapping for data structure analysis. IEEE Trans Comput C-18(5): 401–409

    Article  Google Scholar 

  6. Ultsch A, Siemon HP (1990) Kohonen self-organization feature maps for exploratory data analysis. In: Proceedings of international neural network conference. Kulwer Academic, Dordrecht, pp 305–308

  7. Ultsch A (2003) U*-matrix: a tool to visualize clusters in high dimensional data. Tech. Rep. 36, Department of Computer Science, University of Marburg, Marburg

  8. Vesanto J (1999) SOM-based data visualization methods. Intell Data Anal 3: 111–126

    Article  MATH  Google Scholar 

  9. Kaski S, Nikkila J, Kohonen T (1998) Methods for interpreting a self-organized map in data analysis. In: Proceedings of European symposium on artificial neural networks, Bruges

  10. Mao I, Jain AK (1995) Artificial neural networks for feature extraction and multivariate data projection. IEEE Trans Neural Netw 6(2): 296–317

    Article  Google Scholar 

  11. De Runz C, Desjardin E, Herbin M (2012) Unsupervised visual data mining using self-organizing maps and a data-driven color mapping. In: 16th International conference information visualisation. IEEE Computer Society, Washington, DC, pp 241–245

  12. Shieh S, Liao I (2012) A new approach for data clustering and visualization using self-organizing maps. Expert Syst Appl 39(5): 11924–11933

    Article  Google Scholar 

  13. Yin H (2002) ViSOM: a novel method for multivariate data projection and structure visualization. IEEE Trans Neural Netw 13(1): 237–243

    Article  Google Scholar 

  14. Su M-C, Chang H-T (2001) A new model of self-organizing neural networks and its application in data projection. IEEE Trans Neural Netw 123(1): 153–158

    Google Scholar 

  15. Wu S, Chow T (2005) Prsom: a new visualization method by hybridizing multidimensional scaling and self-organizing map. IEEE Trans Neural Netw 16(6): 1362–1380

    Article  Google Scholar 

  16. Xu L, Xu Y, Chow TW (2010) PolSOM: a new method for multidimentional data visualizatio. Pattern Recogn 43: 1668–1675

    Article  MATH  Google Scholar 

  17. Xu Y, Xu L, Chow T (2011) PPoSOM: a new variant of PoLSOM by using probabilistic assignment for multidimensional data visualization. Neurocomputing 74(11): 2018–2027

    Article  Google Scholar 

  18. Xu L, Chow T (2011) Multivariate data classification using PoLSOM. In: Prognostics and system health management conference (PHM-Shenzhen), IEEE Conference Publications, Shenzhen, pp 1–4

  19. Bogdan M, Rosenstiel W (2001) Detection of cluster in self-organizing maps for controlling a prostheses using nerve signals. In: Proceedings of 9th European symposium on artificial neural networks (ESANN’ 2001), D-facto, Evere, pp 131–136

  20. Brugger D, Bogdan M, Rosenstiel W (2008) Automatic cluster detection in Kohonen’s som. IEEE Trans Neural Netw 19(3): 442–459

    Article  Google Scholar 

  21. Haraguchi T, Matsushita H, Nishio Y (2009) Community self-organizing map and its application to data extraction. In: International joint conference on neural networks (IJCNN 2009), IEEE, San Jose, pp 1107–1114

  22. Li Z, Wang R, Chen L (2009) Extracting community structure of complex networks by self-organizing maps. In: Proceedings of the third international symposium on optimization and systems biology (OSB09), pp 48–56

  23. Kamimura R (2010) Information-theoretic enhancement learning and its application to visualization of self-organizing maps. Neurocomputing 73(13–15): 2642–2664

    Article  Google Scholar 

  24. Kamimura R (2011) Selective information enhancement learning for creating interpretable representations in competitive learning. Neural Netw 24(4): 387–405

    Article  Google Scholar 

  25. Kiviluoto K (1996) Topology preservation in self-organizing maps. In: Proceedings of the IEEE international conference on neural networks (ICANN’96), pp 294–299

  26. Villmann T, Herrmann RDM, Martinez T (1997) Topology preservation in self-organizing feature maps: exact definition and measurment. IEEE Trans Neural Netw 8(2): 256–266

    Article  Google Scholar 

  27. Bauer H-U, Pawelzik K (1992) Quantifying the neighborhood preservation of self-organizing maps. IEEE Trans Neural Netw 3(4): 570–578

    Article  Google Scholar 

  28. Kaski S, Nikkila J, Oja M, Venna J, Toronen P, Castren E (2003) Trustworthiness and metrics in visualizing similarity of gene expression. BMC Bioinformatics 4: 48

    Article  Google Scholar 

  29. Venna J, Kaski S (2001) Neighborhood preservation in nonlinear projection methods: an experimental study. In: Lecture notes in computer science, vol 2130, Springer, Heidelberg, pp 485–491

  30. Polzlbauer G (2004) Survey and comparison of quality measures for self-organizing maps. In: Proceedings of the fifth workshop on data analysis (WDA04), Elfa Academic Press, Slovakia, pp 67–82

  31. Lee JA, Verleysen M (2008) Quality assessment of nonlinear dimensionality reduction based on K-ary neighborhoods. In: JMLR: Workshop and conference proceedings, vol 4, pp 21–35

  32. Kamimura R, Kamimura T, Shultz TR (2001) Information theoretic competitive learning and linguistic rule acquisition. Trans Jpn Soc Artif Intell 16(2): 287–298

    Article  Google Scholar 

  33. Kamimura R (2003) Information-theoretic competitive learning with inverse Euclidean distance output units. Neural Process Lett 18: 163–184

    Article  Google Scholar 

  34. Rose K, Gurewitz E, Fox GC (1990) Statistical mechanics and phase transition in clustering. Phys Rev Lett 65(8): 945–948

    Article  Google Scholar 

  35. Rose K, Gurewitz E, Fox GC (1992) Vector quantization by deterministic annealing. IEEE Trans Inf Theory 38(4): 1249–1257

    Article  MATH  Google Scholar 

  36. Ueda N, Nakano R (1995) Deterministic annealing variant of the EM algorithm. In: Advances in neural information processing systems, pp 545–552

  37. Ueda N, Nakano R (1998) Deterministic annealing EM algorithm. Neural Netw 11: 271–282

    Article  Google Scholar 

  38. Graepel T, Burger M (1997) Obermayer K Phase transitions in stochastic self-organizing maps. Phys Rev E 56(4): 3876

    Article  Google Scholar 

  39. Heskes T (2001) Self-organizing maps, vector quantization, and mixture modeling. IEEE Trans Neural Netw 12(6): 1299–1305

    Article  Google Scholar 

  40. Kamimura R (2008) Free energy-based competitive learning for mutual information maximization. In: Proceedings of IEEE conference on systems, man, and cybernetics, pp 223–227

  41. Kamimura R (2008) Free energy-based competitive learning for self-organizing maps. In: Proceedings of artificial intelligence and applications, pp 414–419

  42. Kamimura R (2011) Constrained information maximization by free energy minimization. Int J Gen Syst 40(7): 701–725

    Article  MathSciNet  MATH  Google Scholar 

  43. Kamimura R (2011) Self-enhancement learning: target-creating learning and its application to self-organizing maps. Biol Cybernet 104(4–5): 305–338

    Article  MathSciNet  MATH  Google Scholar 

  44. Vesanto J, Himberg J, Alhoniemi E, Parhankangas J (2000) SOM Toolbox for Matlab 5. Tech. Rep. A57, Laboratory of Computer and Information Science, Helsinki University of Technology, Helsinki

  45. Venna J (2007) Dimensionality reduction for visual exploration of similarity structures. Dissertation, Helsinki University of Technology

  46. Frank A, Asuncion A (2010) UCI machine learning repository. School of Information and Computer Sciences, University of California, Irvine. http://archive.ics.uci.edu/ml

  47. Nakai K, Kanehisa M (1991) Expert system for predicting protein localization sites in gram-negative bacteria. Proteins: Struct, Funct, Bioinform 11(2): 95–110

    Article  Google Scholar 

  48. Nakai K, Kanehisa M (1992) A knowledge base for predicting protein localization sites in eukaryotic cells. Genomics 14(4): 897–911

    Article  Google Scholar 

  49. Horton P, Nakai K (1996) A probabilistic classification system for predicting the cellular localization sites of proteins. In: Proceedings of the fourth international conference on intelligent systems for molecular biology, AAAI Press, St. Louis, pp 109–115

  50. Merenyi E, Jain A, Villmann T (2007) Explicit magnification control of self-organizing maps for forbidden data. IEEE Trans Neural Netw 18(3): 786–797

    Article  Google Scholar 

  51. Merenyi E, Jain A (2004) Forbidden magnification? II. In: Proceedings of 12th European symposium on artificial neural networks (ESANN’2004), Bruges, pp 57–62

  52. Kamimura R (2012) Comprehensibility maximization and humanly comprehensible representations. Int J Gen Syst 41(3): 265–287

    Article  MathSciNet  MATH  Google Scholar 

  53. Van Hulle MM (1999) Faithful representations with topographic maps. Neural Networks 12(6): 803–823

    Article  Google Scholar 

  54. Van Hulle MM (2004) Entropy-based kernel modeling for topographic map formation. IEEE Trans Neural Netw 15(4): 850–858

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. IT Education Center, 1117 Kitakaname, Hiratsuka, Kanagawa, 259-1292, Japan

    Ryotaro Kamimura

Authors
  1. Ryotaro Kamimura
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ryotaro Kamimura.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kamimura, R. Controlling Relations between the Individuality and Collectivity of Neurons and its Application to Self-Organizing Maps. Neural Process Lett 38, 177–203 (2013). https://doi.org/10.1007/s11063-012-9256-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-012-9256-7

Keywords

Search

Navigation