Abstract
Self-organizing maps (SOMs) are well appropriate for visualizing high-dimensional data sets. Training SOMs on raw high-dimensional data with classic metrics often leads to problems arising from the curse-of-dimensionality effect. To achieve more valuable semantic maps of high-dimensional data sets, we assume that higher-level features are necessary. We propose to gather such higher-level features from pre-trained convolutional layers, i.e., filter banks of convolutional neural networks (CNNs). Appropriately pre-trained CNNs are required, e.g., from the same or related domains, or in semi-supervised scenarios. We introduce SOM quality measures and analyze the new approach on two benchmark image data sets considering different convolutional network levels.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
Also rectangle shapes are possible.
- 2.
MUX and quality measures are currently not used for the training process itself.
- 3.
The distance matrix contains the pairwise distances between patterns respectively positions on the map.
- 4.
The weights of the neurons cannot be used directly for visualization in the ConvSOM as they have a different format.
References
András P (2002) Kernel-kohonen networks. Int J Neural Syst 12(02):117–135. https://doi.org/10.1142/S0129065702001084
Coelho, D.N., Barreto, G.A., Medeiros, C.M.S.: Detection of short circuit faults in 3-phase converter-fed induction motors using kernel SOMs. In: WSOM, pp 1–7 (2017). https://doi.org/10.1109/WSOM.2017.8020016
Dozono H, Niina G, Araki S (2016) Convolutional self organizing map. In: 2016 international conference on computational science and computational intelligence (CSCI), pp 767–771. https://doi.org/10.1109/CSCI.2016.0149
Ferles C, Papanikolaou Y, Naidoo KJ (2018) Denoising autoencoder self-organizing map (DASOM). Neural Netw 105:112–131. https://doi.org/10.1016/j.neunet.2018.04.016
Kohonen T (1982) Self-organized formation of topologically correct feature maps. Biol Cybern 43(1):59–69. https://doi.org/10.1007/BF00337288
Krizhevsky A, Sutskever I, Hinton GE (2017) ImageNet classification with deep convolutional neural networks. Commun ACM 60(6):84–90. https://doi.org/10.1145/3065386
Kruskal JB (1964) Nonmetric multidimensional scaling: a numerical method. Psychometrika 29(2):115–129. https://doi.org/10.1007/BF02289694
Kutics A, O’Connell C, Nakagawa A (2013) Segment-based image classification using layered-SOM. In: 2013 IEEE international conference on image processing, pp 2430–2434. https://doi.org/10.1109/ICIP.2013.6738501
LeCun Y, Boser B, Denker JS, Henderson D, Howard RE, Hubbard W, Jackel LD (1989) Backpropagation applied to handwritten zip code recognition. Neural Comput 1(4):541–551. https://doi.org/10.1162/neco.1989.1.4.541
MacDonald D, Fyfe C (2000) The kernel self-organising map. In: KES 2000, fourth international conference on knowledge-based intelligent engineering systems and allied technologies, proceedings (Cat. No. 00TH8516), vol 1, pp 317–320. https://doi.org/10.1109/KES.2000.885820
Meier A, Kramer O (2017) An experimental study of dimensionality reduction methods. In: Kern-Isberner G, Fürnkranz J, Thimm M (eds) KI 2017, advances in artificial intelligence. Springer International Publishing, pp 178–192. https://doi.org/10.1007/978-3-319-67190-1_14
Miikkulainen R (1990) Script recognition with hierarchical feature maps. Connection Sci 2(1–2):83–101. https://doi.org/10.1080/09540099008915664
Platon L, Zehraoui F, Tahi F (2017) Self-organizing maps with supervised layer. In: WSOM, pp 1–8. https://doi.org/10.1109/WSOM.2017.8020022
Schleif F (2017) Small sets of random Fourier features by kernelized matrix LVQ. In: WSOM, pp 1–5. https://doi.org/10.1109/WSOM.2017.8020026
Villmann T, Biehl M, Villmann A, Saralajew S (2017) Fusion of deep learning architectures, multilayer feedforward networks and learning vector quantizers for deep classification learning. In: 2017 12th international workshop on self-organizing maps and learning vector quantization, clustering and data visualization (WSOM), pp 1–8. https://doi.org/10.1109/WSOM.2017.8020009
Wang M, Zhou W, Tian Q, Pu J, Li H (2017) Deep supervised quantization by self-organizing map. In: Proceedings of the 25th ACM international conference on multimedia, MM 2017. ACM, New York, pp 1707–1715. https://doi.org/10.1145/3123266.3123415
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Elend, L., Kramer, O. (2020). Self-Organizing Maps with Convolutional Layers. In: Vellido, A., Gibert, K., Angulo, C., Martín Guerrero, J. (eds) Advances in Self-Organizing Maps, Learning Vector Quantization, Clustering and Data Visualization. WSOM 2019. Advances in Intelligent Systems and Computing, vol 976. Springer, Cham. https://doi.org/10.1007/978-3-030-19642-4_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-19642-4_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-19641-7
Online ISBN: 978-3-030-19642-4
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)