Skip to main content
SpringerLink
Log in

A Convolutional Deep Self-Organizing Map Feature extraction for machine learning

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In this work we propose a new Unsupervised Deep Self-Organizing Map (UDSOM) algorithm for feature extraction, quite similar to the existing multi-layer SOM architectures. The principal underlying idea of using SOMs is that if a neuron is wins n times, these n inputs that activated this neuron are similar. The basic principle consists of an alternation of phases of splitting and abstraction of regions, based on a non-linear projection of high-dimensional data over a small space using Kohonen maps following a deep architecture. The proposed architecture consists of a splitting process, layers of alternating self-organizing, a rectification function RELU and an abstraction layer (convolution-pooling). The self-organizing layer is composed of a few SOMs with each map focusing on modelling a local sub-region. The most winning neurons of each SOM are then organized in a second sampling layer to generate a new 2D map. In parallel to this transmission of the winning neurons, an abstraction of the data space is obtained after the convolution-pooling module. The ReLU is then applied. This treatment is applied more than once, changing the size of the splitting window and the displacement step on the reconstructed input image each time. In this way, local information is gathered to form more global information in the upper layers by applying each time a convolution filter of the level. The architecture of the Unsupervised Deep Self-Organizing Map is unique and retains the same principle of deep learning algorithms. This architecture can be very interesting in a Big Data environment for machine learning tasks. Experiments have been conducted to discuss how the proposed architecture shows this performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. [13] “In this paper, we propose CNN which uses Self Organizing Map (SOM)… as neural network”

References

  1. Aissa FB, Sakkari M, Ejbali R, Zaied M (2017) Unsupervised features extraction using a multi-view self organizing map for image classification. In 2017 IEEE/ACS 14th international conference on computer systems and applications (AICCSA) (pp. 196-201). IEEE

  2. Ao D (2014) Integration of unsupervised feature learning and neural networks applied to image recognition, pp. 19–37. South China University of Technology

  3. Bakalos N, Voulodimos A, Doulamis N, Doulamis A, Ostfeld A, Salomons E, … Li P (2019) Protecting water infrastructure from cyber and physical threats: using multimodal data fusion and adaptive deep learning to monitor critical systems. IEEE Signal Process Mag 36(2):36–48

    Article  Google Scholar 

  4. Barbalho JM, Costa JAF, Neto ADD, Netto MLA (2003) Hierarchical and dynamic SOM applied to image compression. In proceedings of the international joint conference on neural networks, 2003. (Vol. 1, pp. 753-758). IEEE

  5. Bengio Y (2009) Learning deep architectures for AI. Foundations and trends® in Machine Learning 2(1):1–127

    Article  MATH  Google Scholar 

  6. Bengio Y, Lamblin P, Popovici D, Larochelle H (2007) Greedy layer-wise training of deep networks. In advances in neural information processing systems (pp. 153-160)

  7. Bo L, Ren X, Fox D. (2013) Unsupervised feature learning for RGB-D based object recognition. In: Desai J., Dudek G., Khatib O., Kumar V. (eds) experimental robotics. Springer tracts in advanced robotics, vol 88. Springer, Heidelberg

  8. Chan TH, Jia K, Gao S, Lu J, Zeng Z, Ma Y (2015) PCANet: a simple deep learning baseline for image classification? IEEE Trans Image Process 24(12):5017–5032

    Article  MathSciNet  MATH  Google Scholar 

  9. Chan TH, Jia K, Gao S, Lu J, Zeng Z, Ma Y (2015) PCANet: a simple deep learning baseline for image classification? IEEE Trans Image Process 24(12):5017–5032

    Article  MathSciNet  MATH  Google Scholar 

  10. Coates A, Ng A, Lee H (2011) An analysis of single-layer networks in unsupervised feature learning. In proceedings of the fourteenth international conference on artificial intelligence and statistics (pp. 215-223)

  11. Collobert R, Weston J (2008) A unified architecture for natural language processing: deep neural networks with multitask learning. In proceedings of the 25th international conference on machine learning (pp. 160-167). ACM

  12. Dosovitskiy A, Springenberg JT, Brox T (2013) Unsupervised feature learning by augmenting single images. arXiv preprint. arXiv:1312.5242.

  13. Dozono H, Niina G, Araki S (2016) Convolutional self organizing map. In 2016 international conference on computational science and computational intelligence (CSCI) (pp. 767-771). IEEE

  14. Dundar A, Jin J, Culurciello E (2015) Convolutional clustering for unsupervised learning. arXiv preprint. arXiv:1511.06241.

  15. Elend L, Kramer O (2019) Self-organizing maps with convolutional layers. In international workshop on self-organizing maps (pp. 23-32). Springer, Cham

  16. Furao S, Hasegawa O (2006) An incremental network for on-line unsupervised classification and topology learning. Neural Netw 19(1):90–106

    Article  MATH  Google Scholar 

  17. Gens R, Domingos P (2012) Discriminative learning of sum-product networks. In: Advances in Neural Information Processing Systems pp 3239–3247

  18. Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT press

  19. Guo Y, Liu Y, Oerlemans A, Lao S, Wu S, Lew MS (2016) Deep learning for visual understanding: a review. Neurocomputing 187:27–48

    Article  Google Scholar 

  20. Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504–507

    Article  MathSciNet  MATH  Google Scholar 

  21. Hinton GE, Osindero S, Teh YW (2006) A fast learning algorithm for deep belief nets. Neural Comput 18(7):1527–1554

    Article  MathSciNet  MATH  Google Scholar 

  22. Jaitly N, Nguyen P, Senior A, Vanhoucke V (2012) Application of pretrained deep neural networks to large vocabulary speech recognition. In Thirteenth Annual Conference of the International Speech Communication Association

  23. Joshi AV (2020) Unsupervised learning. In machine learning and artificial intelligence (pp. 133–140). Springer, Cham

  24. Kohonen T (1990) The self-organizing map. Proc IEEE 78(9):1464–1480

    Article  Google Scholar 

  25. Kohonen T, Oja E, Simula O, Visa A, Kangas J (1996) Engineering applications of the self-organizing map. Proc IEEE 84(10):1358–1384

    Article  Google Scholar 

  26. LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  27. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444

    Article  Google Scholar 

  28. Lichodzijewski P, Zincir-Heywood AN, Heywood MI (2002) Host-based intrusion detection using self-organizing maps. In proceedings of the 2002 international joint conference on neural networks. IJCNN'02 (cat. No. 02CH37290) (Vol. 2, pp. 1714-1719). IEEE

  29. Liu N, Wang J, Gong Y (2015) Deep self-organizing map for visual classification. In 2015 international joint conference on neural networks (IJCNN) (pp. 1-6). IEEE.

  30. Miclut B (2014) Committees of Deep Feedforward Networks Trained with Few Data. In: Jiang X., Hornegger J., Koch R. (eds) Pattern Recognition. GCPR 2014. Lecture notes in computer science, vol 8753. Springer, Cham

  31. Najjar T, Hasegawa O (2013) Self-organizing incremental neural network (SOINN) as a mechanism for motor babbling and sensory-motor learning in developmental robotics. In international work-conference on artificial neural networks (pp. 321-330). Springer, Berlin, Heidelberg

  32. Nakayama H (2013) Efficient discriminative convolution using fisher weight map. In: BMVC.

  33. Ranzato MA, Szummer M (2008) Semi-supervised learning of compact document representations with deep networks. In proceedings of the 25th international conference on machine learning (pp. 792-799). ACM

  34. Ren X, Guo H, Li S, Wang S, Li J (2017) A novel image classification method with CNN-XGBoost model. In international workshop on digital watermarking (pp. 378-390). Springer, Cham

  35. Sakkari M, Ejbali R, Zaied M (2017) Deep SOMs for automated feature extraction and classification from big data streaming. In ninth international conference on machine vision (ICMV 2016) (Vol. 10341, p. 103412L). International Society for Optics and Photonics

  36. Shen F, Hasegawa O (2008) A fast nearest neighbor classifier based on self-organizing incremental neural network. Neural Netw 21(10):1537–1547

    Article  MATH  Google Scholar 

  37. Shen F, Hasegawa O (2010) Self-organizing incremental neural network and its application. In international conference on artificial neural networks (pp. 535-540). Springer, Berlin, Heidelberg

  38. Vellido A (2019) Advances in self-organizing maps, learning vector quantization, clustering and data visualization: proceedings of the 13th international workshop, WSOM+ 2019, Barcelona, Spain, June 26–28, 2019. Springer

  39. Voulodimos A, Doulamis N, Doulamis A, Protopapadakis E (2018) Deep learning for computer vision: a brief review. Computational intelligence and neuroscience, 2018.

  40. Walczak S (2019) Artificial neural networks. In advanced methodologies and Technologies in Artificial Intelligence, computer simulation, and human-computer interaction (pp. 40-53). IGI global

  41. Wicramasinghe CS, Amarasinghe K, Manic M (2019) Deep self-organizing maps for unsupervised image classification. IEEE Transactions on Industrial Informatics

  42. Wong HS, Ma B, Sha Y, Ip HH (2008) 3D head model retrieval in kernel feature space using HSOM. Pattern Recogn 41(2):468–483

    Article  MATH  Google Scholar 

Download references

Acknowledgements

The authors would like to acknowledge the financial support of this work by grants from General Direction of Scientific Research (DGRST), Tunisia, under the ARUB program.

Author information

Authors and Affiliations

  1. RTIM: Research Team in Intelligent Machines, University of Gabes, National School of Engineers of Gabes, 77 Rue de Barcelone, Zrig, 6033, Gabes, Tunisia

    Mohamed Sakkari & Mourad Zaied

Authors
  1. Mohamed Sakkari
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mourad Zaied
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohamed Sakkari.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sakkari, M., Zaied, M. A Convolutional Deep Self-Organizing Map Feature extraction for machine learning. Multimed Tools Appl 79, 19451–19470 (2020). https://doi.org/10.1007/s11042-020-08822-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-08822-9

Keywords

Search

Navigation