Skip to main content
SpringerLink
Log in

Feature Extraction-Based Deep Self-Organizing Map

  • Published:
Circuits, Systems, and Signal Processing Aims and scope Submit manuscript

Abstract

We propose in this work, a new approach for feature extraction based on deep Self-Organizing Map (SOM) network, named Generalized Unsupervised Deep SOM (G-UDSOM). This work presents an enhancement of the classic unsupervised deep SOM (UDSOM) algorithm in two ways. First, we modify the UDSOM Sub-sampling module in such a way that the image reconstruction phase is applied only in the feature construction phase. Second, the modified sub-sampling module learn feature of different sizes and resolutions through different map and patch sizes, which allows building parallel learning architecture. We add compact reconstruction constraint through the SOM learning based on convolutional auto-encoder. Thus. G-UDSOM allows extracting latent representation through its internal layer, by minimizing cost function. This function is composed of two terms: the cost SOM function mean square error of reconstruction. The second objective of our extended version is to achieve reduced computation time while improving accuracy and generalization capability. Different from the earlier proposed UDSOM architectures, G-UDSOM uses parallel SOMs without any reconstruction in the hidden layers. We evaluate our proposed approach on MNIST and STL-10 datasets. Experiment results show that our proposed approach outperforms deep SOMs models in terms of classification accuracy, training time and computational complexity. Our method is also can be generalized across different dataset without any pre-trained models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Data Availability

The datasets analyzed during the current study are publically available. MNIST: http://yann.lecun.com/exdb/mnist/. STL-10: https://cs.stanford.edu/~acoates/stl10/.AFHQ: https://www.kaggle.com/andrewmvd/animal-faces. USC-SIPI: https://sipi.usc.edu/database/.

References

  1. F.B. Aissa, M. Sakkari, R. Ejbali, M. Zaied, Unsupervised features extraction using a multi-view self organizing map for image classification, in IEEE/ACS 14th International Conference on Computer Systems and Applications (2017), pp. 196–201. https://doi.org/10.1109/AICCSA.2017.104

  2. S. Aly, S. Almotairi, Deep convolutional self-organizing map network for robust handwritten digit recognition. IEEE Access 8, 107035–107045 (2020). https://doi.org/10.1109/ACCESS.2020.3000829

    Article  Google Scholar 

  3. D. Ao, Integration of Unsupervised Feature Learning and Neural Networks Applied to Image Recognition (South China University of Technology, Guangzhou, 2014), pp. 19–37

    Google Scholar 

  4. P. Arena, A. Basile, M. Bucolo, L. Fortuna, An object oriented segmentation on analog CNN chip. IEEE Trans. Circuits Syst. I Fundam. Theory Appl. 50(7), 837–846 (2003). https://doi.org/10.1109/TCSI.2003.813985

    Article  Google Scholar 

  5. B.M. Barbalho, J.A.F. Costa, A.D.D. Neto, M.L.A. Netto, Hierarchical and dynamic SOM applied to image compression, in Proceedings of the International Joint Conference on Neural Networks (2003), pp. 753–758. https://doi.org/10.1109/IJCNN.2003.1223472

  6. J.M. Barbalho, A. Duarte, D. Neto, J.A.F. Costa and M.L.A. Netto, Hierarchical SOM applied to image compression, in International Joint Conference on Neural Networks (2001), pp. 442–447. https://doi.org/10.1109/IJCNN.2001.939060

  7. Y. Bengio, Learning deep architectures for AI. Found. Trends Mach. Learn. 2(1), 1–127 (2009). https://doi.org/10.1561/2200000006

    Article  MathSciNet  MATH  Google Scholar 

  8. Y. Bengio, P. Lamblin, D. Popovici, H. Larochelle, Greedy layer-wise training of deep networks, in Proceedings of the 19th International Conference on Neural Information Processing Systems (2006), pp. 153–160

  9. L. Bo, X. Ren, D. Fox, Unsupervised feature learning for RGB-D based object recognition, in Experimental Robotics. Springer Tracts in Advanced Robotics, vol. 88, ed. by J. Desai, G. Dudek, O. Khatib, V. Kumar (Springer, Heidelberg, 2013). https://doi.org/10.1007/978-3-319-00065-7_27

    Chapter  Google Scholar 

  10. J. Bruna, S. Mallat, Invariant scattering convolution networks. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1872–1886 (2013). https://doi.org/10.1109/TPAMI.2012.230

    Article  Google Scholar 

  11. T. Chan, K. Jia, S. Gao, J. Lu, Z. Zeng, Y. Ma, PCANet: a simple deep learning baseline for image classification? IEEE Trans. Image Process. 24(12), 5017–5032 (2015). https://doi.org/10.1109/TIP.2015.2475625

    Article  MathSciNet  MATH  Google Scholar 

  12. R. Collobert, J. Weston, A unified architecture for natural language processing: deep neural networks with multitask learning, in Proceedings of the 25th International Conference on Machine Learning (2008), pp. 160–167. https://doi.org/10.1145/1390156.1390177

  13. A. Dosovitskiy, J.T. Springenberg, T. Brox, Unsupervised feature learning by augmenting single images, in International Conference on Learning Representations (2013)

  14. H. Dozono, M. Tanaka, Application of self organizing map to preprocessing input vectors for convolutional neural network, in International Conference on Artificial Neural Networks ICANN, vol. 11728, pp. 96-100 (2019). https://doi.org/10.1007/978-3-030-30484-3_8

  15. A. Dundar, J. Jin, E. Culurciello, Convolutional clustering for unsupervised learning. arXiv:1511.06241 (2015)

  16. H. Elmannai, M. Hamdi, A. AlGarni, Deep learning models combining for breast cancer histopathology image classification. Int. J. Comput. Intell. Syst. 14(1), 1003–1013 (2021). https://doi.org/10.2991/ijcis.d.210301.002

    Article  Google Scholar 

  17. H. Elmannai, M.S. Naceur, M.A. Loghmari, A. AlGarni, A new feature extraction approach based on non linear source separation. Int. J. Electr. Comput. Eng. 11(5), 4082–4094 (2021). https://doi.org/10.11591/ijece.v11i5.pp4082-4094

    Article  Google Scholar 

  18. F. Forest, M. Lebbah, H. Azzag, J. Lacaille, Deep architectures for joint clustering and visualization with self-organizing maps, in Trends and Applications in Knowledge Discovery and Data Mining, Lecture Notes in Computer Science, vol. 11607 (2019)

  19. S. Furao, O. Segawa, An incremental network for on-line unsupervised classification and topology learning. Neural Netw. 19(1), 90–106 (2006). https://doi.org/10.1016/j.neunet.2005.04.006

    Article  MATH  Google Scholar 

  20. S. Furao, T. Ogura, O. Hasegawa, An enhanced self-organizing incremental neural network for online unsupervised learning. Neural Netw. 20(8), 893–903 (2007). https://doi.org/10.1016/j.neunet.2007.07.008

    Article  MATH  Google Scholar 

  21. R. Gens, P. Domingos, Discriminative learning of sum-product networks, in Advances in Neural Information Processing Systems (2012), pp. 3239–3247

  22. Y. Guo, Y. Liu, A. Oerlemans, S. Lao, S. Wu, M.S. Lew, Deep learning for visual understanding: a review. Neurocomputing 187, 27–48 (2016). https://doi.org/10.1016/j.neucom.2015.09.116

    Article  Google Scholar 

  23. M. Hamdi, H. Bouhamed, A. AlGarni, H. Elmannai, S. Meshoul, Deep learning and uniform LBP histograms for position recognition of elderly people with privacy preservation. Int. J. Comput. Commun. Control (2021). https://doi.org/10.15837/ijccc.2021.5.42562

    Article  Google Scholar 

  24. G.E. Hinton, S. Osindero, Y.W. Teh, A fast learning algorithm for deep belief nets. Neural Comput. 18(7), 1527–1554 (2006). https://doi.org/10.1162/neco.2006.18.7.1527

    Article  MathSciNet  MATH  Google Scholar 

  25. G.E. Hinton, R.R. Salakhutdinov, Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006). https://doi.org/10.1126/science.1127647

    Article  MathSciNet  MATH  Google Scholar 

  26. A.V. Joshi, Unsupervised learning in machine, in Learning and Artificial Intelligence. ed. by A.V. Joshi (Springer, Cham, 2020), pp. 133–140

    Google Scholar 

  27. T. Kohonen, E. Oja, O. Simula, A. Visa, J. Kangas, Engineering applications of the self-organizing map. Proc. IEEE 84(10), 1358–1384 (1996). https://doi.org/10.1109/5.537105

    Article  Google Scholar 

  28. N. Liu, J. Wang, A. Gong, Deep self-organizing map for visual classification, in International Joint Conference on Neural Networks (2015), pp. 1–6. https://doi.org/10.1109/IJCNN.2015.7280357

  29. B. Miclut, Committees of deep feedforward networks trained with few data, in German Conference on Pattern Recognition (2014), pp. 736–742

  30. H. Nakayama, Efficient discriminative convolution using fisher weight map, in British Machine Vision Conference (2013), pp. 1–11

  31. G. Qian, L. Zhang, A simple feedforward convolutional conceptor neural network for classification. Appl. Soft Comput. 70, 1034–1041 (2018). https://doi.org/10.1016/j.asoc.2017.08.016

    Article  Google Scholar 

  32. M.A. Ranzato, M. Szummer, Semi-supervised learning of compact document representations with deep networks, in 25th International Conference on Machine Learning (2008), pp. 792–799. https://doi.org/10.1145/1390156.1390256

  33. X. Ren, H. Guo, S. Li, S. Wang, J. Li, A novel image classification method with CNN-XGBoost model, in: C. Kraetzer, Y. Shi, J. Dittmann, H. Kim (eds.), Digital Forensics and Watermarking. Lecture Notes in Computer Science, 10431 (2014). https://doi.org/10.1007/978-3-319-64185-0_28

  34. S. Rifai, P. Vincent, X. Muller, X. Glorot, Y. Bengio, Contractive auto-encoders: explicit invariance during feature extraction, in International Conference on International Conference on Machine Learning (2011), pp. 833–840

  35. M. Sakkari, R. Ejbali, M. Zaied, Deep SOMs for automated feature extraction and classification from big data streaming, in Proceedings of the Ninth International Conference on Machine Vision. International Society for Optics and Photonics, vol. 10341 (2017). https://doi.org/10.1117/12.2269082

  36. M. Sakkari, M. Zaied, A convolutional deep self-organizing map feature extraction for machine learning. Multimedia Tools Appl 79, 19451–19470 (2020). https://doi.org/10.1007/s11042-020-08822-9

    Article  Google Scholar 

  37. F. Shen, O. Hasegawa, A fast nearest neighbor classifier based on self-organizing incremental neural network. Neural Netw. 21(10), 1537–1547 (2008). https://doi.org/10.1016/j.neunet.2008.07.001

    Article  MATH  Google Scholar 

  38. F. Shen, O. Hasegawa, Self-organizing incremental neural network and its application, in Artificial Neural Networks. ICANN 2010. Lecture Notes in Computer Science, vol. 6354, ed. by K. Diamantaras, W. Duch, L. Iliadis (Springer, Berlin, 2010). https://doi.org/10.1007/978-3-642-15825-4_74

    Chapter  Google Scholar 

  39. F. Sapuppo, M. Intaglietta, M. Bucolo, Bio-microfluidics real-time monitoring using CNN technology. IEEE Trans. Biomed. Circuits Syst. 2(2), 78–87 (2008). https://doi.org/10.1109/TBCAS.2008.925642

    Article  Google Scholar 

  40. A. Voulodimos, N. Doulamis, A. Doulamis, E. Protopapadakis, Deep learning for computer vision: a brief review. Comput. Intell. Neurosci. 2018, 1–13 (2018). https://doi.org/10.1155/2018/7068349

    Article  Google Scholar 

  41. S. Walczak, Artificial neural networks, in Advanced Methodologies and Technologies in Artificial Intelligence, Computer Simulation, and Human-Computer Interaction, pp. 40–53. IGI Global (2019). https://doi.org/10.4018/978-1-5225-7368-5

  42. C.S. Wickramasinghe, K. Amarasinghe, M. Manic, Deep self-organizing maps for unsupervised image classification. IEEE Trans. Ind. Inf. 15(11), 5837–5845 (2019). https://doi.org/10.1109/TII.2019.2906083

    Article  Google Scholar 

  43. J.J. Winston, G.F. Turker, U. Kose, D.J. Hemanth, Novel optimization based hybrid self-organizing map classifiers for iris image recognition. Int. J. Comput. Intell. Syst. 13(1), 1048–1058 (2020). https://doi.org/10.2991/ijcis.d.200721.001

    Article  Google Scholar 

  44. H. Zihang, Mnist_transformation_test_dataset, IEEE Dataport. https://ieee-dataport.org/documents/mnisttransformationtestdataset (2018). Accessed June 2019. https://doi.org/10.21227/r1jn-zf11

Download references

Acknowledgements

This research project was funded by the Deanship of Scientific Research, Princess Nourah bint Abdulrahman University, through the Program of Research Project Funding After Publication, grant No. (41-PRF-A-P-13)

Author information

Authors and Affiliations

  1. RTIM: Research Team in Intelligent Machines, Department of Electric Engineering, National School of Engineers of Gabès, University of Gabès, 6029, Gabès, Tunisia

    Mohamed Sakkari & Mourad Zaied

  2. Information Technology Department, College of Computer and Information Sciences, Princess Nourah Bint Abdulrahman University, Riyadh, 11671, Saudi Arabia

    Monia Hamdi, Hela Elmannai & Abeer AlGarni

Authors
  1. Mohamed Sakkari
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Monia Hamdi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hela Elmannai
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Abeer AlGarni
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mourad Zaied
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Monia Hamdi.

Ethics declarations

Conflict of interest

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sakkari, M., Hamdi, M., Elmannai, H. et al. Feature Extraction-Based Deep Self-Organizing Map. Circuits Syst Signal Process 41, 2802–2824 (2022). https://doi.org/10.1007/s00034-021-01914-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00034-021-01914-3

Keyword

Search

Navigation