Skip to main content

Robust and Reliable Feature Extractor Training by Using Unsupervised Pre-training with Self-Organization Map

  • Conference paper
Robot Intelligence Technology and Applications 3

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 345))

  • 3902 Accesses

Abstract

Recent research has shown that deep neural network is very powerful for object recognition task. However, training the deep neural network with more than two hidden layers is not easy even now because of regularization problem. To overcome such a regularization problem, some techniques like dropout and de-noising were developed. The philosophy behind de-noising is to extract more robust features from the training data. For that purpose, randomly corrupted input data are used for training an auto-encoder or Restricted Boltzmann machine (RBM). In this paper, we propose unsupervised pre-training with a Self-Organization Map (SOM) to increase robustness and reliability of feature extraction. The basic idea is that instead of random corruption, our proposed algorithm works as a feature extractor so that corrupted input maintains the main skeleton or structure of original data. As a result, our proposed algorithm can extract more robust features related to input data.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Bengio, Y.: Learning deep architectures for AI. Foundations and Trends® in Machine Learning 2(1), 1–127 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  2. Aarts, E.H., Jan, H.M.: Boltzmann machines and their applications. In: Treleaven, P.C., Nijman, A.J., de Bakker, J.W. (eds.) PARLE 1987. LNCS, vol. 258, pp. 34–50. Springer, Heidelberg (1987)

    Chapter  Google Scholar 

  3. Kohonen, T.: The self-organizing map. Proceedings of the IEEE 78(9), 1464–1480 (1990)

    Article  Google Scholar 

  4. Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back-propagating errors. Cognitive Modeling (1988)

    Google Scholar 

  5. Rojas, R.: Unsupervised Learning and Clustering Algorithms. In: Neural Networks, pp. 99–121. Springer, Heidelberg (1996)

    Chapter  Google Scholar 

  6. Vincent, P., et al.: Extracting and composing robust features with denoising autoencoders. In: Proceedings of the 25th International Conference on Machine Learning. ACM (2008)

    Google Scholar 

  7. Hinton, G.: A practical guide to training restricted Boltzmann machines. Momentum 9(1), 926 (2010)

    Google Scholar 

  8. Le, Q.V.: Building high-level features using large scale unsupervised learning. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE (2013)

    Google Scholar 

  9. Lee, H., et al.: Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In: Proceedings of the 26th Annual International Conference on Machine Learning. ACM (2009)

    Google Scholar 

  10. Hinton, G.E.: Deep belief networks. Scholarpedia 4(5), 5947 (2009)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to You-Min Lee .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Lee, YM., Kim, JH. (2015). Robust and Reliable Feature Extractor Training by Using Unsupervised Pre-training with Self-Organization Map. In: Kim, JH., Yang, W., Jo, J., Sincak, P., Myung, H. (eds) Robot Intelligence Technology and Applications 3. Advances in Intelligent Systems and Computing, vol 345. Springer, Cham. https://doi.org/10.1007/978-3-319-16841-8_16

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-16841-8_16

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-16840-1

  • Online ISBN: 978-3-319-16841-8

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics