Skip to main content
SpringerLink
Log in

Feature Extraction Using Independent Components of Each Category

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

We describe an application of independent component analysis (ICA) to pattern recognition in order to evaluate the effectiveness of features extracted by ICA. We propose a recognition method suitable for independent components that consists of modules for each category. A module has two parts: feature extraction and classification. Features are independent components estimated by ICA and outputs of modules are candidates for categories. These candidates are combined and categories are decided with a majority rule. This recognition method is applied to two tasks: hand-written digits in the MNIST database and acoustic diagnosis for a compressor as real-world tasks. A FastICA algorithm is applied to extracting independent features in the proposed method. Through recognition experiments, we demonstrate that the ICA of each category extracts useful features for these tasks and the independent components are superior to the principal components in recognition accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Abbreviations

ICA:

Independent Component Analysis

PCA:

Principal Comoponent Analysis

References

  1. B. Olshausen D. Field (1996) ArticleTitleWavelet-like receptive fields emerge from a network that learns sparse codes for natural images Nature 381 607–609

    Google Scholar 

  2. A. J. Bell T. J. Sejnowski (1997) ArticleTitleThe independent components of natural scenes are edge filters Vision Research 37 3327–3338 Occurrence Handle10.1016/S0042-6989(97)00121-1 Occurrence Handle9425547

    Article  PubMed  Google Scholar 

  3. A. Hyvärinen P. Hoyer (2000) ArticleTitleEmergence of phase and shift invariant features by decomposition of natural images into independent feature subspaces Neural Computation 12 IssueID,1705–1720 12 ,1705–1720

    Google Scholar 

  4. A.J. Bell T.J. Sejnowski (1996) ArticleTitleLearning the higher-order structure of a natural sound Network: Computation in Neural Systems 7 IssueID261–266 7, 261–266

    Google Scholar 

  5. Lewicki, M. S. and Sejnowski, T. J.: Learning Nonlinear Overcomplete Representations for Efficient Coding, In: Advances in Neural Information Processing Systems 10, The MIT Press, (1998), 556–562.

  6. A.D. Back A.S. Weigend (1997) ArticleTitleA First application of independent component analysis to extracting structure from stock returns International Journal of Neural Systems 8 IssueID4 473–484 Occurrence Handle10.1142/S0129065797000458 Occurrence Handle9730022

    Article  PubMed  Google Scholar 

  7. Weingessel, A., Natter, M. and Hornik, K.: Using independent component analysis for feature extraction and multivariate data projection In: Working Papers SFB “Adaptive information Systems and Modeling in Economics and Management Science” (1998).

  8. M. S. Bartlett J.R. Movellan T.J. Sejnowski (2002) ArticleTitleFace recognition by independent component analysis IEEE Transaction on Neural Networks 13 IssueID6 1450–1464 Occurrence Handle10.1109/TNN.2002.804287

    Article  Google Scholar 

  9. Lee J. H., Jung, H. J., Lee, T. W. and Lee, S. Y.: Speed feature extraction using independent component analysis. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing,III (2000), 1631–1634.

  10. Ozawa. S., Tusjimoto, T., Kotani, M. and Baba, N.: Application of independent component analysis to hand-written Japanese character recognition. In: Proceeding of the International Conference on Neural Networks(1999).

  11. Ozawa, S. and Kotani, M.: A study of feature extraction and selection using independent component analysis, In: Proceedings of the International Conference on Neural Information Processing (2000).

  12. A.J. Bell T.J. Sejnowski (1995) ArticleTitleAn information maximization approach to blind separation and blind deconvolution Neural Computation 7 1129–1159 Occurrence Handle7584893

    PubMed  Google Scholar 

  13. Kwak, N., Choi, C. H. and Choi J. Y.: Feature extraction using ICA. In: Proceedings of the International Conference on Artificial Neural Networks (200l), 568–573.

  14. S. Akaho (2002) ArticleTitleConditionally Independent component analysis for supervised feature extraction Neurocomputing 49 139–150 Occurrence Handle10.1016/S0925-2312(02)00518-0

    Article  Google Scholar 

  15. J. Karhunen S. Malaroiu M. Ilinoniemi (2000) ArticleTitleLocal linear independent component analysis using clustering International Journal of Neural Systems 10 IssueID6 439–451 Occurrence Handle11307858

    PubMed  Google Scholar 

  16. T. W. Lee M. S. Lewicki. T. J. Sejnowski (2000) ArticleTitleICA mixture models for unsuper vised classification of non-Gaussian classes and automatic context switching in blind signal separation IEEE Transaction on Pattern Analysis and Machine Intelligence 20 IssueID10 1078–1089

    Google Scholar 

  17. Hyvärinen, A., Karhunen, J. and Oja, E.: Independent Component Analysis, John Wiley and Sons and (2001)

  18. A. Hyvärinen O. Oja (1997) ArticleTitleA fast fixed-point algorithm for independent component analysis Neural Computation 9 1483–1492

    Google Scholar 

  19. Cichocki, A. and Amari, S.: Adaptive Blind Signal and Image Processing. Wiley (2002).

  20. C. Jutten J. Herault (1991) ArticleTitleBlind. Separation of Sources, Part I: An Adaptive Algorithm, Based on Neuromimetic Architecture Signal Processing 24 1–10 Occurrence Handle10.1016/0165-1684(91)90079-X

    Article  Google Scholar 

  21. J.F. Cardoso A. Souloumiac (1993) ArticleTitleBlind beamforming for non-Gaussian signals IEE proceedings-F 140 IssueID6 362–370

    Google Scholar 

  22. Amari, S., Cihocki, A. and Yang, H.: A new learning algorithm for blind signal separation, In: Advances in Neural Information Processing Systems 8,The MIT Press (l996), 757–763.

  23. X. Giannakopoulos J. Karhunen E. Oja (1999) ArticleTitleAn experimental comparison of neural algorithms for independent component analysis and blind separation International journal of Neural, Systems 9 997–114

    Google Scholar 

  24. Duda, R. O., Hart, P. E. and Stork, D. G. Pattern classification (Second Edition), John Wiley and Sons (2001).

  25. Vapnik, V. N.: The Nature of Statistical Learning Theory (Second Edition), Springer (1999).

  26. Rumelhart, D. E., Hinton, G. E. and Williams. R. J.: Learning international representations by error propagation. In: Parallel Distributed Processing: Exploration in the Microstructure of Cognition I, The MIT Press (1986).

  27. Belongie, S. and Malik, J.; Matching with shape contexts, In: Proceedings of the IEEE Workshop on Content-based Across of Image and Video Libraries(2000), 20–26.

  28. Kotani, M., Kajiki, A. and Akazawa, K.: A structural learning algorithm for multi-layered neural networks, In: Proceedings of the International Conference on Neural Networks II (1997), 1105–1110.

Download references

Author information

Authors and Affiliations

  1. Faculty of Engineering, Kobe University, Japan

    Manabu Kotani

  2. Graduate School of Science and Technology, Kobe University, 1-1 Rokkodai, Nada, Kobe, 657-8501, Japan

    Seiichi Ozawa

Authors
  1. Manabu Kotani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Seiichi Ozawa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Manabu Kotani.

Additional information

Manabu Kotani - Deceased

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kotani, M., Ozawa, S. Feature Extraction Using Independent Components of Each Category. Neural Process Lett 22, 113–124 (2005). https://doi.org/10.1007/s11063-004-0634-7

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-004-0634-7

Keywords

Search

Navigation