Skip to main content

Advertisement

Springer Nature Link
Log in

A criterion for measuring the separability of clusters and its applications to principal component analysis

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Reducing the dimensionality of the data as a pre-processing step of a pattern recognition application is very important. While applying the well-known Principal Component Analysis to a large data set, it is not always clear how to choose the dimension of the reduced feature space so that it would reflect appropriately the inherent dimensionality of the original feature space. We propose a simple criterion to select the reduced dimension for the sake of separability or classifiability. Essentially, the proposed criterion should be applicable to many types of data. Extensive implementation has been done to test the validity and efficiency of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bishop C.M.: Pattern Recognition and Machine Learning. Springer, New York (2006)

    MATH  Google Scholar 

  2. Duda R.O., Hart P.E., Stork D.G.: Pattern Classification, 2nd edn. Wiley, New York (2001)

    MATH  Google Scholar 

  3. Jain, A.K., Chandrasekaran, B.: Dimensionality and sample size considerations in pattern recognition practice. In: Handbook of Statistics, pp. 835–855. North Holland, Amsterdam (1982)

  4. Cangelosi, R., Goriely, A.: Component retention in principal component analysis with application to cDNA microarray data. Biol. Direct 2(2) (2007). http://www.biology-direct.com/content/2/1/2

  5. Hadsell, A., Chopra, S., LeCun, Y.: Dimensionality reduction by learning an invariant mapping. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1735–1742 (2006)

  6. Jolliffe I.T.: Principal Component Analysis, 2nd edn. Springer, Berlin (2002)

    MATH  Google Scholar 

  7. Rayner M.L., Punch W.F., Goodman E.D., Kuhn L.A., Jain A.K.: Dimensionality reduction using genetic algorithms. IEEE Trans. Evol. Comput. 4(2), 164–171 (2000)

    Article  Google Scholar 

  8. Tipping M.E., Bishop C.M.: Probabilistic principal component analysis. J. R. Stat. Soc. Lond. U.K., Ser. B 61(3), 611–622 (1999)

    Article  MATH  MathSciNet  Google Scholar 

  9. Fukunaga K., Olsen D.R.: An algorithm for finding intrinsic dimensionality of data. IEEE Trans. Comp. 20(2), 176–183 (1971)

    Article  MATH  Google Scholar 

  10. Jackson J.E.: A User’s Guide to Principal Components. Wiley, New York (2003)

    Google Scholar 

  11. Vailara A., Zhang H., Yang C., Liu F.I., Jain A.K.: Automatic image orientation detection. IEEE Trans. Image Process 11(7), 746–755 (2003)

    Article  Google Scholar 

  12. Brunelli R., Poggio T.: Face recognition: features vursus templates. IEEE Trans. Pattern Anal. Mach. Intell. 15(10), 1042–1052 (1993)

    Article  Google Scholar 

  13. Fukunaga K.: Introduction to Statistical Pattern Recognition, 2nd edn. Academic Press, New York (1990)

    MATH  Google Scholar 

  14. Yektaii, M., Bhattacharya, P.: Cumulative global distance for dimension reduction in handwritten digits database. In: International Conference on Visual Information Systems. Springer Lecture Note Series in Computer Science, vol. 4781, pp. 219–225. Springer, Berlin (2007)

  15. AT&T Laboratories Cambridge, UK: http://www.cl.cam.ac.uk/Research/DTG/attarchive:pub/data/att_faces.zip

  16. Nene, S.A., Nayar, S.K., Murase, H.: Columbia Object Image Library. Columbia University, Department of Computer Science, New York, Technical Report. CUCS-005-96 (1996)

  17. Weyrauch, B., Huang, J., Heisele, B., Blanz, V.: Component-based face recognition with 3D morphable models. In: First IEEE Workshop on Face Processing in Video, pp. 1–5 (2004)

  18. LeCun Y., Bottou L., Bengio Y., Haffner P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)

    Article  Google Scholar 

  19. Georghiades A.S., Belhumeur P.N., Kriegman D.J.: From few to many: illumination cone models for face recognition under variable lighting and pose. IEEE Trans. Pattern Anal. Mach. Intell. 23(6), 643–660 (2001)

    Article  Google Scholar 

  20. Fasel I., Bret F., Movellan J.L.: A generative framework for real-time object detection and classification. Comput. Vis. Image Underst. 98, 182–210 (2005)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Concordia University, 1515 St. Catherine West, Montreal, QC, H3G 2W1, Canada

    Mahdi Yektaii & Prabir Bhattacharya

Authors
  1. Mahdi Yektaii
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Prabir Bhattacharya
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Mahdi Yektaii.

Additional information

This work has been supported by NSERC and Canada Research Chair Foundation.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Yektaii, M., Bhattacharya, P. A criterion for measuring the separability of clusters and its applications to principal component analysis. SIViP 5, 93–104 (2011). https://doi.org/10.1007/s11760-009-0145-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-009-0145-0

Keywords