Skip to main content
Log in

A new LDA-KL combined method for feature extraction and its generalisation

  • Original Article
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

An Erratum to this article was published on 25 June 2004

Abstract

Linear discriminant analysis (LDA) is a well-known feature extraction technique. In this paper, we point out that LDA is not perfect because it only utilises the discriminatory information existing in the first-order statistical moments and ignores the information contained in the second-order statistical moments. We enhance LDA using the idea of a K-L expansion technique and develop a new LDA-KL combined method, which can make full use of both sections of discriminatory information. The proposed method is tested on the Concordia University CENPARMI handwritten numeral database. The experimental results indicate that the proposed LDA-KL method is more powerful than the existing techniques of LDA, K-L expansion and their combination: OLDA-PCA. What is more, the proposed method is further generalised to suit for feature extraction in the complex feature space and can be an effective tool for feature fusion.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1.

Similar content being viewed by others

References

  1. Fisher RA (1936) The use of multiple measurements in taxonomic problems. Annal Eug 7:178–188

    Google Scholar 

  2. Wilks SS (1962) Mathematical statistics. Wiley, New York

  3. Duda R, Hart P (1973) Pattern classification and scene analysis. Wiley, New York

  4. Swets DL, Weng J (1996) Using discriminant eigenfeatures for image retrieval. IEEE Trans Patt Anal Mach Intell 18(8):831–836

    Article  Google Scholar 

  5. Belhumeur PN (1997) Hespanha JP, Kriengman DJ (1997) Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans Patt Anal Mach Intell 19(7):711–720

    Article  Google Scholar 

  6. Liu CJ, Wechsler H (2001) A shape- and texture-based enhanced Fisher classifier for face recognition. IEEE Trans Imag Proc 10(4):598–608

    Article  MATH  Google Scholar 

  7. Foley DH, Sammon JW Jr (1975) An optimal set of discriminant vectors. IEEE Trans Comp 24(3):281–289

    MATH  Google Scholar 

  8. Sammon JW (1970) An optimal discriminant plane. IEEE Trans Comp C-19:826–829

    Google Scholar 

  9. Duchene J, Leclercq S (1988) An optimal transformation for discriminant and principal component analysis. IEEE Trans Patt Anal Mach Intell 10(6):978–983

    Article  MATH  Google Scholar 

  10. Fukunaga K (1990) Introduction to statistical pattern recognition. Academic Press, New York

  11. Fukunaga K, Short RD (1978) Nonlinear feature extraction with a general criterion. IEEE Trans Inform Theor IT-24:600–607

    Google Scholar 

  12. Fukunaga K, Ando S (1977) The optimum nonlinear features for a scatter criterion in discriminant analysis. IEEE Trans Inform Theor IT-23:453–459

    Google Scholar 

  13. Fukunaga K, Mantock JM (1983) Nonparametric discriminant analysis. IEEE Trans Patt Anal Mach Intell 5:671–678

    MATH  Google Scholar 

  14. Kittler J (1977) On the discriminant vector method of feature selection. IEEE Trans Comp 26(6):604–606

    Google Scholar 

  15. Liu K, Cheng YQ, Yang JY (1992) A generalized optimal set of discriminant vectors. Patt Recog 25(1):731–739

    Article  Google Scholar 

  16. Guo YF, Li SJ, Yang JY, Shu TT, Wu LD (2003) A generalized Foley-Sammon transform based on generalized Fisher discriminant criterion and its application to face recognition. Patt Recog Lett 24(1):147–158

    Article  MATH  Google Scholar 

  17. Jin Z, Yang JY, Hu ZS, Lou Z (2001) Face recognition based on uncorrelated discriminant transformation. Patt Recog 34(7):1405–1416

    Article  MATH  Google Scholar 

  18. Jin Z, Yang JY, Tang ZM, Hu ZS (2001) A theorem on uncorrelated optimal discriminant vectors. Patt Recog 34(10):2041-2047

    Article  MATH  Google Scholar 

  19. Yang J, Yang JY (2002) What’s wrong with the Fisher criterion? Pattern Recognition 35(11):2665–2668

    Article  MATH  Google Scholar 

  20. Mika S, Rätsch G, Weston J, Schölkopf B, Müller KR (1999) Fisher discriminant analysis with kernels. In: Hu YH, Larsen J, Wilson E, Douglas S (eds) IEEE neural networks for signal processing IX, IEEE Signal Processing Society, Los Alamitos, CA

  21. Mika S, Rätsch G, Weston J, Schölkopf B, Smola A, Müller K-R (2003) Constructing descriptive and discriminative non-linear features: Rayleigh coefficients in kernel feature spaces. IEEE Trans Patt Anal Mach Intell 25(5):623–628

    Article  Google Scholar 

  22. Baudat G, Anouar F (2000) Generalized discriminant analysis using a kernel approach. Neur Comp 12 (10):2385–2404

    Article  Google Scholar 

  23. Devijver PA, Kittler J (1982) Pattern recognition: a statistical approach. Prentice-Hall International, London

    Google Scholar 

  24. Young T, Fu K-S (1986) Handbook of pattern recognition and image processing. Academic Press, New York

  25. Webb A (1999) Statistical pattern recognition. Arnold, London

  26. Yang J, Yang JY (2002) Generalized K-L transformed based combined feature extraction. Patt Recog 35(1):295–297

    Article  MATH  Google Scholar 

  27. Yang J, Yang JY, Zhang D, Lu JF (2003) Feature fusion: parallel strategy vs. serial strategy. Patt Recog 36(6):1369–1381

    Article  MATH  Google Scholar 

  28. Yang J, Yang JY (2001) An optimal K-L transform method for feature extraction. SPIE Proc Imag Extract Seg Recog 4550:239–244

    Article  Google Scholar 

  29. Ding XR, Cai MK (1995) Matrix theory in engineering. Tianjin University Press, Tian-jin, China

  30. Watanable S, Takvasa N (1973) Subspace methods in pattern recognition. In: Proceedings of the 1st International Joint Conference on Pattern Recognition, Washington, DC, 1973

  31. Hamamoto Y, Uchimura S, Watanabe M, Yasuda T, Tomita S (1996) Recognition of handwritten numerals using Gabor features, In: Proceedings of the 13th International Conference on Pattern Recognition, Vienna, Austria, August 1996

  32. Liao SX, Pawlak M (1996) On image analysis by moments. IEEE Trans Patt Anal Mach Intell 18(3):254–266

    Article  MATH  Google Scholar 

  33. Scholkopf B, Smola A, Muller KR (1998) Nonlinear component analysis as a kernel eigenvalue problem. Neur Comp 10(5):1299–1319

    Article  Google Scholar 

  34. Yang J, Frangi AF, Yang JY (2004) A new Kernel Fisher Discriminant Algorithm with Application to Face Recognition. Neurocomputing, vol.56, 415–421

Download references

Acknowledgements

We would like to thank K. Liu and C. Y. Suen for their provision of the CENPARMI handwritten numerical database. In addition, we would like to thank the anonymous reviewers for their constructive advice.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jian Yang.

Additional information

An erratum to this article can be found at http://dx.doi.org/10.1007/s10044-004-0221-6

Rights and permissions

Reprints and permissions

About this article

Cite this article

Yang, J., Ye, H. & Zhang, D. A new LDA-KL combined method for feature extraction and its generalisation. Pattern Anal Applic 7, 40–50 (2004). https://doi.org/10.1007/s10044-004-0205-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-004-0205-6

Keywords

Navigation