Abstract
Linear discriminant analysis (LDA) is a well-known feature extraction technique. In this paper, we point out that LDA is not perfect because it only utilises the discriminatory information existing in the first-order statistical moments and ignores the information contained in the second-order statistical moments. We enhance LDA using the idea of a K-L expansion technique and develop a new LDA-KL combined method, which can make full use of both sections of discriminatory information. The proposed method is tested on the Concordia University CENPARMI handwritten numeral database. The experimental results indicate that the proposed LDA-KL method is more powerful than the existing techniques of LDA, K-L expansion and their combination: OLDA-PCA. What is more, the proposed method is further generalised to suit for feature extraction in the complex feature space and can be an effective tool for feature fusion.

Similar content being viewed by others
References
Fisher RA (1936) The use of multiple measurements in taxonomic problems. Annal Eug 7:178–188
Wilks SS (1962) Mathematical statistics. Wiley, New York
Duda R, Hart P (1973) Pattern classification and scene analysis. Wiley, New York
Swets DL, Weng J (1996) Using discriminant eigenfeatures for image retrieval. IEEE Trans Patt Anal Mach Intell 18(8):831–836
Belhumeur PN (1997) Hespanha JP, Kriengman DJ (1997) Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans Patt Anal Mach Intell 19(7):711–720
Liu CJ, Wechsler H (2001) A shape- and texture-based enhanced Fisher classifier for face recognition. IEEE Trans Imag Proc 10(4):598–608
Foley DH, Sammon JW Jr (1975) An optimal set of discriminant vectors. IEEE Trans Comp 24(3):281–289
Sammon JW (1970) An optimal discriminant plane. IEEE Trans Comp C-19:826–829
Duchene J, Leclercq S (1988) An optimal transformation for discriminant and principal component analysis. IEEE Trans Patt Anal Mach Intell 10(6):978–983
Fukunaga K (1990) Introduction to statistical pattern recognition. Academic Press, New York
Fukunaga K, Short RD (1978) Nonlinear feature extraction with a general criterion. IEEE Trans Inform Theor IT-24:600–607
Fukunaga K, Ando S (1977) The optimum nonlinear features for a scatter criterion in discriminant analysis. IEEE Trans Inform Theor IT-23:453–459
Fukunaga K, Mantock JM (1983) Nonparametric discriminant analysis. IEEE Trans Patt Anal Mach Intell 5:671–678
Kittler J (1977) On the discriminant vector method of feature selection. IEEE Trans Comp 26(6):604–606
Liu K, Cheng YQ, Yang JY (1992) A generalized optimal set of discriminant vectors. Patt Recog 25(1):731–739
Guo YF, Li SJ, Yang JY, Shu TT, Wu LD (2003) A generalized Foley-Sammon transform based on generalized Fisher discriminant criterion and its application to face recognition. Patt Recog Lett 24(1):147–158
Jin Z, Yang JY, Hu ZS, Lou Z (2001) Face recognition based on uncorrelated discriminant transformation. Patt Recog 34(7):1405–1416
Jin Z, Yang JY, Tang ZM, Hu ZS (2001) A theorem on uncorrelated optimal discriminant vectors. Patt Recog 34(10):2041-2047
Yang J, Yang JY (2002) What’s wrong with the Fisher criterion? Pattern Recognition 35(11):2665–2668
Mika S, Rätsch G, Weston J, Schölkopf B, Müller KR (1999) Fisher discriminant analysis with kernels. In: Hu YH, Larsen J, Wilson E, Douglas S (eds) IEEE neural networks for signal processing IX, IEEE Signal Processing Society, Los Alamitos, CA
Mika S, Rätsch G, Weston J, Schölkopf B, Smola A, Müller K-R (2003) Constructing descriptive and discriminative non-linear features: Rayleigh coefficients in kernel feature spaces. IEEE Trans Patt Anal Mach Intell 25(5):623–628
Baudat G, Anouar F (2000) Generalized discriminant analysis using a kernel approach. Neur Comp 12 (10):2385–2404
Devijver PA, Kittler J (1982) Pattern recognition: a statistical approach. Prentice-Hall International, London
Young T, Fu K-S (1986) Handbook of pattern recognition and image processing. Academic Press, New York
Webb A (1999) Statistical pattern recognition. Arnold, London
Yang J, Yang JY (2002) Generalized K-L transformed based combined feature extraction. Patt Recog 35(1):295–297
Yang J, Yang JY, Zhang D, Lu JF (2003) Feature fusion: parallel strategy vs. serial strategy. Patt Recog 36(6):1369–1381
Yang J, Yang JY (2001) An optimal K-L transform method for feature extraction. SPIE Proc Imag Extract Seg Recog 4550:239–244
Ding XR, Cai MK (1995) Matrix theory in engineering. Tianjin University Press, Tian-jin, China
Watanable S, Takvasa N (1973) Subspace methods in pattern recognition. In: Proceedings of the 1st International Joint Conference on Pattern Recognition, Washington, DC, 1973
Hamamoto Y, Uchimura S, Watanabe M, Yasuda T, Tomita S (1996) Recognition of handwritten numerals using Gabor features, In: Proceedings of the 13th International Conference on Pattern Recognition, Vienna, Austria, August 1996
Liao SX, Pawlak M (1996) On image analysis by moments. IEEE Trans Patt Anal Mach Intell 18(3):254–266
Scholkopf B, Smola A, Muller KR (1998) Nonlinear component analysis as a kernel eigenvalue problem. Neur Comp 10(5):1299–1319
Yang J, Frangi AF, Yang JY (2004) A new Kernel Fisher Discriminant Algorithm with Application to Face Recognition. Neurocomputing, vol.56, 415–421
Acknowledgements
We would like to thank K. Liu and C. Y. Suen for their provision of the CENPARMI handwritten numerical database. In addition, we would like to thank the anonymous reviewers for their constructive advice.
Author information
Authors and Affiliations
Corresponding author
Additional information
An erratum to this article can be found at http://dx.doi.org/10.1007/s10044-004-0221-6
Rights and permissions
About this article
Cite this article
Yang, J., Ye, H. & Zhang, D. A new LDA-KL combined method for feature extraction and its generalisation. Pattern Anal Applic 7, 40–50 (2004). https://doi.org/10.1007/s10044-004-0205-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10044-004-0205-6