Abstract
Kernel discriminant analysis (KDA) is a widely used tool in feature extraction community. However, for high-dimensional multi-class tasks such as face recognition, traditional KDA algorithms have the limitation that the Fisher criterion is nonoptimal with respect to classification rate. Moreover, they suffer from the small sample size problem. This paper presents a variant of KDA called kernel-based improved discriminant analysis (KIDA), which can effectively deal with the above two problems. In the proposed framework, origin samples are projected firstly into a feature space by an implicit nonlinear mapping. After reconstructing between-class scatter matrix in the feature space by weighted schemes, the kernel method is used to obtain a modified Fisher criterion directly related to classification error. Finally, simultaneous diagonalization technique is employed to find lower-dimensional nonlinear features with significant discriminant power. Experiments on face recognition task show that the proposed method is superior to the traditional KDA and LDA.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Baudat G, Anouar F (2000) Generalized discriminant analysis using a kernel approach. Neural Computat 12:2385–2404
Belhumeur P, Hespanha J, Kriegman D (1997) Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19:711–720
Dai G, Qian YT, Jia S (2004) A kernel fractional-step nonlinear discriminant analysis for pattern recognition. In: Proceedings of the 18th international conference on pattern recognition, pp.431–434
Dai G, Yeung DY, Qian YT (2007) Face recognition using a kernel fractional-step discriminant analysis algorithm. Pattern Recogn 40:229–243
Etemad K, Chellappa R (1997) Discriminant analysis for recognition of human face images. J Opt Soc Am A Opt Imag Sci Vis 14:1724–1733
Fukunaga K (1990) Introduction to statistical pattern recognition. Academic Press, Boston
Hong ZQ, Yang JY (1991) Optimal discriminant plane for a small number of samples and design method of classifier on the plane. Pattern Recogn 24:317–324
Jain AK, Duin R, Mao J (2000) Statistical pattern recognition: a review. IEEE Trans Pattern Anal Mach Intell 22:4–37
Liu C, Wechsler H (2000) Robust coding schemes for indexing and retrieval from large face databases. IEEE Trans. Image Process 9:132–137
Loog M, Duin RPW, Haeb-Umbach R (2001) Multiclass linear dimension reduction by weighted pairwise Fisher criteria. IEEE Trans Pattern Anal Mach Intell 23:762–766
Lotlikar R, Kothari R (2000) Fractional-step dimension reduction. IEEE Trans Pattern Anal Mach Intell 22:623–627
Lu J, Plataniotis KN, Venetsanopoulos AN (2003a) Face recognition using kernel direct discriminant analysis algorithms. IEEE Trans on Neural Netw 14:117–126
Lu J, Plataniotis KN, Venetsanopoulos AN (2003b) Face recognition using LDA-based algorithms. IEEE Trans Neural Netw 14(1):195–200
Lu J, Plataniotis KN, Venetsanopoulos AN (2005) Regularization studies of linear discriminant analysis in small sample size scenarios with application to face recognition. Pattern Recogn Lett 26:181–191
Ma J (2003) Function replacement vs. kernel trick. Neurocomputing 50:479–483
Martnez AM, Kak AC (2001) PCA versus LDA. IEEE Trans Pattern Anal Mach Intell 23:228–233
Mika S, Ratsch G, Weston J, Scholkopf B, Muller KR (1999) Fisher Discriminant Analysis with Kernels. In: Proceedings of IEEE international workshop neural networks for signal processing IX, pp 41–48
Mika S, Ratsch G, Weston J, Scholkopf B, Smola A, Muller KR (2003) Constructing descriptive and discriminative nonlinear features: Rayleigh coefficients in kernel feature spaces. IEEE Trans Pattern Anal Mach Intell 25:623–628
Scholkopf B, Smola A, Moller KR (1999) Nonlinear component analysis as a kernel eigenvalue problem. Neural Computat 10:1299–1319
Swets DL, Weng J (1996) Using discriminant eigenfeatures for image retrieval. IEEE Trans Pattern Anal Mach Intell 18:831–836
Yang MH (2002) Kernel eigenfaces vs. kernel fisherfaces: face recognition using kernel methods. In: Proceedings of fifth IEEE international conference automatic face and gesture recognition. IEEE Press, New York, pp 215–220
Yang J, Jin Z, Yang J, Zhang D (2004) The essence of kernel Fisher discriminant: KPCA plus LDA. Pattern Recogn 37:2097–2100
Yang J, Frangi AF, Yang JY, Zhang D (2005) KPCA plus LDA: a complete kernel Fisher discriminant framework for feature extraction and recognition. IEEE Trans Pattern Anal Mach Intell 27:230–244
Yu H, Yang J (2001) A direct LDA algorithm for high-dimensional data with application to face recognition. Pattern Recognit 34:2067–2070
Vapnik V (1998) Statistical learning theory. Wiley, New York
Zelnik-Manor L, Perona P (2004) Self-tuning spectral clustering. Adv Neural Inform Process Syst 17:1601–1608
Zhou D, Yang X, Peng N (2006) Improved-LDA based face recognition using both facial global and local information. Pattern Recogn Lett 27:537–543
Acknowledgements
This work was supported by China Postdoctoral Science Foundation under the grant of 20060390286, and Postdoctoral Science Foundation of Jiangsu Province of China under the grant of 0601006B. The authors would also like to thank the anonymous reviewers for their critical and constructive comments and suggestions.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Zhou, D., Tang, Z. Kernel-based improved discriminant analysis and its application to face recognition. Soft Comput 14, 103–111 (2010). https://doi.org/10.1007/s00500-009-0443-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-009-0443-z