Abstract
Linear discriminant analysis (LDA) is one of the most effective feature extraction methods in statistical pattern recognition, which extracts the discriminant features by maximizing the so-called Fisher’s criterion that is defined as the ratio of between-class scatter matrix to within-class scatter matrix. However, classification of high-dimensional statistical data is usually not amenable to standard pattern recognition techniques because of an underlying small sample size (SSS) problem. A popular approach to the SSS problem is the removal of non-informative features via subspace-based decomposition techniques. Motivated by this viewpoint, many elaborate subspace decomposition methods including Fisherface, direct LDA (D-LDA), complete PCA plus LDA (C-LDA), random discriminant analysis (RDA) and multilinear discriminant analysis (MDA), etc., have been developed, especially in the context of face recognition. Nevertheless, how to search a set of complete optimal subspaces for discriminant analysis is still a hot topic of research in area of LDA. In this paper, we propose a novel discriminant criterion, called optimal symmetrical null space (OSNS) criterion that can be used to compute the Fisher’s maximal discriminant criterion combined with the minimal one. Meanwhile, by the reformed criterion, the complete symmetrical subspaces based on the within-class and between-class scatter matrices are constructed, respectively. Different from the traditional subspace learning criterion that derives only one principal subspace, in our approach two null subspaces and their orthogonal complements were all obtained through the optimization of OSNS criterion. Therefore, the algorithm based on OSNS has the potential to outperform the traditional LDA algorithms, especially in the cases of small sample size. Experimental results conducted on the ORL, FERET, XM2VTS and NUST603 face image databases demonstrate the effectiveness of the proposed method.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Belhumeur PN, Hespanha JP, Kriegman DJ (1997) Eigenfaces vs. Fisherfaces: recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19(7):711–720
Burges C (1998) A tutorial on support vector machines for pattern recognition. Data Min Knowl Disc 2(2):121–167
Chen LF, Liao HY, Lin JC et al (2000) A new LDA-based face recognition system which can solve the small sample size problem. Pattern Recognit 33(10):1713–1726
Fukunnaga K (1999) Introduction to statistical pattern recognition, 2nd edn. Academic Press, New York
Kwak N, Oh J (2009) Feature extraction for one-class classification problems: enhancements to biased discriminant analysis. Pattern Recognit 42:17–26
Liang ZZ, Li YF, Shi PF (2008) A note on two-dimensional linear discriminant analysis. Pattern Recognit Lett 29:2122–2128
Liu C, Wechsler H (2000) Robust coding schemes for indexing and retrieval from large face databases. IEEE Trans Image Process 9(1):132–137
Liu C, Wechsler H (2002) Gabor feature based classification using the enhanced Fisher linear discriminant model for face recognition. IEEE Trans Image Process 11(4):467–476
ORL database (1992) The ORL face database at the AT&T (Olivetti) research laboratory. Available from http://www.uk.research.att.com/facedatabase.html
Phillips PJ, Moon H, Rizvi SA et al (2000) The FERET evaluation methodology for face-recognition algorithms. IEEE Trans Pattern Anal Mach Intell 22(10):1090–1104
Rueda L, Herrera M (2008) Linear dimensionality reduction by maximizing the Chernoff distance in the transformed space. Pattern Recognit 41:3138–3152
Skurichina M, Duin RPW (2002) Bagging, boosting, and the random subspace method for linear classifiers. Pattern Anal Appl 5:121–135
Turk M, Pentland A (1991) Face recognition using eigenfaces. In: Proceeding of IEEE conference on computer vision and pattern recognition, Maui, HI, USA, June 1991, pp 586–591
Wang X, Tang X (2005) Subspace analysis using random mixture models. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 574–580
Wang Y, Tan T, Zhu Y (2000) Face verification based on singular value decomposition and radial basis function neural network. In: Proceeding of 4th Asian conference on computer vision (ACCV), Taipei, Taiwan, 2000, pp 432–436
Wang HH, Zhou Y, Ge XL et al (2007) Subspace evolution analysis for face representation and recognition. Pattern Recognit 40:335–338
XM2VTS database (2000) Available from http://www.ee.surrey.ac.uk/CVSSP/xm2vtsdb/
Yan Y, Zhang YJ (2008) A novel class-dependence feature analysis method for face recognition. Pattern Recognit Lett 29:1907–1914
Yan SC, Xu D, Yang Q et al (2007) Multilinear discriminant analysis for face recognition. IEEE Trans Image Process 16(1):212–220
Yang J, Yang JY (2001) Optimal FLD algorithm for facial feature extraction. In: SPIE Proceedings of the intelligent robots and computer vision XX: Algorithms, Techniques, and Active Vision, pp 438–444
Yang J, Yang JY (2002) From image vector to matrix: a straightforward image projection technique-IMPCA vs. PCA. Pattern Recognit 35(9):1997–1999
Yang J, Yang JY (2003) Why can LDA be performed in PCA transformed space? Pattern Recognit 36(2):563–566
Yang J, Zhang D, Frangi AF et al (2004) Two-dimensional PCA: a new approach to appearance-based face representation and recognition. IEEE Trans Pattern Anal Mach Intell 26(1):131–137
Yang J, Frangi AF, Yang JY et al (2005) KPCA plus LDA: a complete kernel Fisher discriminant framework for feature extraction and recognition. IEEE Trans Pattern Anal Mach Intell 27(2):230–244
You QB, Zheng NN, Du SY et al (2007) Neighborhood discriminant projection for face recognition. Pattern Recognit Lett 28:1156–1163
Yu H, Yang J (2001) A direct LDA algorithm for high-dimensional data—with application to face recognition. Pattern Recognit 34(11):2067–2070
Yu J, Tian Q, Rui T et al (2007) Integrating discriminant and descriptive information for dimension reduction and classification. IEEE Trans Circuits Syst Video Technol 17(3):372–377
Zhang XX, Jia YD (2007) A linear discriminant analysis framework based on random subspace for face recognition. Pattern Recognit 40:2585–2591
Zhang L, Li SZ, Qu ZY et al (2004) Boosting local feature based classifiers for face recognition. In: Proceedings of the IEEE conference on CVPR workshop on face processing in video (FPIV 04), pp 87–92
Zhao HT, Yuen PC, Kwok JT et al (2004) Incremental PCA based face recognition. In: Proceedings of the 8th international conference on control, automation, robotics and vision (ICARCV), Kunming, China, Dec 2004, pp 687–691
Zhou DK, Yang X, Peng NS et al (2006) Improved-LDA based face recognition using both facial global and local information. Pattern Recognit Lett 27:536–543
Zhuang XS, Dai DQ (2005) Inverse Fisher discriminant criteria for small sample size problem and its application to face recognition. Pattern Recognit 38:2192–2194
Acknowledgments
The authors would like to thank the anonymous reviewers for their constructive advice. This work was supported by the National Science Foundation of China (nos. 60632050, 60572034 and 60973094), the Program for New Century Excellent Talents in University of China (Grant no. NCET-06-0487), the Natural Science Foundation of the Jiangsu Higher Education Institutions of China (Grant no. 10KJB520006) and the Science Foundation of Jiangsu University of Science and Technology.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Song, X., Yang, J., Wu, X. et al. An optimal symmetrical null space criterion of Fisher discriminant for feature extraction and recognition. Soft Comput 15, 281–293 (2011). https://doi.org/10.1007/s00500-010-0682-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-010-0682-z