Abstract
Classification of high-dimensional data is usually not amenable to standard pattern recognition techniques owing to lack of necessary, underlying structured information of data. In this paper, we propose a new discriminant analysis based on three-step sparse residuals measurement called DA-TSSR to address this problem. Specifically, in the first stage of the proposed method, the contribution in presenting the test sample of any chosen class is respectively calculated by adding up the total contributions of all the training samples of this class, and then a certain class with the smallest contribution score is eliminated from the set of the training samples. This procedure is iteratively carried out for the set of the training samples of the remaining classes till the predefined termination condition is satisfied. The second stage of DA-TSSR seeks to represent the test sample as a linear combination of all the remaining training samples and exploits the representation ability of each training sample to determine M “nearest neighbors” for the test sample. By this means, it generates unequal number of training samples on each candidate class. The third stage of DA-TSSR again determines a new weighted sum of all unequal numbers of training samples from candidate classes, which is approximately equal to the test sample. We use the new weighted sum to perform the designing of sparse residuals grades, which can be incorporated into the typical discriminant analysis criterion. The proposed method not only has a high accuracy but also can be clearly interpreted. Experimental results conducted on the ORL, XM2VTS, FERET and AR face databases demonstrate the effectiveness of the proposed method.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Belhumeur PN, Hespanha JP, Kriegman DJ (1997) Eigenfaces vs Fisherfaces: recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19(7):711–720
Belkin M, Niyogi P (2003) Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput 15:1373–1396
Fan Z, Xu Y, Zhang D (2011) Local linear discriminant analysis framework using sample neighbors. IEEE Trans Neural Netw 22(7):1119–1132
Harandi MT, Ahmadabadi MN, Araabi BN (2009) Optimal local basis: a reinforcement learning approach for face recognition. Int J Comput Vis 81:191–204
He XF, Niyogi P (2004) Locality preserving projections (LPP). In: Proceedings of the NIPS, advances in neural information processing systems. MIT Press, Vancouver, pp 96–103
He X, Cai D, Yan S, Zhang HJ (2005) Neighborhood preserving embedding. In: Proceedings of the 10th IEEE international conference on computer vision, pp 1208–1213
Kwak KC, Pedrycz W (2005) Face recognition using a fuzzy fisherface classifier. Pattern Recognit 38(10):1717–1732
Liu ZY, Chiu KC, Xu L (2003) Improved system for object detection and star/galaxy classification via local subspace analysis. Neural Netw 16:437–451
Mary-Huard T, Robin S, Daudin JJ (2007) A penalized criterion for variable selection in classification. J Multivar Anal 98:695–705
Meytlis M, Sirovich L (2007) On the dimensionality of face space. IEEE Trans Pattern Anal Mach Intell 29(7):1262–1267
Song XN, Liu Z, Yang XB, Yang JY (2013) A parameterized fuzzy adaptive K-SVD approach for the multi-classes study of pursuit algorithms. Neurocomputing. doi:10.1016/j.neucom.2013.06.017
Song XN, Zheng YJ, Wu XJ, Yang XB, Yang JY (2010) A complete fuzzy discriminant analysis approach for face recognition. Appl Soft Comput 10:208–214
Song XN, Yang JY, Wu XJ, Yang XB (2011) An optimal symmetrical null space criterion of Fisher discriminant for feature extraction and recognition. Soft Comput 15:281–293
Sugiyama M (2007) Dimensionality reduction of multimodal labeled data by local fisher discriminant analysis. J Mach Learn Res 8:1027–1061
Vapnik V (1998) Statistical learning theory. Wiley, NewYork
Vural V, Fung G, Krishnapuram B, Dy JG, Rao B (2009) Using local dependencies within batches to improve large margin classifiers. J Mach Learn Res 10:183–206
Wright J, Yang AY, Ganesh A, Sastry SS, Ma Y (2009) Robust face recognition via sparse representation. IEEE Trans Pattern Anal Mach Intell 31(2):210–227
Xu Y, Zuo WM, Fan ZZ (2012) Supervised sparse representation method with a heuristic strategy and face recognition experiments. Neurocomputing 79:125–131
Xu Y, Zhu XJ, Li ZM, Liu GH, Liu YW, Liu H (2013) Using the original and ‘symmetrical face’ training samples to perform representation based two-step face recognition. Pattern Recognit 46:1151–1158
Xu Y, Zhu Q, Fan ZZ, Zhang D, Mi JX, Lai ZH (2013) Using the idea of the sparse representation to perform coarse-to-fine face recognition. Inf Sci 238:138–148
Yang J, Yang JY (2003) Why can LDA be performed in PCA transformed space? Pattern Recognit 36(2):563–566
Yang J, Zhang D, Frangi AF, Yang JY (2004) Two-dimensional PCA: a new approach to appearance-based face representation and recognition. IEEE Trans Pattern Anal Mach Intell 26(1):131–137
Yang J, Zhang L, Xu Y, Yang J-Y (2012) Beyond sparsity: the role of L1-optimizer in pattern classification. Pattern Recognit 45:1104–1118
Yu H, Yang J (2001) A direct LDA algorithm for high-dimensional data—with application to face recognition. Pattern Recognit 34(10):2067–2070
Zhang XX, Jia YD (2007) A linear discriminant analysis framework based on random subspace for face recognition. Pattern Recognit 40:2585–2591
Zhang L, Yang M, Feng X, Ma Y, Zhang D (2012) Collaborative representation based classification for face recognition. arXiv:1204.2358
Acknowledgments
The authors would like to thank the anonymous reviewers for their constructive advice. This work was supported by the National Science Foundation of China (Grant nos. 61233011, 61100116), the Natural Science Foundation of Jiangsu Province (Grant nos. BK2012700), the Foundation of Artificial Intelligence Key Laboratory of Sichuan Province (Grant no. 2012RZY02), the Open Project Program of the State Key Lab of CAD&CG of Zhejiang University (Grant no. A1418) and the Foundation of Key Laboratory of Intelligent Computing & Signal Processing, Ministry of Education, Anhui University.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by V. Loia.
Rights and permissions
About this article
Cite this article
Song, X., Liu, Z., Yang, J. et al. Using idea of three-step sparse residuals measurement to perform discriminant analysis. Soft Comput 19, 2355–2370 (2015). https://doi.org/10.1007/s00500-014-1428-0
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-014-1428-0