Skip to main content
SpringerLink
Log in

From NLDA to LDA/GSVD: a modified NLDA algorithm

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Linear discriminant analysis (LDA) often encounters small sample size (SSS) problem for high-dimensional data. Null space linear discriminant analysis (NLDA) and linear discriminant analysis based on generalized singular value decomposition (LDA/GSVD) are two popular methods that can solve SSS problem of LDA. In this paper, we present the relation between NLDA and LDA/GSVD under a condition and at the same time propose a modified NLDA (MNLDA) algorithm which has the same discriminating power as LDA/GSVD and is more efficient. In addition, we compare the discriminating capability of NLDA and MNLDA and present our interpretation about this. Experimental results on ORL, FERET, Yale face databases, and the PolyU FKP database support our viewpoints.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Tenenbaum JB, de Silva V, Langford JC (2000) A global geometric framework for nonlinear dimensionality reduction. Science 290:2319–2323

    Article  Google Scholar 

  2. Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290:2323–2326

    Article  Google Scholar 

  3. Belkin M, Niyogi P (2003) Laplacian Eigen maps for dimensionality reduction and data representation. Neural Comput 15(6):1373–1396

    Article  MATH  Google Scholar 

  4. He X, Yan S, Hu Y, Niyogi P, Zhang HJ (2005) Face recognition using Laplacian faces. IEEE Transact Pattern Anal Mach Intell 27(3):328–340

    Article  Google Scholar 

  5. Fu Y, Huang TS (2005) Locally linear embedded Eigen space analysis. University Illinois Urbana-Champaign, Urbana, IL. Technical Report IFP-TR, ECE

  6. Yang J, Zhang D, Yang JY, Niu B (2007) Globally maximizing, locally minimizing: unsupervised discriminant projection with applications to face and palm biometrics. IEEE Transact Pattern Anal Mach Intell 29(4):650–664

    Article  Google Scholar 

  7. Yan S, Xu D, Zhang B, Zhang HJ, Yang Q, Lin S (2007) Graph embedding, extensions: a general framework for dimensionality reduction. IEEE Transact Pattern Anal Mach Intell 29(1):40–51

    Article  Google Scholar 

  8. Chen HT, Chang HW, Liu TL (2005) Local discriminant embedding and its variants, In: Proceedings of CVPR 2005

  9. Fu Y, Liu M, Huang TS (2007) Conformal embedding analysis with local graph modeling on the unit hyper sphere, In: Proceedings of CVPR 2007

  10. Cai D, He X, Zhou K, Han J, Bao H (2007) Locality sensitive discriminant analysis, In: Proceedings of IJCAI

  11. Yang WK, Sun CY, Zhang L (2011) A multi-manifold discriminant analysis method for image feature extraction. Pattern Recogn 44(8):1649–1657

    Article  MATH  Google Scholar 

  12. Yang WK, Wang JG, Ren MW, Yang JY, Zhang L, Liu GH (2009) Feature extraction based on Laplacian bidirectional maximum margin criterion. Pattern Recogn 42(11):2327–2334

    Article  MATH  Google Scholar 

  13. Duda RO, Hart PE, Stork DG (2001) Pattern classification. Wiley, NY

    MATH  Google Scholar 

  14. Belhumeur P, Hespanha J, Kriegman D (1997) Eigen faces versus fisher faces: recognition using class specific linear projection. IEEE Transact Pattern Anal Mach Intell 19(7):711–720

    Article  Google Scholar 

  15. Zuo W, Zhang D, Yang J, Wang K (2006) BDPCA plus LDA: a novel fast feature extraction technique for face recognition. IEEE Transact Syst, Man, Cybern-Part B: Cybern 36(4):946–953

    Article  Google Scholar 

  16. Jing XY, Zhang D, Tang YY (2004) An improved LDA approach. IEEE Transact Syst, Man, Cybern-Part B: Cybern 34(5):1942–1951

    Article  Google Scholar 

  17. Friedman JH (1989) Regularized discriminant analysis. J Am Stat Assoc 84(405):165–175

    Article  Google Scholar 

  18. Chen L-F, Liao H-YM, Ko M-T, Lin J-C, Yu G-J (2000) A new LDA-based face recognition system which can solve the small sample size problem. Pattern Recogn 33(10):1713–1726

    Article  Google Scholar 

  19. Ye J, Xiong T (2006) Computational and theoretical analysis of null space and orthogonal linear discriminant analysis. J Mach Learn Res 7:1183–1204

    MathSciNet  MATH  Google Scholar 

  20. Yu H, Yang J (2001) A direct LDA algorithm for high dimensional data with application to face recognition. Pattern Recogn 34(10):2067–2070

    Article  MATH  Google Scholar 

  21. Yang J, Yang JY (2003) Why can LDA be performed in PCA transformed space? Pattern Recogn 36(2):563–566

    Article  Google Scholar 

  22. Yang J, Alejandro FF, Yang J, Zhang D, Jin Z (2005) KPCA plus LDA: a complete kernel fisher discriminant framework for feature extraction and recognition. IEEE Transact Pattern Anal Mach Intell 27(2):230–244

    Article  Google Scholar 

  23. Zhuang XS, Dai DQ (2005) Inverse fisher discriminate criteria for small sample size problem and its application to face recognition. Pattern Recogn 38(11):2192–2194

    Article  Google Scholar 

  24. Howland P, Jeon M, Park H (2003) Structure preserving dimension reduction for clustered text data based on the generalized singular value decomposition. SIAM J Matrix Anal Appl 25(1):165–179

    Article  MathSciNet  MATH  Google Scholar 

  25. Howland P, Park H (2004) Generalizing discriminant analysis using the generalized singular value decomposition. IEEE Transact Pattern Anal Mach Intell 26(8):995–1006

    Article  Google Scholar 

  26. Turk MA, Pentland AP (1991) Face recognition using Eigen faces. In: Proceedings of CVPR 1991

  27. Huang R, Liu Q, Lu H, Ma S (2002) Solving the small sample size problem of LDA. In: Proceedings of ICPR 2002

  28. Paige CC, Saunders MA (1981) Towards a generalized singular value decomposition. SIAM J Num Anal 18(3):398–405

    Article  MathSciNet  MATH  Google Scholar 

  29. http://www.cvc.yale.edu/projects/yalefaces/yalefaces.html

  30. Phillips PJ, Moon H, Rizvi SA, Rauss PJ (2000) The FERET evaluation methodology for face recognition algorithms. IEEE Transact Pattern Anal Mach Intell 22(10):1090–1104

    Article  Google Scholar 

  31. Phillips PJ (2006) The facial recognition technology (FERET) database. http://www.itl.nist.gov/iad/humanid/feret/feret_master.html

  32. Phillips PJ, Wechsler H, Huang J, Rauss P (1998) The FERET database and evaluation procedure for face recognition algorithms. Image Vis Comput 16(5):295–306

    Article  Google Scholar 

  33. http://www.cam-orl.co.uk

  34. Wang X, Tang X (2004) A unified framework for subspace face recognition. IEEE Transact Pattern Anal Mach Intell 26(9):1222–1228

    Article  MathSciNet  Google Scholar 

  35. Zhang L, Zhang L, Zhang D, Zhu HL (2010) Online finger-knuckle-print verification for personal authentications. Pattern Recogn 43(7):2560–2571

    Article  MATH  Google Scholar 

Download references

Acknowledgements

This work is supported by the National Science Foundation of China under Grants No. 60632050, No. 60873151, No. 60973098.

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Nanjing University of Science and Technology, Nanjing, 210094, China

    Jun Yin & Zhong Jin

Authors
  1. Jun Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhong Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jun Yin.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Yin, J., Jin, Z. From NLDA to LDA/GSVD: a modified NLDA algorithm. Neural Comput & Applic 21, 1575–1583 (2012). https://doi.org/10.1007/s00521-011-0728-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-011-0728-x

Keywords

Search

Navigation