Skip to main content
SpringerLink
Log in

Polynomial linear discriminant analysis

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

The traditional linear discriminant analysis (LDA) is a classical dimensionality reduction method. But there are two problems with LDA. One is the small-sample-size (SSS) problem, and the other is the classification ability of LDA is not very strong. In this paper, by studying several common methods of LDA, an implicit rule of the criterion of LDA is found, and then a general framework of LDA combined with matrix function is presented. Based on this framework, the polynomials are used to reconstruct the LDA criterion, and then a new method called polynomial linear discriminant analysis (PLDA) is proposed. The proposed PLDA method has two contributions: first, it addresses the small-sample-size problem of LDA, and second, it enhances the distance of the between-class sample through polynomial function mapping, improving the classification ability. Experimental results on ORL, FERET, AR, and Coil datasets show the superiority of PLDA over existing variations of LDA.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Availability of supporting data

The datasets used or analysed during the current study are available from the corresponding author on reasonable request.

References

  1. Turk M (1991) Pentland. Eigenfaces for recognition. K. Cogn. Neurosc 4, 72–86

  2. Fisher RA (1936) The use of multiple measurements in taxonomic problems. Ann Eugen 7(2):179–188

    Article  Google Scholar 

  3. Hart PE, Stork DG, Duda RO (2000) Pattern classification. Hoboken, New York

    Google Scholar 

  4. Sharma A, Paliwal KK (2015) Linear discriminant analysis for the small sample size problem: an overview. Int J Mach Learn Cybern 6(3):443–454. https://doi.org/10.1007/s13042-013-0226-9

    Article  Google Scholar 

  5. Belhumeur PN, Hespanha JP (1997) Kriegman DJ Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19(7):711–720

    Article  Google Scholar 

  6. Lu J, Plataniotis KN, Venetsanopoulos AN (2005) Regularization studies of linear discriminant analysis in small sample size scenarios with application to face recognition. Pattern Recogn Lett 26(2):181–191

    Article  Google Scholar 

  7. Zhang T, Fang B, Tang YY, Shang Z, Xu B (2009) Generalized discriminant analysis: A matrix exponential approach. IEEE Trans Syst Man Cybern Part B (Cybern) 40(1), pp 186–197

  8. Murthy KR, Ghosh A (2017) Noisy-free length discriminant analysis with cosine hyperbolic framework for dimensionality reduction. Expert Syst Appl 81:88–107

    Article  Google Scholar 

  9. Chen L-F, Liao H-YM, Ko M-T, Lin J-C, Yu G-J (2000) A new LDA-based face recognition system which can solve the small sample size problem. Pattern Recogn 33(10):1713–1726

    Article  Google Scholar 

  10. Yu H, Yang J (2001) A direct LDA algorithm for high-dimensional data-with application to face recognition. Pattern Recogn 34(10):2067–2070

    Article  Google Scholar 

  11. Lu J, Plataniotis KN, Venetsanopoulos AN (2003) Face recognition using LDA-based algorithms. IEEE Trans Neural Netw 14(1):195–200

    Article  Google Scholar 

  12. Lotlikar R, Kothari R (2000) Fractional-step dimensionality reduction. IEEE Trans Pattern Anal Mach Intell 22(6):623–627

    Article  Google Scholar 

  13. Sharma A, Paliwal KK (2012) A two-stage linear discriminant analysis for face-recognition. Pattern Recogn Lett 33(9):1157–1162

    Article  Google Scholar 

  14. Li H, Jiang T, Zhang K (2003) Efficient and robust feature extraction by maximum margin criterion. Adv Neural Inf Process Syst 16

  15. Liu S, Feng L, Qiao H (2014) Scatter balance: an angle-based supervised dimensionality reduction. IEEE Trans Neural Netw Learn Syst 26(2):277–289

    MathSciNet  Google Scholar 

  16. Zhao H, Wang Z, Nie F (2018) A new formulation of linear discriminant analysis for robust dimensionality reduction. IEEE Trans Knowl Data Eng 31(4):629–640

    Article  Google Scholar 

  17. Zhong F, Zhang J (2013) Linear discriminant analysis based on L1-norm maximization. IEEE Trans Image Process 22(8):3018–3027

    Article  MathSciNet  Google Scholar 

  18. Ye Q, Fu L, Zhang Z, Zhao H, Naiem M (2018) Lp-and ls-norm distance based robust linear discriminant analysis. Neural Netw 105:393–404

    Article  Google Scholar 

  19. Ye Q, Li Z, Fu L, Zhang Z, Yang W, Yang G (2019) Nonpeaked discriminant analysis for data representation. IEEE Trans Neural Netw Learn Syst 30(12):3818–3832

    Article  MathSciNet  Google Scholar 

  20. Mika S, Ratsch G, Weston J, Scholkopf B, Mullers K-R (1999) Fisher discriminant analysis with kernels. In: Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (cat. No. 98th8468), pp 41–48. IEEE

  21. Baudat G, Anouar F (2000) Generalized discriminant analysis using a kernel approach. Neural Comput 12(10):2385–2404

    Article  Google Scholar 

  22. Schölkopf B, Smola A, Müller K-R (1998) Nonlinear component analysis as a kernel eigenvalue problem. Neural Comput 10(5):1299–1319

    Article  Google Scholar 

  23. Schölkopf B, Smola A, Müller K-R (1997) Kernel principal component analysis. In: International Conference on Artificial Neural Networks, pp 583–588. Springer

  24. He X, Niyogi P (2003) Locality preserving projections. Adv Neural Inf Process Syst 16

  25. Zhang D, He J, Zhao Y, Luo Z, Du M (2014) Global plus local: a complete framework for feature extraction and recognition. Pattern Recogn 47(3):1433–1442

    Article  Google Scholar 

  26. Shu X, Gao Y, Lu H (2012) Efficient linear discriminant analysis with locality preserving for face recognition. Pattern Recogn 45(5):1892–1898

    Article  Google Scholar 

  27. Li Y, Liu B, Yu Y, Li H, Sun J, Cui J (2021) 3E-LDA: three enhancements to linear discriminant analysis. ACM Trans Knowl Discov Data (TKDD) 15(4):1–20

    Article  Google Scholar 

  28. Zhao M, Zhang Z, Chow TW (2012) Trace ratio criterion based generalized discriminative learning for semi-supervised dimensionality reduction. Pattern Recogn 45(4):1482–1499

    Article  Google Scholar 

  29. Zhao M, Zhang Z, Chow TW, Li B (2014) A general soft label based linear discriminant analysis for semi-supervised dimensionality reduction. Neural Netw 55:83–97

    Article  Google Scholar 

  30. Zhao M, Zhang Z, Chow TW, Li B (2014) Soft label based linear discriminant analysis for image recognition and retrieval. Comput Vis Image Underst 121:86–99

    Article  Google Scholar 

  31. Veeramani K, Jaganathan S (2014) Intensified regularized discriminant analysis technique. In: International Conference on Recent Advances and Innovations in Engineering (ICRAIE-2014), pp 1–6 . IEEE

  32. Golub GH, Van Loan CF (2013) Matrix computations. JHU press, Maryland

    Book  Google Scholar 

  33. Rudin W (1976) Principles of mathematical analysis, vol 3. McGraw-hill, New York

    Google Scholar 

  34. Cai D, He X, Han J (2007) SRDA: an efficient algorithm for large-scale discriminant analysis. IEEE Trans Knowl Data Eng 20(1):1–12

    Google Scholar 

  35. Gao J, Li L (2019) A robust geometric mean-based subspace discriminant analysis feature extraction approach for image set classification. Optik 199:163368

    Article  Google Scholar 

  36. Li D-L, Prasad M, Hsu S-C, Hong C-T, Lin C-T (2012) Face recognition using nonparametric-weighted fisherfaces. EURASIP J Adv Signal Process 2012(1):1–11

    Article  Google Scholar 

  37. Hu L, Zhang W (2020) Orthogonal neighborhood preserving discriminant analysis with patch embedding for face recognition. Pattern Recogn 106:107450

    Article  Google Scholar 

  38. He Z, Wu M, Zhao X, Zhang S, Tan J (2021) Representative null space LDA for discriminative dimensionality reduction. Pattern Recogn 111:107664

    Article  Google Scholar 

  39. Sun K, Zhang J, Yong H, Liu J (2019) Fpcanet: fisher discrimination for principal component analysis network. Knowl-Based Syst 166:108–117

    Article  Google Scholar 

  40. Kewei T, Jun Z, Changsheng Z, Lijun W, Yun Z, Wei J (2021) Unsupervised, supervised and semi-supervised dimensionality reduction by low-rank regression analysis. Chin J Electron 30(4):603–610

    Article  Google Scholar 

  41. Yang R, Meng X, Feng L (2020) Scatter balance based semi-supervised dimensional reduction. In: 2020 IEEE International Conference on Smart Internet of Things (SmartIoT), pp 168–175 . IEEE

  42. Olivetti & Oracle Research Laboratory (1994) The Olivenfi d Oracle Research Laboratory Face Database of Faces. https://wwwcam-orl.co.uk/facedatabase.html

  43. Phillips PJ, Moon H, Rizvi SA, Rauss PJ (2000) The FERET evaluation methodology for face-recognition algorithms. IEEE Trans Pattern Anal Mach Intell 22(10):1090–1104. https://doi.org/10.1109/34.879790

    Article  Google Scholar 

  44. The AR Face Database, CVC Technical Report, 24 (1998). https://portalrecerca.uab.cat/en/publications/the-ar-face-database-cvc-technical-report-24

  45. Columbia object image library (Coil-100)), Technical report CUCS-006-96 (1996). https://www.cs.columbia.edu/CAVE/software/softlib/coil-100.php

Download references

Acknowledgements

This work was supported by National Natural Science Foundation of China under grant 61876026, Chongqing Technology Innovation and Application Development Project under Grant cstc2020jscx-msxmX0190 and cstc2019jscxmbdxX0061, Science and Technology Research Program of Chongqing Municipal Education Commission under grant KJZD-K202100505.

Funding

This work was supported by National Natural Science Foundation of China under grant 61876026, Chongqing Technology Innovation and Application Development Project under Grant cstc2020jscx-msxmX0190 and cstc2019jscxmbdxX0061, Science and Technology Research Program of Chongqing Municipal Education Commission under Grant KJZD-K202100505.

Author information

Authors and Affiliations

  1. The College of Computer and Information Science, Chongqing Normal University, Huxi street, Chongqing, 401331, Chongqing, China

    Ruisheng Ran, Ting Wang & Zheng Li

  2. The College of Computer Science, Chongqing University, Huxi street, Chongqing, 400044, Chongqing, China

    Bin Fang

Authors
  1. Ruisheng Ran
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ting Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zheng Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bin Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

RR contributed to Conceptualization, methodology, writing-review and editing; TW contributed to software, validation writing and preparation; ZL contributed to formal analysis and visualization; BF contributed to supervision; RR and BF contributed to project administration and funding acquisition.

Corresponding author

Correspondence to Bin Fang.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest.

Ethics approval

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ran, R., Wang, T., Li, Z. et al. Polynomial linear discriminant analysis. J Supercomput 80, 413–434 (2024). https://doi.org/10.1007/s11227-023-05485-9

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-023-05485-9

Keywords

Search

Navigation