Skip to main content

Advertisement

Springer Nature Link
Log in

Robust Two-Dimensional Linear Discriminant Analysis via Information Divergence

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Due to the complexities of collected data that may contain outliers and noises, many variants of LDA and 2DLDA have been proposed to address this problem and have shown that the improved methods produce much better performance than classical LDA and 2DLDA. In this paper we propose a novel two-dimensional linear discriminant analysis method via information divergence. The proposed method applies the weighted L21 norm to learn a robust projection matrix in the image space. In the proposed model, we introduce the weights into the within-class scatter and the total scatter simultaneously, and learn the weights by imposing information divergence on the objective functions. To handle the proposed model, we resort to Dinkelbach’s extended algorithm to solve the proposed ratio minimization problem. Considering the characteristics of the subproblems, we exploit an equivalent representation of subproblems which can be solved by alternating optimization techniques where each block of variables has good optimization properties. The proposed model not only overcomes the small-sample-size problem, but also suppresses outliers by an adaptively weighted scheme with the guidance of information divergences. The experiments on several image data sets demonstrate that the classification performance of the proposed method is superior to that of some existing methods in the presence of outliers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Wang G, Gong L, Pang Y, Shi N (2020) Dimensionality reduction using discriminant collaborative locality preserving projections. Neural Process Lett 51:611–638

    Google Scholar 

  2. Shan T, Jiang M (2019) Fisher discriminative coupled dictionaries learning. Neural Process Lett 50:2991–3008

    Google Scholar 

  3. Kwak N (2008) Principal component analysis based on L1-norm maximization. IEE Trans Pattern Anal Mach Intell 30:1672–1680

    Google Scholar 

  4. Kwak N (2014) Principal component analysis by Lp-norm maximization. IEEE Trans Cybern 44(5):594–609

    Google Scholar 

  5. Zheng Q, Yang M et al (2018) Improvement of generalization ability of deep CNN via implicit regularization in two-stage training process. IEEE Access 6:15844–15869

    Google Scholar 

  6. Zheng Q, Tian X et al (2019) Layer-wise learning based stochastic gradient descent method for the optimization of deep convolutional neural network. J Intell Fuzzy Syst 37(4):5641–5654

    Google Scholar 

  7. Zheng Q, Tian X, Yang M et al (2020) C-Bayesian framework based drop-path method for 2D discriminative convolutional network pruning. Multidimens Syst Signal Process 31(3):793–827

    MathSciNet  MATH  Google Scholar 

  8. Zheng Q, Yang M et al (2020) A full stage data augmentation method in deep convolutional neural network for natural image classification. Discrete Dyn Nat Soc 2020:1–11

    MATH  Google Scholar 

  9. Wang K, Hu H, Li L, Liu T (2020) Discriminative face recognition methods with structure and label information vial 2-norm regularization. Neural Process Lett 51:639–655

    Google Scholar 

  10. Guo T, Tan X, Zhang L, Liu Q (2019) Learning robust weighted group sparse graph for discriminant visual analysis. Neural Process Lett 49:203–226

    Google Scholar 

  11. Li L, Ge H, Gao J, Zhang Y, Tongu Y, Sun J (2020) A novel geometric mean feature space discriminant analysis method for hyperspectral image feature extraction. Neural Process Lett 51(1):515–542

    Google Scholar 

  12. Chen L, Liao Ko M T et al (2000) A new LDA-based face recognition system which can solve the small sample size problem. Pattern Recognit 33(10):1713–1726

    Google Scholar 

  13. Yu H, Yang J (2001) A direct LDA algorithm for high-dimensional data with application to face recognition. Pattern Recognit 34(10):2067-2-7

    MATH  Google Scholar 

  14. LiuY Gao Q, Miao S et al (2017) A non-greedy algorithm for L1-norm LDA. IEEE Trans Image Process 26(2):684–695

    MathSciNet  MATH  Google Scholar 

  15. Ye Q, Yang J, Liu F (2018) L1-norm distance linear discriminant analysis based on an effective iterative algorithm. IEEE Trans Circuits Syst Video Technol 28(1):114–129

    Google Scholar 

  16. Zhong F, Zhang J (2013) Linear discriminant analysis based on L1 norm. IEEE Trans Image Process 26(2):3018–3027

    MathSciNet  MATH  Google Scholar 

  17. Wang H, Lu X, Hu Z et al (2014) Fisher discriminant analysis with L1 norm. IEEE Trans Cybern 44(6):828–842

    Google Scholar 

  18. Oh J, Kwak N (2013) Generalization of linear discriminant analysis using Lp-norm. Pattern Recognit Lett 34(6):679–685

    Google Scholar 

  19. Zhao G, Wu Y (2019) Efficient large margin-based feature extraction. Neural Process Lett 50:1257–1279

    Google Scholar 

  20. Yuan S, Mao X, Chen L (2019) Sparsity regularization discriminant projection for feature extraction. Neural Process Lett 49:539–553

    Google Scholar 

  21. Liu X, Ma Z (2018) Discriminant analysis with local Gaussian similarity preserving for feature extraction. Neural Process Lett 47:39–55

    Google Scholar 

  22. Lu G, Zou J, Wang Y et al (2018) Sparse L1-norm-based linear discriminant analysis. Multimed Tools Appl 77(13):16155–16175

    Google Scholar 

  23. Li C, Zheng Z, Liu M et al (2017) Robust recursive absolute value inequalities discriminant analysis with sparseness. Neural Netw 93:205–218

    MATH  Google Scholar 

  24. Li C, Shang M, Shao Y et al (2019) Sparse L1 norm two dimensional linear discriminant analysis via generalized elastic net regularization. Neurocomputing 33:780–796

    Google Scholar 

  25. Li C, Shao Y, Wang Z et al (2019) Robust Bhattacharyya bound linear discrinimant analysis though an adaptive algorithm. Knowl-based Syst 183:1–13

    Google Scholar 

  26. Li X, Hua W, Wang H et al (2010) Linear discriminant analysis using rotational invariant L1 norm. Neurocomputing 73(13–15):2571–2579

    Google Scholar 

  27. Nie F, Wang Z, Wang R, Wang Z, Li X (2020) Towards robust discriminative projections learning via non-greedy \(l_{2,1}\) norm minmax. IEEE Trans Pattern Anal Mach Intell. https://doi.org/10.1109/tpami.2019.2961877

    Article  Google Scholar 

  28. Flamary R, Cuturi M, Courty N, Rakotomamonjy A (2018) Wasserstein discriminant analysis. Mach Learn 107:1923–1945

    MathSciNet  MATH  Google Scholar 

  29. Kong H, Wang L, Teoh E, Li X, Wang J, Venkateswarlu R (2005) Generalized 2d principal component analysis for face image representation and recognition. Neural Netw 18(5–6):585–594

    Google Scholar 

  30. Mahanta M, Plataniotis K (2015) 2DLDA matrix-variate formulation of a separable 1DLDA. Pattern Recognit Lett 68:169–175

    Google Scholar 

  31. Wang J (2016) Generalized 2-d principal component analysis by Lp-norm for image analysis. IEEE Trans Cybern 46(3):792–803

    Google Scholar 

  32. Yang J, Zhang D, Frangi AF et al (2004) Two-dimensional PCA: a new approach to appearance-based face representation and recognition. IEEE Trans Pattern Anal Mach Intell 26(1):131–137

    Google Scholar 

  33. Li M, Yuan B (2005) 2D-LDA: a statistical linear discriminant analysis for image matrix. Pattern Recognit Lett 26(5):527–53

    Google Scholar 

  34. Li C, Shao Y, Deng N (2015) Robust L1-norm two-dimensional linear discriminant analysis. Neural Netw 65:92–104

    MATH  Google Scholar 

  35. Wang Q, Gao Q, Xie D (2018) Robust DLPP with non-greedy L1 norm minimization and maximization. IEEE Trans Neural Netw Learn Syst 29(3):738–743

    MathSciNet  Google Scholar 

  36. Ye J, Janardan R, Li Q (2004) Two-dimensional linear discriminant analysis. In: Advances in neural information processing systems, pp 1–8

  37. Kong H, Wang L, Teoh EK, Wang J, Venkateswarlu R (2005) A framework of 2d fisher discriminant analysis: application to face recognition with small number of training samples. Proc Comput Vis Pattern Recognit 2:1083–1088

    Google Scholar 

  38. Li X, Pang Y, Yuan Y (2010) L1-Norm-based 2DPCA. IEEE Trans Syst Man Cybern Part B 40(4):1170–1175

    Google Scholar 

  39. Li M, Wang J, Wang Q et al (2017) Trace ratio 2DLDA with L1-norm optimization. Neurocomputing 266(29):216–225

    Google Scholar 

  40. Du H, Zhao Z, Wang S (2017) Two-dimensional discriminant analysis based on Schatten p-norm for image feature extraction. J Vis Commun Image Represent 45:87–94

    Google Scholar 

  41. Li CN, Shao YH, Wang Z, Deng N (2019) Robust bilateral lp norm two-dimensional linear discriminant analysis. Inf Sci 337:274–297

    MathSciNet  Google Scholar 

  42. Bishop C (2006) Pattern recognition and machine learning. Springer series on information science and statistics. Springer, Berlin

    MATH  Google Scholar 

  43. Liang Z, Chen X, Zhang L, Zhou Y (2020) Correlation classifier based on data perturbation: new formulations and algorithms. Pattern Recognit 100(4):107106

    Google Scholar 

  44. Rodenas R, Lopez M, Verastegui D (1999) Extensions of Dinkelbach’s algorithm for solving nonlinear fractional programming problem. Soc Estad Investig Oper 7(1):33–70

    MATH  Google Scholar 

  45. Chen M, Wang Q, Li X (2018) Robust adaptive sparse learning method for graph clustering. In: 25th IEEE international conference on image processing (ICIP), pp 1618–1622

  46. Liu R, Mulin, Chen M, Wang Q, Li X (2020) Robust rank constrained sparse learning: a graph-based method for clustering. In: IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 1–5

  47. Jenatton R, Obozinski G, Bach F (2010) Structured sparse principle component analysis. In: Proceeding of the 13 international conference on artificial intelligence and statistics, pp 366–373

  48. Liang Z, Xia S et al (2013) Feature extraction based on Lp norm generalized principal component analysis. Pattern Recognit Lett 34(9):1037–1045

    Google Scholar 

  49. Hu D, Feng G, Zhou Z (2007) Two-dimensional locality preserving projections (2DLPP) with its application to palmprint recognition. Pattern Recognit 40(1):339–342

    MATH  Google Scholar 

  50. Nie F, Li J, Li C et al (2017) A generalized power iteration method for solving quadratic problem on the Stiefel manifold Science China. Inf Sci 60(11):112101

    MathSciNet  Google Scholar 

  51. Samaria FS, Harter AC, Harter A (1994) Parameterisation of a stochastic model for human face identification. In: Proceedings of the second IEEE workshop on applications of computer vision, pp. 138–142

  52. Chen W, Li C, Shao Y et al (2019) 2DRLPP robust tow-dimensional locality preserving projection with regularization. Knowl-based Syst 169:53–66

    Google Scholar 

  53. Georghiades A, Belhumeur P, Kriegman D (2002) From few to many: illumination cone models for face recognition under variable lighting and pose. IEEE Trans Pattern Anal Mach Intell 23(6):643–660

    Google Scholar 

Download references

Funding

This work is supported by The Double First Rate Special Fund for Construction of China University of Mining and Technology, No. 2018ZZCX14.

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, 221116, China

    Lei Zhang & Zhizheng Liang

  2. Digitization of Mine, Engineering Research Center of Ministry of Education, Xuzhou, 221116, China

    Lei Zhang & Zhizheng Liang

Authors
  1. Lei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhizheng Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lei Zhang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, L., Liang, Z. Robust Two-Dimensional Linear Discriminant Analysis via Information Divergence. Neural Process Lett 52, 2513–2535 (2020). https://doi.org/10.1007/s11063-020-10359-9

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-020-10359-9

Keywords

Search

Navigation