Skip to main content
SpringerLink
Log in

Linear discriminant analysis with worst between-class separation and average within-class compactness

  • Research Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

Linear discriminant analysis (LDA) is one of the most popular supervised dimensionality reduction (DR) techniques and obtains discriminant projections by maximizing the ratio of average-case between-class scatter to average-case within-class scatter. Two recent discriminant analysis algorithms (DAS), minimal distance maximization (MDM) and worst-case LDA (WLDA), get projections by optimizing worst-case scatters. In this paper, we develop a new LDA framework called LDA with worst between-class separation and average within-class compactness (WSAC) by maximizing the ratio of worst-case between-class scatter to average-case within-class scatter. This can be achieved by relaxing the trace ratio optimization to a distance metric learning problem. Comparative experiments demonstrate its effectiveness. In addition, DA counterparts using the local geometry of data and the kernel trick can likewise be embedded into our framework and be solved in the same way.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Fukunaga K. Statistical Pattern Recognition. Academic Press, 1990

    MATH  Google Scholar 

  2. Duda R. Hart P. Stork D. Pattern Classification. 2nd ed. New York: John Wiley & Sons, Inc., 2001

    MATH  Google Scholar 

  3. Yang B. Chen S. Wu X. A structurally motivated framework for discriminant analysis. Pattern Analysis and Application, 2011, 14(4): 349–367

    Article  MathSciNet  Google Scholar 

  4. Jae H. Nojun K. Generalization of linear discriminant analysis using Lp-norm. Pattern Recognition Letters, 2013, 34(6): 679–685

    Article  Google Scholar 

  5. Ching W. Chu D. Liao L. Wang X. Regularized orthogonal linear discriminant analysis. Pattern Recognition, 2012, 45(7): 2719–2732

    Article  MATH  Google Scholar 

  6. Li H. Jiang T. Zhang K. Efficient and robust feature extraction by maximum margin criterion. IEEE Transaction on Neural Networks, 2006, 17(1): 157–165

    Article  Google Scholar 

  7. Nenadic Z. Information discriminant analysis: feature extraction with an information-theoretic objective. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2007, 29(8): 1394–1407

    Article  Google Scholar 

  8. Zhu M. Martinez A. Subclass discriminant analysis. IEEE Transactions on Pattern Analysis andMachine Intelligence, 2006, 28(8): 1274–1286

    Article  Google Scholar 

  9. Gao Q. Liu J. Zhang H. Hou J. Yang X. Enhanced fisher discriminant criterion for image recognition. Pattern Recognition, 2012, 45(10): 3717–3724

    Article  Google Scholar 

  10. Cai D. He X. Kun Z. Han J. Bao H. Local sensitive discriminant analysis. In Proceedings of the international joint conference on artificial intelligence. 2007, 141–146

    Google Scholar 

  11. Fan Z. Xu Y. Zhang D. Local linear discriminant analysis framework using sample neighbors. IEEE Transaction on Neural Networks, 2011, 22(7): 1119–1132

    Article  Google Scholar 

  12. Xu B. Huang K. Liu C. Dimensionality reduction by minimal distance maximization. In: Proceedings of 20th International Conference on Pattern Recognition, 2010, 569–572

    Google Scholar 

  13. Zhang Y. Yeung D. Worst-case linear discriminant analysis. In: Proceedings of Advances in Neural Information Processing Systems. 2010, 2568–2576

    Google Scholar 

  14. Ying Y. Li P. Distance metric learning with eigenvalue optimization. Journal of Machine Learning Research, 2012, 13: 1–26

    MathSciNet  MATH  Google Scholar 

  15. Overton M. Womersley R. Optimality conditions and duality theory for minimizing sums of the largest eigenvalues of symmetric matrices. Math Programming, 1993, 62(2): 321–357

    Article  MathSciNet  MATH  Google Scholar 

  16. Overton M. On minimizing the maximum eigenvalue of a symmetric matrix. SIAM Journal on Matrix Analysis and Applications, 1988, 9(2): 256–268

    Article  MathSciNet  MATH  Google Scholar 

  17. Zhang Y. Yeung D. Semi-supervised generalized discriminant analysis. IEEE Transactions on Neural Network, 2011, 22(8): 1207–1217

    Article  Google Scholar 

  18. Mika S. Ratsch G. Weston J. Scholkopf B. Smola A. Muller K. Constructing descriptive and discriminative nonlinear features: Rayleigh coefficients in kernel feature spaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2003, 25(5): 623–628

    Article  Google Scholar 

  19. Duin R. Loog M. Linear dimensionality reduction via a heteroscedastic extension of LDA: the Chernoff criterion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004, 26(6): 732–739

    Article  Google Scholar 

  20. Frank A. Asuncion A. UCI Machine Learning Repository. http://archive.ics.uci.edu/ml

  21. Belhumeur P. Hespanha J. Kriegman D. Eigenfaces vs fisherfaces: recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1997, 19(7): 711–720

    Article  Google Scholar 

  22. Martinez A. Benavente R. The AR-face database. Technical Report 24, CVC, 1998. http://www2.ece.ohiostate.edu/?aleix/ARdatabase.html

    Google Scholar 

  23. Nene S. Nayar S. Murase H. Columbia Object Image Library (COIL-20). Technical Report005, CUCS, 1996. http://www1.cs.columbia.edu/CAVE/software/softlib/coil-20.php

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing, 210016, China

    Leilei Yang & Songcan Chen

Authors
  1. Leilei Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Songcan Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Songcan Chen.

Additional information

Leilei Yang received his BS degree in computer science and technology from Nanjing University of Aeronautics & Astronautics (NUAA) in 2011. Since September 2011, he began his MS degree in computer applications at NUAA. His research interests include pattern recognition, machine learning and neural computing.

Songcan Chen received his BS degree in mathematics from Hangzhou University (now merged into Zhejiang University) in 1983. In December 1985, he completed his MS degree in computer applications at Shanghai Jiaotong University and then worked at the Nanjing University of Aeronautics & Astronautics (NUAA) in January 1986 as an assistant lecturer. Since 1998, as a full-time professor, he has been with the College of Computer Science and Technology at NUAA. His research interests include pattern recognition, machine learning and neural computing.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, L., Chen, S. Linear discriminant analysis with worst between-class separation and average within-class compactness. Front. Comput. Sci. 8, 785–792 (2014). https://doi.org/10.1007/s11704-014-3337-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11704-014-3337-x

Keywords

Search

Navigation