Skip to main content
SpringerLink
Log in

On the optimal solution to maximum margin projection pursuit

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Most existing dimensionality reduction methods have been applied as a separable data preprocessing step before classification algorithms. This reduces the flexibility of classification algorithms. To handle this limitation, recently, a novel method, namely maximum margin projection pursuit (MMPP), was developed by simultaneously taking into account dimensionality reduction and classification in the criterion function. MMPP alternatively updates the projection matrix and normal vector of classifications hyperplane by optimizing the min-max problem. This results in the following two problems: (1) It does not guarantee both the convergence of the proposed iterative algorithm in real applications and minimization of the objective function; (2) It heavily depends on learning rate and does not get the global optimal solution. In this paper, we simultaneously solve both the projection matrix and norm vector of classification hyperplane by non-iterative method which not only optimizes the criterion function but also is faster than traditional MMPP algorithm. Furthermore, we extend our method to solve multiclass classification problems. Experiments on the Yale, ORL, AR and COIL20 databases have been conducted to evaluate our method. The results illustrate that, compared with the iterative algorithm, our no-iteration algorithm achieves higher efficiency, more stable recognition, and smaller objective value.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Adankon MM, Cheriet M (2009) Support vector machine. Encyclopedia of biometrics: 1303–1308. Springer

  2. Anton B, Fein J, To T, Li X, Silberstein L, Evans CJ (1996) Immunohistochemical localization of orl-1 in the central nervous system of the rat. J Comp Neurol 368(2):229–251

    Article  Google Scholar 

  3. Belhumeur PN, Hespanha JP, Kriegman DJ (1997) Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19(7):711–720

    Article  Google Scholar 

  4. Belkin M, Niyogi P (2003) Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput 15(6):1373–1396

    Article  MATH  Google Scholar 

  5. Cai D, He X, Han J, Zhang HJ (2006) Orthogonal laplacianfaces for face recognition. IEEE Trans Image Process 15(11):3608–3614

    Article  Google Scholar 

  6. Cai D, He X, Han J et al. (2007) Isometric projection. AAAI: 528–533

  7. Cai D, He X, Zhou K, Han J, Bao H (2007) Locality sensitive discriminant analysis. IJCAI 2007:1713–1726

    Google Scholar 

  8. Chen HT, Chang HW, Liu TL (2005) Local discriminant embedding and its variants. Computer vision and pattern recognition, 2005. CVPR 2005. IEEE Comput Soc Conf IEEE 2:846–853

    Google Scholar 

  9. Crammer K, Singer Y (2002) On the learnability and design of output codes for multiclass problems. Mach Learn 47(2):201–233

    Article  MATH  Google Scholar 

  10. Donoho DL (2006) Compressed sensing. IEEE Trans Inf Theory 52(4):1289–1306

    Article  MathSciNet  MATH  Google Scholar 

  11. Fan RE, Chang KW, Hsieh CJ, Wang XR, Lin CJ (2008) Liblinear: a library for large linear classification. J Mach Learn Res 9(Aug):1871–1874

    MATH  Google Scholar 

  12. Fletcher R (2013) Practical methods of optimization. John Wiley & Sons

  13. Gao Q, Liu J, Cui K, Zhang H, Wang X (2014) Stable locality sensitive discriminant analysis for image recognition. Neural Netw 54:49–56

    Article  MATH  Google Scholar 

  14. Gao Q, Ma J, Zhang H, Gao X, Liu Y (2013) Stable orthogonal local discriminant embedding for linear dimensionality reduction. IEEE Trans Image Process 22(7):2521–2531

    Article  Google Scholar 

  15. He X, Cai D, Han J (2008) Learning a maximum margin subspace for image retrieval. IEEE Trans Knowl Data Eng 20(2):189–201

    Article  Google Scholar 

  16. He X, Cai D, Yan S, Zhang HJ (2005) Neighborhood preserving embedding. Comput Vision, 2005. ICCV 2005. Tenth IEEE Int Conf IEEE 2:1208–1213

    Google Scholar 

  17. He X, Niyogi P (2004) Locality preserving projections. Advances in neural information processing systems: 153–160

  18. He X, Yan S, Hu Y, Niyogi P, Zhang HJ (2005) Face recognition using laplacianfaces. IEEE Trans Pattern Anal Mach Intell 27(3):328–340

    Article  Google Scholar 

  19. Hsieh CC, Hsih MH, Jiang MK, Cheng YM, Liang EH (2016) Effective semantic features for facial expressions recognition using svm. Multimed Tools Appl 75(11):6663–6682

    Article  Google Scholar 

  20. Ji S, Ye J (2009) Linear dimensionality reduction for multi-label classification. IJCAI 9:1077–1082

    Google Scholar 

  21. Jolliffe IT (1986) Principal component analysis and factor analysis. Principal component analysis: 115–128. Springer

  22. Kokiopoulou E, Saad Y (2007) Orthogonal neighborhood preserving projections: a projection-based dimensionality reduction technique. IEEE Trans Pattern Anal Mach Intell 29(12):2143–2156

    Article  Google Scholar 

  23. Liu Y, Gao Q, Gao X, Shao L (2018) L2, 1-norm discriminant manifold learning. IEEE Access 6:40723–40734

    Article  Google Scholar 

  24. Liu Y, Gao Q, Miao S, Gao X, Nie F, Li Y (2017) A non-greedy algorithm for l1-norm lda. IEEE Trans Image Process 26(2):684–695

    Article  MathSciNet  MATH  Google Scholar 

  25. Majumdar A, Ward RK (2010) Robust classifiers for data reduced via random projections. IEEE Trans Syst Man Cybernet Part B (Cybernetics) 40(5):1359–1371

    Article  Google Scholar 

  26. Nene SA, Nayar S, Murase H (1996) Columbia object image library (COIL-20). Technical report CUCS-005-96

  27. Nie F, Xiang S, Song Y, Zhang C (2009) Orthogonal locality minimizing globality maximizing projections for feature extraction. Opt Eng 48(1):017202–017202

    Article  Google Scholar 

  28. Nikitidis S, Tefas A, Pitas I (2014) Maximum margin projection subspace learning for visual data analysis. IEEE Trans Image Process 23(10):4413–4425

    Article  MathSciNet  MATH  Google Scholar 

  29. Paul S, Boutsidis C, Magdon-Ismail M, Drineas P (2013) Random projections for support vector machines. Artificial intelligence and statistics: 498–506

  30. Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326

    Article  Google Scholar 

  31. Shi Q, Shen C, Hill R, Hengel AVD (2012) Is margin preserved after random projection? arXiv preprint arXiv:1206.4651

  32. Sugiyama M (2007) Dimensionality reduction of multimodal labeled data by local fisher discriminant analysis. J Mach Learn Res 8(May):1027–1061

    MATH  Google Scholar 

  33. Tenenbaum JB, De Silva V, Langford JC (2000) A global geometric framework for nonlinear dimensionality reduction. Science 290(5500):2319–2323

    Article  Google Scholar 

  34. Varatharajan R, Manogaran G, Priyan M (2018) A big data classification approach using lda with an enhanced svm method for ecg signals in cloud computing. Multimed Tools Appl 77(8):10195–10215

    Article  Google Scholar 

  35. Wang R, Nie F, Hong R, Chang X, Yang X, Yu W (2017) Fast and orthogonal locality preserving projections for dimensionality reduction. IEEE Trans Image Process 26(10):5019–5030

    Article  MathSciNet  MATH  Google Scholar 

  36. Xu Y, Fang X, Wu J, Li X, Zhang D (2016) Discriminative transfer subspace learning via low-rank and sparse representation. IEEE Trans Image Process 25(2):850–863

    Article  MathSciNet  MATH  Google Scholar 

  37. Xu Y, Yang JY, Jin Z (2003) Theory analysis on fslda and ulda. Pattern Recogn 36(12):3031–3033

    Article  MATH  Google Scholar 

  38. Xu Y, Yang JY, Jin Z (2004) A novel method for fisher discriminant analysis. Pattern Recogn 37(2):381–384

    Article  MATH  Google Scholar 

  39. Yan S, Xu D, Zhang B, Zhang HJ, Yang Q, Lin S (2007) Graph embedding and extensions: a general framework for dimensionality reduction. IEEE Trans Pattern Anal Mach Intell 29(1):40–51

    Article  Google Scholar 

  40. Yang B (2009) Svm-induced dimensionality reduction and classification. 2009 Second Int Conf Intell Comput Technol Autom 4:275–278

    Google Scholar 

  41. Yang X, Liu G, Yu Q, Wang R (2017) Stable and orthogonal local discriminant embedding using trace ratio criterion for dimensionality reduction. Multimed Tools Appl 77(3):3071–3081

    Article  Google Scholar 

Download references

Acknowledgements

The work was supported by the Natural Science Foundation of China: 61773302, and the Natural Science Foundation of Ningbo: 2018A610049.

Author information

Authors and Affiliations

  1. The State Key Laboratory of Integrated Services Networks, Xidian University, Xi’an, 710071, People’s Republic of China

    Deyan Xie & Quanxue Gao

  2. School of Computer Science and Center for OPTical IMagery Analysis and Learning (OPTIMAL), Northwestern Polytechnical University, Xi’an, 710072, People’s Republic of China

    Feiping Nie

  3. Xidian-Ningbo Information Technology Institute, Ningbo, 315000, People’s Republic of China

    Quanxue Gao

Authors
  1. Deyan Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Feiping Nie
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Quanxue Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Feiping Nie or Quanxue Gao.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xie, D., Nie, F. & Gao, Q. On the optimal solution to maximum margin projection pursuit. Multimed Tools Appl 79, 35441–35461 (2020). https://doi.org/10.1007/s11042-019-07749-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-07749-0

Keywords

Search

Navigation