Skip to main content
SpringerLink
Log in

Projection multi-birth support vector machinea for multi-classification

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

As an important multi-classification learning tool, multi-birth support vector machine (MBSVM) has been widely studied and applied due to its low computational complexity and good generalization. In this paper, a new multi-birth support vector machine is proposed to handle multi-class classification problem, called projection multi-birth support vector machine (PMBSVM). Specifically, we intend to seek a projection direction wk for k-th class, so that the covariance of remaining samples (except the k-th class) is as small as possible, and the samples of k-th class are as far as possible from the mean of the remaining samples. The proposed PMBSVM not only inherits the advantages of MBSVM, but also can find a suitable projection direction for each class so that the sample is separable in the projection space. Additionally, a regularization term is introduced to maximize the margin of different classes in the projected space. Moreover, a recursive PMBSVM algorithm is proposed for generating multiple orthogonal projection directions for each class. Then we extend the proposed approaches to nonlinear situations through kernel technology. Simulation results on benchmark datasets show that the proposed algorithms improve the generalization in most cases.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Vapnik VN (1995) The Nature of Statistical Learning Theory. Statistics for Engineering and Information Science

  2. Seehapoch T, Wongthanavasu S (2013) Speech emotion recognition using Support Vector Machines. 2013 5th International Conference on Knowledge and Smart Technology (KST), pp 86–91

  3. Pai P, Hsu M, Wang M (2011) A support vector machine-based model for detecting top management fraud. Knowl-Based Syst 24:314–321

    Article  Google Scholar 

  4. Shen L, Chen H, Yu Z, Kang W, Zhang B, Li HZ, Yang B, Liu D (2016) Evolving support vector machines using fruit fly optimization for medical data classification. Knowl-Based Syst 96:61–75

    Article  Google Scholar 

  5. Keerthi SS, Shevade SK, Bhattacharyya C, Murthy KR (2001) Improvements to Platt’s SMO Algorithm for SVM Classifier Design. Neural Comput 13:637–649

    Article  Google Scholar 

  6. Boser BE, Guyon I, Vapnik VN (1992) A training algorithm for optimal margin classifiers. Proc Ann Workshop Comput Learn Theory 5:144–152

    Google Scholar 

  7. Mangasarian OL, Wild EW (2006) Multisurface proximal support vector machine classification via generalized eigenvalues. IEEE Trans Pattern Anal Mach Intell 28:69–74

    Article  Google Scholar 

  8. Shao Y, Deng N, Chen W, Wang Z (2013) Improved Generalized Eigenvalue Proximal Support Vector Machine. IEEE Signal Process Lett 20:213–216

    Article  Google Scholar 

  9. Jayadeva KR, Chandra S (2007) Twin Support Vector Machines for Pattern Classification. IEEE Trans Pattern Anal Mach Intell 29:905–910

    Article  Google Scholar 

  10. Shao Y, Zhang C, Wang X, Deng N (2011) Improvements on Twin Support Vector Machines. IEEE Trans Neural Netw 22:962–968

    Article  Google Scholar 

  11. Kumar MA, Gopal M (2008) Application of smoothing technique on twin support vector machines. Pattern Recogn Lett 29:1842–1848

    Article  Google Scholar 

  12. Peng X (2010) TSVR: An efficient Twin Support Vector Machine for regression. Neural Netw Official J Int Neural Netw Soc 23(3):365-72

  13. Ghorai S, Mukherjee A, Dutta PK (2009) Nonparallel plane proximal classifier. Signal Process 89:510–522

    Article  Google Scholar 

  14. Peng X (2010) Least squares twin support vector hypersphere (LS-TSVH) for pattern recognition. Expert Syst Appl 37:8371–8378

    Article  Google Scholar 

  15. Xie J, Hone KS, Xie W, Gao X, Shi Y, Liu X (2013) Extending twin support vector machine classifier for multi-category classification problems. Intell Data Anal 17:649–664

    Article  Google Scholar 

  16. Hastie TJ, Tibshirani R (1997) Classification by Pairwise Coupling. NIPS

  17. Subirats JL, Jerez JM, Gómez I, Franco L (2010) Multiclass Pattern Recognition Extension for the New C-Mantec Constructive Neural Network Algorithm. Cogn Comput 2:285–290

    Article  Google Scholar 

  18. Hao Z, Liu B, Yang X (2006) A Comparision of Multiclass Support Vector Machine Algorithms. 2006 International Conference on Machine Learning and Cybernetics, pp 4221–4226

  19. Kressel U (1999) Pairwise classification and support vector machines

  20. Yang Z, Shao Y, Zhang X (2012) Multiple birth support vector machine for multi-class classification. Neural Comput Appl 22:153–161

    Article  Google Scholar 

  21. Xu Y, Guo R, Wang L (2012) A Twin Multi-Class Classification Support Vector Machine. Cogn Comput 5:580–588

    Article  Google Scholar 

  22. Nasiri JA, Charkari NM, Jalili S (2014) Least squares twin multi-class classification support vector machine. Pattern Recognition 48(3)

  23. Lima MD, Costa NL, Barbosa RM (2018) Improvements on least squares twin multi-class classification support vector machin. Neurocomputing 313:196–205

    Article  Google Scholar 

  24. Belhumeur PN, Hespanha JP, Kriegman DJ, Aeigenfaces V (1996) Fisherfaces: recognition using class specific linear projection

  25. Yang J, Yang J (2003) Why can LDA be performed in PCA transformed space?. Pattern Recogn 36:563–566

    Article  Google Scholar 

  26. Bishop C (2006) Pattern recognition and machine learning. Springer, New York

    MATH  Google Scholar 

  27. Ye Q, Zhao C, Ye N, Chen Y (2010) Multi-weight vector projection support vector machines. Pattern Recogn Lett 31(13):2006–2011

    Article  Google Scholar 

  28. Chen X, Yang J, Ye Q, Liang J (2011) Recursive projection twin support vector machine via within-class variance minimization. Pattern Recogn 44(10-11):2643–2655

    Article  Google Scholar 

  29. Shao YH, Wang Z, Chen WJ, Deng NY (2013) A regularization for the projection twin support vector machine. Knowl-Based Syst 37(none):203—210

  30. Vapnik VN (1998) Statistical learning theory. Wiley, New York

    MATH  Google Scholar 

  31. Müller K, Mika S, Rätsch G, Tsuda K, Schölkopf B (2001) An introduction to kernel-based learning algorithms. IEEE trans Neural Netw 12(2):181–201

  32. Shao YH, Deng NY, Yang Z (2012) Least squares recursive projection twin support vector machine for classification. Pattern Recogn 45(6):2299–2307

    Article  Google Scholar 

  33. Peng X, Chen D (2018) Ptsvrs: regression models via projection twin support vector machine. Inf Sci 435:1–14

    Article  MathSciNet  Google Scholar 

  34. Li CN, Huang YF, Wu HJ, Shao YH, Yang Z (2016) Multiple recursive projection twin support vector machine for multi-class classification. Int J Mach Learn Cybern 7(5):729–740

    Article  Google Scholar 

  35. Mangasarian OL (1994) Nonlinear programming. SIAM

  36. Blake CL, Merz CJ (1998) UCI Repository for machine learning databases. http://www.ics.uci.edu/mlearn/MLRepository.html

Download references

Acknowledgments

This work was supported in part by National Natural Science Foundation of China (No11471010) and Chinese Universities Scientific Fund.

Author information

Authors and Affiliations

  1. College of Information and Electrical Engineering, China Agricultural University, Beijing, 100083, China

    Yakun Wen, Jun Ma & Chao Yuan

  2. College of Science, China Agricultural University, Beijing, 100083, China

    Liming Yang

Authors
  1. Yakun Wen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jun Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chao Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Liming Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Liming Yang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wen, Y., Ma, J., Yuan, C. et al. Projection multi-birth support vector machinea for multi-classification. Appl Intell 50, 3040–3056 (2020). https://doi.org/10.1007/s10489-020-01699-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-020-01699-z

Keywords

Search

Navigation