Skip to main content
SpringerLink
Log in

Least squares recursive projection twin support vector machine for multi-class classification

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Multiple recursive projection twin support vector machine (MPTSVM) is a recently proposed classifier and has been proved to be outstanding in pattern recognition. However, MPTSVM is computationally expensive since it involves solving a series of quadratic programming problems. To relieve the training burden, in this paper, we propose a novel multiple least squares recursive projection twin support vector machine (MLSPTSVM) based on least squares recursive projection twin support vector machine (LSPTSVM) for multi-class classification problem. For a \(K(K>2)\) classes classification problem, MLSPTSVM aims at seeking K groups of projection axes, one for each class that separates it from all the other. Due to solving a series of linear equations, our algorithm tends to relatively simple and fast. Moreover, a recursive procure is introduced to generate multiple orthogonal projection axes for each class to enhance its performance. Experimental results on several synthetic and UCI datasets, as well as on relatively large datasets demonstrate that our MLSPTSVM has comparable classification accuracy while takes significantly less computing time compared with MPTSVM, and also obtains better performance than several other SVM related methods being used for multi-class classification problem.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Cortes C, Vapnik V (1995) Support vector networks. Mach Learn 20(3):273–297

    MATH  Google Scholar 

  2. Burges C (1998) A tutorial on support vector machines for pattern recognition. Data Min Knowl Discov 2:121–167

    Article  Google Scholar 

  3. Noble W (2004) Support vector machine applications in computational biology. In: Schöelkopf B, Tsuda K, Vert J-P (eds) Kernel methods in computational biology. MIT Press, Cambridge, pp 71–92

  4. Li Y, Shao Y, Jing L, Deng N (2011) An efficient support vector machine approach for identifying protein s-nitrosylation sites. Protein Pept Lett 18(6):573–587

    Article  Google Scholar 

  5. Li Y, Shao Y, Deng N (2011) Improved prediction of palmitoylation sites using PWMs and SVM. Protein Pept Lett 18(2):186–193(8)

  6. Vapnik V (1998) Statistical learning theory. Wiley, New York

    MATH  Google Scholar 

  7. Suykens J, Vandewalle J (1999) Least squares support vector machine classifiers. Neural Process Lett 9(3):293–300

    Article  MathSciNet  MATH  Google Scholar 

  8. Fung G, Mangasarian O (2005) Multicategory proximal support vector machine classifiers. Mach Learn 59:77–97

    Article  MATH  Google Scholar 

  9. Mangasarian O, Wild E (2006) Multisurface proximal support vector classification via generalize eigenvalues. IEEE Trans Pattern Anal Mach Intell 28(1):69–74

    Article  Google Scholar 

  10. Jayadeva R, Khemchandani R, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910

    Article  MATH  Google Scholar 

  11. Qi Z, Tian Y, Shi Y (2015) Successive overrelaxation for laplacian support vector machine. IEEE Trans Neural Netw Learn Syst 26(4):674–683

    Article  MathSciNet  Google Scholar 

  12. Tanveer M (2015) Robust and sparse linear programming twin support vector machines. Cogn Comput 7(1):137–149

    Article  Google Scholar 

  13. Shao Y, Deng N, Chen W, Wang Z (2013) Improved generalized eigenvalue proximal support vector machine. IEEE Signal Process Lett 20(3):213–216

    Article  Google Scholar 

  14. Kumar M, Gopal M (2009) Least squares twin support vector machines for pattern classification. Expert Syst Appl 36(4):7535–7543

    Article  Google Scholar 

  15. Shao Y, Zhang C, Wang X, Deng N (2011) Improvements on twin support vector machines. IEEE Trans Neural Netw 22(6):962–968

    Article  Google Scholar 

  16. Shao Y, Deng N (2012) A coordinate descent margin based-twin support vector machine for classification. Neural Netw 25:114–121

    Article  MATH  Google Scholar 

  17. Qi Z, Tian Y, Shi Y (2012) Laplacian twin support vector machine for semi-supervised classification. Neural Netw 35:46–53

    Article  MATH  Google Scholar 

  18. Shao Y, Chen W, Deng NY (2014) Nonparallel hyperplane support vector machine for binary classification problems. Inf Sci 263(1):22–35

    Article  MathSciNet  MATH  Google Scholar 

  19. Tian Y, Qi Z, Ju X (2014) Nonparallel support vector machine for pattern classification. IEEE Trans Cybern 44(7):1067–1079

    Article  Google Scholar 

  20. Wang Z, Shao Y, Bai L, Deng N (2014) Twin support vector machine for clustering. IEEE Trans Neural Netw Learn Syst. doi:10.1109/TNNLS.2014.2379930

  21. Ye Q, Zhao C, Ye N, Chen Y (2010) Multi-weight vector projection support vector machines. Pattern Recognit Lett 31(13):2006–2011

    Article  Google Scholar 

  22. Chen X, Yang J, Ye Q, Liang J (2011) Recursive projection twin support vector machine via within-class variance minimization. Pattern Recognit 44(10):2643–2655

    Article  MATH  Google Scholar 

  23. Shao Y, Wang Z, Chen W, Deng N (2013) A regularization for the projection twin support vector machine. Knowl-Based Syst 37:203–210

    Article  Google Scholar 

  24. Shao Y, Deng N, Yang Z (2012) Least squares recursive projection twin support vector machine for classification. Pattern Recognit 45(6):2299–2307

    Article  MATH  Google Scholar 

  25. Weston J, Watkins C (1998) Multi-class support vector machines. Technical report CSD-TR-98-04

  26. Schwenker F (2000) Hierarchical support vector machines for multi-class pattern recognition. In: Fourth international conference on knowledge-based intelligent information engineering systems & allied technologies, vol 2, pp 561–565

  27. Lee Y, Lin Y, Wahba G (2004) Multicategory support vector machines: theory and application to the classification of microarray data and satellite radiance data. J Am Stat Assoc 99(465):67–81

    Article  MathSciNet  MATH  Google Scholar 

  28. Yang Z, Shao Y, Zhang X (2013) Multiple birth support vector machine for multi-class classification. Neural Comput Appl 22(1Suppl):153–161

    Article  Google Scholar 

  29. Li C, Huang Y, Wu H, Shao Y, Yang Z (2014) Multiple recursive projection twin support vector machine for multi-class classification. Int J Mach Learn Cybern. doi:10.1007/s13042-014-0289-2

  30. Golub G, Van Loan C (1996) Matrix computations, 3rd edn. Johns Hopkins University Press, Baltimore

    MATH  Google Scholar 

  31. Belhumeur P, Hespanha J, Kriegman D (1997) Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19(7):711–720

    Article  Google Scholar 

  32. Yang J (2003) Why can LDA be performed in PCA transformed space? Pattern Recognit 36(2):563–566

    Article  Google Scholar 

  33. Bishop C (2006) Pattern recognition and machine learning. Springer, New York

    MATH  Google Scholar 

  34. Tao Q, Chu D, Wang J (2008) Recursive support vector machines for dimensionality reduction. IEEE Trans Neural Netw 19(1):189–193

    Article  Google Scholar 

  35. Lee YJ, Mangasarian O (2001) RSVM: reduced support vector machines. Technical report 00-07. Data Mining Institute, Computer Sciences Department, University of Wisconsin, Madison (2001)

  36. Blake C, Merz C (1998) UCI repository for machine learning databases. http://www.ics.uci.edu/mlearn/MLRepository.html

  37. Chang C, Lin C (2011) LIBSVM: a library for support vector machines. http://www.csie.ntu.edu.tw/cjlin/libsvmtools/datasets

  38. Chang C, Lin C (2011) LIBSVM : a library for support vector machines. ACM Trans Intell Syst Technol 2(27):1–27

    Article  Google Scholar 

  39. Duda R, Hart P, Stork D (2012) Pattern classification, 2nd edn.   Wiley, New York

    MATH  Google Scholar 

  40. Jin Z, Yang J, Hu Z, Lou Z (2001) Face recognition based on the uncorrelated discriminant transformation. Pattern Recognit 34:1405–1416

    Article  MATH  Google Scholar 

Download references

Acknowledgments

The authors would like to thank the editors and reviewers for their valuable comments and helpful suggestions, which improved the quality of this paper. This work is supported by the National Natural Science Foundation of China (Nos. 11201426, 11371365, 11426202 and 11426200), the Zhejiang Provincial Natural Science Foundation of China (Nos. LQ12A01020, LQ13F030010, LY15F030013, and LQ14G010004), the Ministry of Education, Humanities and Social Sciences Research Project of China (No. 13YJC910011) and the Science Foundation of Chongqing Municipal Commission of Science and Technology (Grant No. CSTC201 4jcyjA40011) and Scientific and Technological Research Program of Chongqing Municipal Education Commission (Grant No. KJ1400513).

Author information

Authors and Affiliations

  1. Zhijiang College, Zhejiang University of Technology, Hangzhou, 310024, People’s Republic of China

    Zhi-Min Yang, Chun-Na Li & Yuan-Hai Shao

  2. College of Science, Zhejiang University of Technology, Hangzhou, 310023, People’s Republic of China

    He-Ji Wu

Authors
  1. Zhi-Min Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. He-Ji Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chun-Na Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yuan-Hai Shao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yuan-Hai Shao.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, ZM., Wu, HJ., Li, CN. et al. Least squares recursive projection twin support vector machine for multi-class classification. Int. J. Mach. Learn. & Cyber. 7, 411–426 (2016). https://doi.org/10.1007/s13042-015-0394-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-015-0394-x

Keywords

Search

Navigation