Skip to main content
SpringerLink
Log in

Least squares twin parametric-margin support vector machine for classification

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

In this paper, we propose a novel least squares twin parametric-margin support vector machine (TPMSVM) for binary classification, called LSTPMSVM for short. LSTPMSVM attempts to solve two modified primal problems of TPMSVM, instead of two dual problems usually solved. The solution of the two modified primal problems reduces to solving just two systems of linear equations as opposed to solving two quadratic programming problems along with two systems of linear equations in TPMSVM, which leads to extremely simple and fast algorithm. Classification using nonlinear kernel with reduced technique also leads to systems of linear equations. Therefore our LSTPMSVM is able to solve large datasets accurately without any external optimizers. Further, a particle swarm optimization (PSO) algorithm is introduced to do the parameter selection. Our experiments on synthetic as well as on several benchmark data sets indicate that our LSTPMSVM has comparable classification accuracy to that of TPMSVM but with remarkably less computational time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Cortes C, Vapnik VN (1995) Support vector networks. Mach Learn 20:273–297

    MATH  Google Scholar 

  2. Vapnik VN (1998) Statistical learning theory. Wiley, New York

    MATH  Google Scholar 

  3. Deng NY, Tian YJ, Zhang CH (2013) Support vector machines: optimization based theory, algorithms, and extensions. CRC Press, Boca Raton

    Google Scholar 

  4. Burges CJC (1998) A tutorial on support vector machines for pattern recognition. Data Min Knowl Discov 2:121–167

    Article  Google Scholar 

  5. Smola A, Schölkopf B (2004) A tutorial on support vector regression. Stat Comput 14:199–222

    Article  MathSciNet  Google Scholar 

  6. Hao PY, Chiang JH, Lin YH (2009) A new maximal-margin spherical-structured multi-class support vector machine. Appl Intell 30(2):98–111

    Article  Google Scholar 

  7. Lee LH, Wan CH, Rajkumar R, Isa D (2012) An enhanced support vector machine classification framework by using Euclidean distance function for text document categorization. Appl Intell 37(1):80–99

    Article  MATH  Google Scholar 

  8. Lee LH, Rajkumar R, Isa D (2012) Automatic folder allocation system using Bayesian-support vector machines hybrid classification approach. Appl Intell 36(2):295–307

    Article  Google Scholar 

  9. Li C, Liu K, Wang H (2011) The incremental learning algorithm with support vector machine based on hyperplane-distance. Appl Intell 34(1):19–27

    Article  MATH  Google Scholar 

  10. Platt J (1999) Fast training of support vector machines using sequential minimal optimization. In: Advances in kernel methods-support vector learning. MIT Press, Cambridge

    Google Scholar 

  11. Joachims T (1998) Making large-scale support vector machine learning practical. In: Advances in kernel methods-support vector learning. MIT Press, Cambridge

    Google Scholar 

  12. Chang C, Lin C (2001) LIBSVM: a library for support vector machines. Technical report, Department of computer science and information engineering, National Taiwan University

  13. Fan RE, Chang KW, Hsieh CJ, Wang XR, Lin CJ (2008) LIBLINEAR: a library for large linear classification. J Mach Learn Res 9(6):1871–1874

    MATH  Google Scholar 

  14. Suykens JAK, Lukas L, VanDooren P, DeMoor B, Vandewalle J (1999) Least squares support vector machine classifiers: a large scale algorithm. In: European conference of circuit theory design, pp 839–842

    Google Scholar 

  15. Suykens JAK, Vandewalle J (1999) Least squares support vector machine classifiers. Neural Process Lett 9:293–300

    Article  MathSciNet  Google Scholar 

  16. Tian YJ, Shi Y, Liu XH (2012) Recent advances on support vector machines research. Technol Econ Dev Econ 18(1):5–33

    Article  Google Scholar 

  17. Mangasarian O, Wild E (2006) Multisurface proximal support vector classification via generalize deigenvalues. IEEE Trans Pattern Anal Mach Intell 28(1):69–74

    Article  Google Scholar 

  18. Jayadeva, Khemchandani R, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910

    Article  Google Scholar 

  19. Shao YH, Zhang CH, Wang XB, Deng NY (2011) Improvements on twin support vector machines. IEEE Trans Neural Netw 22(6):962–968

    Article  Google Scholar 

  20. Shao YH, Zhang CH, Yang ZM, Jing L, Deng NY (2012) An ε-twin support vector machine for regression. Neural Comput Appl. doi:10.1007/s00521-012-0924-3

    Google Scholar 

  21. Shao YH, Deng NY (2012) A novel margin based twin support vector machine with unity norm hyperplanes. Neural Comput Appl. doi:10.1007/s00521-012-0894-5

    Google Scholar 

  22. Peng XJ (2011) TPMSVM: a novel twin parametric-margin support vector machine for pattern recognition. Pattern Recognit 44(10–11):2678–2692

    Article  MATH  Google Scholar 

  23. Kumar MA, Gopal M (2009) Least squares twin support vector machines for pattern classification. Expert Syst Appl 36(4):7535–7543

    Article  Google Scholar 

  24. Shao YH, Deng NY, Yang ZM (2012) Least squares recursive projection twin support vector machine for classification. Pattern Recognit 45(6):2299–2307

    Article  MATH  Google Scholar 

  25. Wang Z, Shao YH, Wu TR (2012) Proximal parametric-margin support vector classifier and its applications. Neural Comput Appl. doi:10.1007/s00521-012-1278-6

    Google Scholar 

  26. Lee YJ, Mangasarian OL (2001) RSVM: reduced support vector machines. In: First SIAM international conference on data mining, pp 5–7

    Google Scholar 

  27. Schölkopf B, Smola A (2002) Learning with kernels. MIT Press, Cambridge

    Google Scholar 

  28. Qi ZQ, Tian YJ, Shi Y (2012) Laplacian twin support vector machine for semi-supervised classification. Neural Netw 2012(35):46–53

    Article  Google Scholar 

  29. Qi ZQ, Tian YJ, Shi Y (2013) Robust twin support vector machine for pattern classification. Pattern Recognit 46(1):305–316

    Article  MATH  Google Scholar 

  30. Shao YH, Wang Z, Chen WJ, Deng NY (2013) A regularization for the projection twin support vector machine. Knowl-Based Syst 37:203–210

    Article  Google Scholar 

  31. Shao YH, Deng NY, Yang ZM, Chen WJ, Wang Z (2012) Probabilistic outputs for twin support vector machines. Knowl-Based Syst 33:145–151

    Article  Google Scholar 

  32. Yang ZX, Shao YH, Zhang XS (2012) Multiple birth support vector machine for multi-class classification. Neural Comput Appl. doi:10.1007/s00521-012-1108-x

    Google Scholar 

  33. Golub GH, Van Loan CF (1996) Matrix computations, 3rd edn. The John Hopkins University Press, Baltimore, p 50

    MATH  Google Scholar 

  34. Shao YH, Deng NY (2012) A coordinate descent margin based-twin support vector machine for classification. Neural Netw 25:114–121

    Article  MATH  Google Scholar 

  35. Lee YJ, Huang SY (2007) Reduced support vector machines: a statistical theory. IEEE Trans Neural Netw 18(1):1–13

    Article  Google Scholar 

  36. Kennedy J, Eberhart RC (1995) Particle swarm optimization. In: Proceedings of IEEE conference on neural network, vol 4, pp 1942–1948

    Google Scholar 

  37. Shi Y, Eberhart RC (1998) A modified particle swarm optimizer. In: Proceeding of the IEEE congress on evolutionary computation, pp 69–73

    Google Scholar 

  38. Lin S-W, Lee Z-J, Chen S-C, et al (2008) Parameter determination of support vectormachines and feature selection using simulated annealing approach. Appl Soft Comput 8:1505–1512

    Article  Google Scholar 

  39. Lin S-W Ying K-C Chen S-C, et al (2008) Particle swarm optimization for parameter determination and feature selection of support vector machines. Expert Syst Appl 35:1817–1824

    Article  Google Scholar 

  40. García-Nieto J, Alba E (2012) Parallel multi-swarm optimizer for gene selection in DNA microarrays. Appl Intell 37(2):255–266

    Article  Google Scholar 

  41. Ahn CW, Ramakrishna RS (2010) A diversity preserving selection in multiobjective evolutionary algorithms. Appl Intell 32(3):231–248

    Article  Google Scholar 

  42. Blake CI, Merz CJ (1998) UCI repository of machine learning databases. http://www.ics.uci.edu/~mlearn/MLRepository.html

  43. Musicant DR (1998) NDC: normally distributed clustered datasets. www.cs.wisc.edu/dmi/svm/ndc/

  44. MathWorks (2007). MATLAB. http://www.mathworks.com

  45. Duda RO, Hart PE, Stork DG (2001) Pattern classification 2nd edn. Wiley, New York

    MATH  Google Scholar 

  46. Hao PY (2010) New support vector algorithms with parametric insensitive/margin model. Neural Netw 23(1):293–300

    Article  Google Scholar 

  47. Ripley BD (1996) Pattern recognition and neural networks. Cambridge University Press, Cambridge

    MATH  Google Scholar 

Download references

Acknowledgements

We thank the anonymous reviewers for their valuable suggestions. This work is supported by the National Natural Science Foundation of China (No. 10971223, No. 11071252 and No. 11201426), the Zhejiang Provincial Natural Science Foundation of China (No. LQ12A01020) and the Science and Technology Foundation of Department of Education of Zhejiang Province (No. Y201225179 and No. Y201225256).

Author information

Authors and Affiliations

  1. Zhijiang College, Zhejiang University of Technology, Hangzhou, 310024, P.R. China

    Yuan-Hai Shao & Wei-Jie Chen

  2. Mathematics Colledge of Jilin University, Changchun, 130012, P.R. China

    Zhen Wang

  3. College of Science China Agricultural University, Beijing, 100083, P.R. China

    Nai-Yang Deng

Authors
  1. Yuan-Hai Shao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhen Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wei-Jie Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nai-Yang Deng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nai-Yang Deng.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Shao, YH., Wang, Z., Chen, WJ. et al. Least squares twin parametric-margin support vector machine for classification. Appl Intell 39, 451–464 (2013). https://doi.org/10.1007/s10489-013-0423-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-013-0423-y

Keywords

Search

Navigation