Skip to main content
SpringerLink
Log in

Robust energy-based least squares twin support vector machines

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Twin support vector machine (TSVM), least squares TSVM (LSTSVM) and energy-based LSTSVM (ELS-TSVM) satisfy only empirical risk minimization principle. Moreover, the matrices in their formulations are always positive semi-definite. To overcome these problems, we propose in this paper a robust energy-based least squares twin support vector machine algorithm, called RELS-TSVM for short. Unlike TSVM, LSTSVM and ELS-TSVM, our RELS-TSVM maximizes the margin with a positive definite matrix formulation and implements the structural risk minimization principle which embodies the marrow of statistical learning theory. Furthermore, RELS-TSVM utilizes energy parameters to reduce the effect of noise and outliers. Experimental results on several synthetic and real-world benchmark datasets show that RELS-TSVM not only yields better classification performance but also has a lower training time compared to ELS-TSVM, LSPTSVM, LSTSVM, TBSVM and TSVM.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Balasundaram S, Tanveer M (2013) On Lagrangian twin support vector regression. Neural Comput & Applic 22(1):257–267

    Article  Google Scholar 

  2. Burges CJC (1998) A tutorial on support vector machines for pattern recognition. Data Min Knowl Disc 2:1–43

    Article  Google Scholar 

  3. Chang CC, Lin CJ (2011) LIBSVM: a library for support vector machines. ACM Trans Intell Syst Technol (TIST) 2(3):27

    Google Scholar 

  4. Cortes C, Vapnik VN (1995) Support vector networks. Mach Learn 20:273–297

    MATH  Google Scholar 

  5. Cristianini N, Shawe-Taylor J (2000) An introduction to support vector machines and other kernel based learning method. Cambridge University Press, Cambridge

    Book  MATH  Google Scholar 

  6. Demsar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30

    MathSciNet  MATH  Google Scholar 

  7. Duda RO, Hart PR, Stork DG (2001) Pattern Classification, 2nd. John Wiley and Sons

  8. Fung G, Mangasarian OL (2001) Proximal support vector machine classifiers. In: Proceedings of 7th international conference on knowledge and data discovery, San Fransisco, pp 77–86

  9. Golub GH (2012) C.F.V. Loan, Matrix Computations, vol 3. JHU Press

  10. Hsu CW, Lin CJ (2002) A comparison of methods for multi-class support vector machines. IEEE Trans Neural Networks 13:415–425

    Article  Google Scholar 

  11. Hua X, Ding S (2015) Weighted least squares projection twin support vector machines with local information. Neurocomputing 160:228–237. doi:10.1016/j.neucom.2015.02.021

    Article  Google Scholar 

  12. Jayadeva, Khemchandani R, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910

    Article  MATH  Google Scholar 

  13. Joachims T (1999) Making large-scale support vector machine learning practical, Advances in Kernel Methods. MIT Press, Cambridge

    Google Scholar 

  14. Kumar MA, Gopal M (2009) Least squares twin support vector machines for pattern classification. Expert Systems with Applications 36:7535–7543

    Article  Google Scholar 

  15. Kumar MA, Khemchandani R, Gopal M, Chandra S (2010) Knowledge based least squares twin support vector machines. Inf Sci 180(23):4606–4618

    Article  MathSciNet  MATH  Google Scholar 

  16. Lal TN, Hinterberger T, Widman G, Schrder M, Hill J, Rosenstiel W, Elger C, Schlkopf B, Birbaumer N (2004) Methods towards invasive human brain computer interfaces. Advances in Neural Information Processing Systems (NIPS)

  17. Lee YJ, Mangasarian OL (2001a) RSVM: Reduced support vector machines. In: Proceedings of the 1st SIAM international conference on data mining, pp 5–7

  18. Lee YJ, Mangasarian OL (2001b) SSVM: A smooth support vector machine for classification. Comput Optim Appl 20(1): 5–22

    Article  MathSciNet  MATH  Google Scholar 

  19. Mangasarian OL, Musicant DR (2001) Lagrangian support vector machines. J Mach Learn Res 1:161–177

    MathSciNet  MATH  Google Scholar 

  20. Mangasarian OL, Wild EW (2006) Multisurface proximal support vector classification via generalized eigenvalues. IEEE Trans Pattern Anal Mach Intell 28(1):69–74

    Article  Google Scholar 

  21. Mehrkanoon S, Huang X, Suykens JAK (2014) Non-parallel support vector classifiers with different loss functions. Neurocomputing 143(2):294–301

    Article  Google Scholar 

  22. Murphy PM, Aha DW (1992) UCI repository of machine learning databases. University of California, Irvine. http://www.ics.uci.edu/~mlearn

    Google Scholar 

  23. Nasiri JA, Charkari NM, Mozafari K (2014) Energy-based model of least squares twin support vector machines for human action recognition. Signal Process 104:248–257

    Article  Google Scholar 

  24. Peng X (2010) TSVR: An efficient twin support vector machine for regression. Neural Netw 23(3):365–372

    Article  Google Scholar 

  25. Platt J (1999) Fast training of support vector machines using sequential minimal optimization. In: Scholkopf B, Burges CJC, Smola AJ (eds) Advances in Kernel Methods-Support Vector Learning. MIT Press, Cambridge, MA, pp 185– 208

    Google Scholar 

  26. Ripley BD (2007) Pattern recognition and neural networks, Cambridge University Press

  27. Tanveer M (2015) Robust and sparse linear programming twin support vector machines. Cogn Comput 7:137–149

    Article  Google Scholar 

  28. Shao YH, Zhang CH, Wang XB, Deng NY (2011) Improvements on twin support vector machines. IEEE Trans Neural Networks 22(6):962–968

    Article  Google Scholar 

  29. Shao YH, Deng NY, Yang ZM (2012) Least squares recursive projection twin support vector machine for classification. Pattern Recognit 45(6):2299–2307

    Article  MATH  Google Scholar 

  30. Shao YH, Chen WJ, Wang Z, Li CN, Deng NY (2014) Weighted linear loss twin support vector machine for large scale classification. Knowl-Based Syst 73:276–288

    Article  Google Scholar 

  31. Tanveer M, Mangal M, Ahmad I, Shao YH (2016) One norm linear programming support vector regression. Neurocomputing 173:1508–1518. doi:10.1016/j.neucom.2015.09.024

    Article  Google Scholar 

  32. Tanveer M (2015) Application of smoothing techniques for linear programming twin support vector machines. Knowl Inf Syst 45(1):191–214. doi:10.1007/s10115-014-0786-3

    Article  Google Scholar 

  33. Tian Y, Ping Y (2014) Large-scale linear nonparallel support vector machine solver. Neural Netw 50:166–174

    Article  MATH  Google Scholar 

  34. Vapnik VN (1998) Statistical Learning Theory. Wiley, New York

    MATH  Google Scholar 

  35. Vapnik VN (2000) The nature of statistical learning theory 2nd Edition. Springer, New York

    Book  MATH  Google Scholar 

  36. Ye Q, Zhao C, Ye N (2012) Least squares twin support vector machine classification via maximum one-class within class variance. Optimization methods and software 27(1): 53–69

    Article  MathSciNet  MATH  Google Scholar 

  37. Zhang Z, Zhen L, Deng NY (2014) Sparse least square twin support vector machine with adaptive norm. Appl Intell 41(4):1097–1107

    Article  Google Scholar 

Download references

Acknowledgments

The authors gratefully acknowledge the helpful comments and suggestions of the reviewers, which have improved the presentation.

Author information

Authors and Affiliations

  1. School of Computer Engineering, Nanyang Technological University, 50 Nanyang Avenue, Singapore, 639798, Singapore

    Mohammad Tanveer & Shen-Shyang Ho

  2. Department of Electronics and Communication Engineering, The LNM Institute of Information Technology, Jaipur, 302 031, India

    Mohammad Asif Khan

Authors
  1. Mohammad Tanveer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohammad Asif Khan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shen-Shyang Ho
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohammad Tanveer.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tanveer, M., Khan, M.A. & Ho, SS. Robust energy-based least squares twin support vector machines. Appl Intell 45, 174–186 (2016). https://doi.org/10.1007/s10489-015-0751-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-015-0751-1

Keywords

Search

Navigation