Skip to main content
SpringerLink
Log in

Generalized Twin Support Vector Machines

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

In this paper, we propose two efficient approaches of twin support vector machines (TWSVM). The first approach is to reformulate the TWSVM formulation by introducing \(L_1\) and \(L_\infty \) norms in the objective functions, and convert into linear programming problems termed as LTWSVM for binary classification. The second approach is to solve the primal TWSVM, and convert into completely unconstrained minimization problem. Since the objective function is convex, piecewise quadratic but not twice differentiable, we present an efficient algorithm using the generalized Newton’s method termed as GTWSVM. Computational comparisons of the proposed LTWSVM and GTWSVM on synthetic and several real-world benchmark datasets exhibits significantly better performance with remarkably less computational time in comparison to relevant baseline methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Alon U, Barkai N, Notterman DA, Gish K, Ybarra S, Mack D, Levine AJ (1999) Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays. Proc Natl Acad Sci 96:6745–6750

    Article  Google Scholar 

  2. Arabasadi Z, Alizadehsani R, Roshanzamir M, Moosaei H, Yarifard AA (2017) Computer aided decision making for heart disease detection using hybrid neural network-genetic algorithm. Comput Methods Prog Biomed 141:19–26

    Article  Google Scholar 

  3. Armijo L (1996) Minimization of functions having lipschitz continuous first partial derivatives. Pacific J Math 16:1–3

    Article  MathSciNet  Google Scholar 

  4. Bazaraa MS, Sherali HD, Shetty CM (2013) Nonlinear programming: theory and algorithms. Wiley, Hoboken

    MATH  Google Scholar 

  5. Bazikar F, Ketabchi S, Moosaei H (2020) Dc programming and dca for parametric-margin \(\nu \)-support vector machine. Appl Intell 50:1–12

    Article  Google Scholar 

  6. Burges CJ (1998) A tutorial on support vector machines for pattern recognition. Data Min Knowl Discov 2:121–167

    Article  Google Scholar 

  7. Cao L (2003) Support vector machines experts for time series forecasting. Neurocomputing 51:321–339

    Article  Google Scholar 

  8. Clarke F (1990) Optimization and Nonsmooth Analysis, Society for Industrial and Aplied Mathematics

  9. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20:273–297

    MATH  Google Scholar 

  10. Deng N, Tian Y, Zhang C (2012) Support vector machines: optimization based theory, algorithms, and extensions. CRC Press, Boca Raton

    Book  Google Scholar 

  11. Déniz O, Castrillon M, Hernández M (2003) Face recognition using independent component analysis and support vector machines. Pattern Recogn Lett 24:2153–2157

    Article  Google Scholar 

  12. Ding S, Shi S, Jia W (2019) Research on fingerprint classification based on twin support vector machine. IET Image Process 14:231–235

    Article  Google Scholar 

  13. Ding S, Zhang N, Zhang X, Wu F (2017) Twin support vector machine: theory, algorithm and applications. Neural Comput Appl 28:3119–3130

    Article  Google Scholar 

  14. Dua D, Graff C (2019) Uci machine learning repository, 2017, http://archive.ics.uci.edu/ml, 37

  15. Fung GM, Mangasarian OL (2005) Multicategory proximal support vector machine classifiers. Mach Learn 59:77–97

    Article  Google Scholar 

  16. Georgiev PG, Theis FJ (2009) Optimization techniques for date representions with biomedical applications. In: Handbook of optimization in medicine, Springer, pp 1–38

  17. Guyon I, Weston J, Barnhill S, Vapnik V (2002) Gene selection for cancer classification using support vector machines. Mach Learn 46:389–422

    Article  Google Scholar 

  18. Hiriart-Urruty JB, Strodiot JJ, Nguyen VH (1984) Generalized hessian matrix and second-order optimality conditions for problems withc 1, 1 data. Appl Math Optim 11:43–56

    Article  MathSciNet  Google Scholar 

  19. Hong ZQ, Yang JY (1991) Optimal discriminant plane for a small number of samples and design method of classifier on the plane. Pattern Recogn 24:317–324

    Article  MathSciNet  Google Scholar 

  20. Khemchandani Jayadeva R, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29:905–910

    Article  Google Scholar 

  21. Joachims T (1999) Making large-scale svm learning practical. advances in kernel methods-support vector learning, http://svmlight.joachims.org/

  22. Ketabchi S, Moosaei H (2012) Minimum norm solution to the absolute value equation in the convex case. J Optim Theory Appl 154:1080–1087

    Article  MathSciNet  Google Scholar 

  23. Ketabchi S, Moosaei H, Razzaghi M, Pardalos PM (2019) An improvement on parametric \(\nu \) -support vector algorithm for classification. Ann Oper Res 276:155–168

    Article  MathSciNet  Google Scholar 

  24. Khemchandani R, Saigal P, Chandra S (2016) Improvements on \(\nu \)-twin support vector machine. Neural Netw 79:97–107

    Article  Google Scholar 

  25. Mangasarian OL (1994) Nonlinear programming, SIAM

  26. Mangasarian OL (2002) A finite newton method for classification. Optim Methods Softw 17:913–929

    Article  MathSciNet  Google Scholar 

  27. Mangasarian OL, Musicant DR (2001) Lagrangian support vector machines. J Mach Learn Res 1:161–177

    MathSciNet  MATH  Google Scholar 

  28. Mangasarian OL, Wild EW (2001) Proximal support vector machine classifiers. In: Proceedings KDD-2001: knowledge discovery and data mining, Citeseer

  29. Mangasarian OL, Wild EW (2005) Multisurface proximal support vector machine classification via generalized eigenvalues. IEEE Trans Pattern Anal Mach Intell 28:69–74

    Article  Google Scholar 

  30. Mangasarian OL, Wild EW (2006) Multisurface proximal support vector machine classification via generalized eigenvalues. IEEE Trans Pattern Anal Mach Intell 28:69–74

    Article  Google Scholar 

  31. Molina GNG, Ebrahimi T, Vesin JM (2003) Joint time-frequency-space classification of eeg in a brain-computer interface application. EURASIP J Adv Signal Process 2003:253269

    Article  Google Scholar 

  32. Moosaei H, Musicant D, Khosravi S, Hladík M (2020) MC-NDC: multi-class normally distributed clustered datasets. Carleton College, University of Bojnord. https://github.com/dmusican/ndc

  33. Musicant D (1998) NDC: normally distributed clustered datasets

  34. Noble WS et al (2004) Support vector machine applications in computational biology. Kernel Methods Comput Biol 71:92

    Google Scholar 

  35. Pardalos PM, Ketabchi S, Moosaei H (2014) Minimum norm solution to the positive semidefinite linear complementarity problem. Optimization 63:359–369

    Article  MathSciNet  Google Scholar 

  36. Peng X (2010) A \(\nu \)-twin support vector machine (\(\nu \)-tsvm) classifier and its geometric algorithms. Inf Sci 180:3863–3875

    Article  MathSciNet  Google Scholar 

  37. Platt J (1999) Fast training of svms using sequential minimal optimization, advances in kernel methods-support vector learning

  38. Richhariya B, Tanveer M (2018) Eeg signal classification using universum support vector machine. Exp Syste Appl 106:169–182

    Article  Google Scholar 

  39. Richhariya B, Tanveer M, Rashid A, Initiative ADN et al (2020) Diagnosis of alzheimer’s disease using universum support vector machine based recursive feature elimination (usvm-rfe). Biomed Signal Process Control 59:101903

    Article  Google Scholar 

  40. Ripley B (1996) Pattern recognition and neural networks datasets collection

  41. Shao YH, Zhang CH, Wang XB, Deng NY (2011) Improvements on twin support vector machines. IEEE Trans Neural Netw 22:962–968

    Article  Google Scholar 

  42. Tanveer M, Khan MA, Ho SS (2016) Robust energy-based least squares twin support vector machines. Appl Intell 45:174–186

    Article  Google Scholar 

  43. Tanveer M, Richhariya B, Khan R, Rashid A, Khanna P, Prasad M, Lin C (2020) Machine learning techniques for the diagnosis of alzheimer’s disease: a review. ACM Trans Multimedia Comput Commun Appl (TOMM) 16:1–35

    Google Scholar 

  44. Tian Y, Qi Z (2004) Review on: twin support vector machines. Ann Data Sci 1:253–277

    Article  Google Scholar 

  45. Trafalis TB, Ince H (2000) Support vector machine for regression and applications to financial forecasting. In: Proceedings of the IEEE-INNS-ENNS international joint conference on neural networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium, vol 6, IEEE, pp 348–353

  46. Valentini G, Muselli M, Ruffino F (2004) Cancer recognition with bagged ensembles of support vector machines. Neurocomputing 56:461–466

    Article  Google Scholar 

  47. Vapnik V (2013) The nature of statistical learning theory. Springer, Berlin

    MATH  Google Scholar 

  48. Vapnik VN, Chervonenkis AJ (1974) Theory of pattern recognition. Nauka, Moscow

    MATH  Google Scholar 

  49. Vn V (1998) Statistical learning theory. Wiley, New York

    Google Scholar 

  50. Zhang Z, Ding S, Sun Y (2020) A support vector regression model hybridized with chaotic krill herd algorithm and empirical mode decomposition for regression task. Neurocomputing 410:185–201

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, Faculty of Science, University of Bojnord, Bojnord, Iran

    H. Moosaei

  2. Department of Applied Mathematics, Faculty of Mathematical Sciences, University of Guilan, Rasht, Iran

    S. Ketabchi & M. Razzaghi

  3. Department of Mathematics, Indian Institute of Technology Indore, Indore, 453 552, India

    M. Tanveer

Authors
  1. H. Moosaei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. S. Ketabchi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. M. Razzaghi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. M. Tanveer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to H. Moosaei.

Ethics declarations

Conflict of interest

The authors have no conflict of interests to declare.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Moosaei, H., Ketabchi, S., Razzaghi, M. et al. Generalized Twin Support Vector Machines. Neural Process Lett 53, 1545–1564 (2021). https://doi.org/10.1007/s11063-021-10464-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-021-10464-3

Keywords

Mathematics Subject Classification

Search

Navigation