Skip to main content
SpringerLink
Log in

A novel robust nonparallel support vector classifier based on one optimization problem

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

The real data is often contaminated by outliers (label noises), which lead to the deviation of the final separating hyperplane and reduce the prediction accuracy. In this paper, we propose a novel robust geometric twin parametric-margin support vector machine (RGTPSVM) to tackle this issue. The method constructs classifier directly based on the average of two resulting nonparallel boundary hyperplanes, which are obtained by solving one single quadratic programming problem. We show that the boundary hyperplanes sought by RGTPSVM are less sensitive to the outliers, producing a more reasonable and robust final separating hyperplane. To further strengthen the robustness, we also design a rescaled squared hinge loss function for the model, since the hinge loss function is linear related to the margin variable and diverges to infinity. Extensive experiments are conducted to demonstrate the prediction performance of RGTPSVM. Compared with several popular SVMs, our method is stable and outperforms others in most cases.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Data Availability

All data generated or analyzed during this study are included in this published article (and its additional files). The datasets analyzed in Sect. 4.2 are available in the UC Irvine Machine Learning Repository, https://archive-beta.ics.uci.edu/.

References

  1. Beck A (2015) On the convergence of alternating minimization for convex programming with applications to iteratively reweighted least squares and decomposition schemes. SIAM J Optim 1(25):185–209

    Article  MathSciNet  MATH  Google Scholar 

  2. Collobert R, Sinz F, Weston J, Bottou L (2006) Large scale transductive SVMS. J Mach Learn Res 7(62):1687–1712

    MathSciNet  MATH  Google Scholar 

  3. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297

    Article  MATH  Google Scholar 

  4. Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30

    MathSciNet  MATH  Google Scholar 

  5. Hao P (2010) New support vector algorithms with parametric insensitive/margin model. Neural Netw 23(1):60–73

    Article  MATH  Google Scholar 

  6. Huang X, Shi L, Suykens J (2013) Support vector machine classifier with pinball loss. IEEE Trans Pattern Anal Mach Intell 36(5):984–997

    Article  Google Scholar 

  7. Huang X, Shi L, Suykens J (2014) Ramp loss linear programming support vector machine. J Mach Learn Res 15(1):2185–2211

    MathSciNet  MATH  Google Scholar 

  8. Khemchandani R, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910

    Article  Google Scholar 

  9. Liu W, Pokharel P, Principe J (2007) Correntropy: properties and applications in non-gaussian signal processing. IEEE Trans Signal Process 55(11):5286–5298

    Article  MathSciNet  MATH  Google Scholar 

  10. López J, Maldonado S, Carrasco M (2018) Double regularization methods for robust feature selection and SVM classification via DC programming. Inf Sci 429:377–389

    Article  MathSciNet  MATH  Google Scholar 

  11. Ma Y, Liang X, Sheng G, Kwok J, Wang M, Li G (2020) Noniterative sparse LS-SVM based on globally representative point selection. IEEE Trans Neural Netw Learn Syst 1–11

  12. Mangasarian O, Musicant D (1999) Successive overrelaxation for support vector machines. IEEE Trans Neural Netw 10(5):1032–1037

    Article  Google Scholar 

  13. Mangasarian O, Musicant D (2001) Lagrangian support vector machines. J Mach Learn Res 1(3):161–177

    MathSciNet  MATH  Google Scholar 

  14. Mangasarian O, Wild E (2006) Multisurface proximal support vector machine classification via generalized eigenvalues. IEEE Trans Pattern Anal Mach Intell 28(1):69–74

    Article  Google Scholar 

  15. Mercer J (1909) Functions of positive and negative type, and their connection the theory of integral equations. Philos Trans R Soc Lond 209:415–446

    Article  MATH  Google Scholar 

  16. Nikolova M, Ng M (2005) Analysis of half-quadratic minimization methods for signal and image recovery. SIAM J Sci Comput 27(3):937–966

    Article  MathSciNet  MATH  Google Scholar 

  17. Peng X (2011) TPMSVM: a novel twin parametric-margin support vector machine for pattern recognition. Pattern Recogn 44:2678–2692

    Article  MATH  Google Scholar 

  18. Peng X, Chen D, Kong L (2014) A clipping dual coordinate descent algorithm for solving support vector machines. Knowl Based Syst 71:266–278

    Article  Google Scholar 

  19. Qi K, Yang H, Hu Q, Yang D (2019) A new adaptive weighted imbalanced data classifier via improved support vector machines with high-dimension nature. Knowl Based Syst 185:104933

    Article  Google Scholar 

  20. Qi K, Yang H (2021) Elastic net nonparallel hyperplane support vector machine and its geometrical rationality. IEEE Trans Neural Netw Learn Syst. https://doi.org/10.1109/TNNLS.2021.3084404

    Article  Google Scholar 

  21. Ren Z, Yang L (2018) Correntropy-based robust extreme learning machine for classification. Neurocomputing 313:74–84

    Article  Google Scholar 

  22. Schölkopf B, Smola A, Williamson R, Bartlett P (2000) New support vector algorithms. Neural Comput 12(5):1207–1245

    Article  Google Scholar 

  23. Shao Y, Chen W, Deng N (2014) Nonparallel hyperplane support vector machine for binary classification problems. Inf Sci 263:22–35

    Article  MathSciNet  MATH  Google Scholar 

  24. Simian D, Stoica F, Bărbulescu A (2020) Automatic optimized support vector regression for financial data prediction. Neural Comput Appl 32:2383–2396

    Article  Google Scholar 

  25. Singh A, Pokharel R, Principe J (2014) The c-loss function for pattern classification. Pattern Recogn 47(1):441–453

    Article  MATH  Google Scholar 

  26. Sleeman W IV, Krawczyk B (2021) Multi-class imbalanced big data classification on spark. Knowl Based Syst 212(113):106598

    Article  Google Scholar 

  27. Suykens JA, Vandewalle J (1999) Least squares support vector machine classifiers. Neural Process Lett 9(3):293–300

    Article  Google Scholar 

  28. Tian Y, Ju X (2015) Nonparallel support vector machine based on one optimization problem for pattern recognition. J Oper Res Soc China 3(4):499–519

    Article  MathSciNet  MATH  Google Scholar 

  29. Vapnik V (2013) The nature of statistical learning theory. Springer, Berlin

    MATH  Google Scholar 

  30. Wahba G (1999) Support vector machines, reproducing kernel Hilbert spaces and the randomized GACV. Adv Kernel Methods Support Vector Learn 6:69–87

    Google Scholar 

  31. Wang C, Ye Q, Luo P, Ye N, Fu L (2019) Robust capped l1-norm twin support vector machine. Neural Netw 114:47–59

    Article  MATH  Google Scholar 

  32. Wang D, Zhang X, Chen H, Zhou Y, Cheng F (2020) A sintering state recognition framework to integrate prior knowledge and hidden information considering class imbalance. IEEE Trans Ind Electron 1–1

  33. Wang M, Chen H (2020) Chaotic multi-swarm whale optimizer boosted support vector machine for medical diagnosis. Appl Soft Comput 88:105946

    Article  Google Scholar 

  34. Wang X, Yang Z, Chen X, Liu W (2019) Distributed inference for linear support vector machine. J Mach Learn Res 20:1–41

    MathSciNet  MATH  Google Scholar 

  35. Wu Y, Liu Y (2013) Adaptively weighted large margin classifiers. J Comput Graph Stat 22(2):416–432

    Article  MathSciNet  Google Scholar 

  36. Xu B, Shen S, Shen F, Zhao J (2019) Locally linear SVMs based on boundary anchor points encoding. Neural Netw 117:274–284

    Article  Google Scholar 

  37. Xu G, Cao Z, Hu B, Principe J (2017) Robust support vector machines based on the rescaled hinge loss function. Pattern Recogn 63:139–148

    Article  MATH  Google Scholar 

  38. Xu G, Hu B, Principe J (2018) Robust c-loss kernel classifiers. IEEE Trans Neural Netw Learn Syst 29(3):510–522

    Article  MathSciNet  Google Scholar 

  39. Yang L, Dong H (2019) Robust support vector machine with generalized quantile loss for classification and regression. Appl Soft Comput 81:105483

    Article  Google Scholar 

  40. Zhao J, Xu Y, Fujita H (2019) An improved non-parallel Universum support vector machine and its safe sample screening rule. Knowl Based Syst 170:79–88

    Article  Google Scholar 

  41. Zheng X, Zhang L, Yan L (2021) CTSVM: a robust twin support vector machine with correntropy-induced loss function for binary classification problems. Inf Sci 599:22–45

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China [Grant No. 11671059].

Author information

Authors and Affiliations

  1. College of Mathematics and Statistics, Chongqing University, Chongqing, 401331, China

    Kai Qi & Hu Yang

Authors
  1. Kai Qi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hu Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hu Yang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Qi, K., Yang, H. A novel robust nonparallel support vector classifier based on one optimization problem. Neural Comput & Applic 35, 799–814 (2023). https://doi.org/10.1007/s00521-022-07814-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-022-07814-0

Keywords

Search

Navigation