Skip to main content
SpringerLink
Log in

Smooth support vector machine with generalized pinball loss for Pattern Classification

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

The generalized pinball loss function was introduced to improve the noise sensitivity and random instability of the original support vector machine (SVM). However, because the generalized pinball loss function is not smooth, SVM that uses it (GP-loss-SVM) is non-differentiable. From this issue, it makes the several practical algorithms that cannot be used to find the solution. As a result, GP-loss-SVM performs less effectively in practice. In order to solve this problem and improve GP-loss-SVM, in this paper, we construct a new smooth approximation function of the generalized pinball loss function into the SVM. As a result, a smooth SVM with the generalized pinball loss function is obtained. Furthermore, we use an effective method, quasi-Newton–Armijo, to solve our model. Moreover, we prove that our new loss function can estimate the generalized pinball loss function. Finally, we conduct a thorough experimental investigation employing a variety of machine learning benchmark datasets. The experiment results on binary datasets show that the proposed method outperforms the original SVM method by up to 1.59% for linear kernel and 1.45% for nonlinear kernel; the experiment results on multi-class datasets show that the proposed method outperforms the baseline model by up to 1.43% for linear kernels. We also evaluate the statistical significance of the performance values attained for the various models using the Friedman test. The results demonstrate that the average rank of the proposed method is better compared to baseline models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Cortes C, Vapnik VN (1995) Support vector networks. Mach Learn 20(3):273–297

    Article  MATH  Google Scholar 

  2. Zhai Z, Gu B, Li X, Huang H (2020) Safe sample screening for robust support vector machine. AAAI

  3. Liang Z, Li Y (2009) Incremental support vector machine learning in the primal and applications. Neurocomputing 72:2249–2258

    Article  Google Scholar 

  4. Chen WJ, Li CN, Shao YH, Zhang J, Deng NY (2018) Robust l1-norm multi-weight vector projection support vector machine with efficient algorithm. Neurocomputing 315:345–361

    Article  Google Scholar 

  5. Joachims T (1998) Text categorization with suport vector machines: learning with many relevant features. Mach Learn 1398:137–142

    Google Scholar 

  6. Yin H, Jiao X, Chai Y, Fang B (2015) Scene classification based on single-layer sae and svm. Expert Syst Appl 42(7):3368–3380

    Article  Google Scholar 

  7. Dniz O, Castrilln M, Hernndez M (2003) Face recognition using independent component analysis and support vector machines. Pattern Recogn Lett 24(13):2153–2157

    Article  Google Scholar 

  8. Burges CJC (1998) A tutorial on support vector machines for pattern recognition. Data Min Knowl Disc 2(2):121–167

    Article  Google Scholar 

  9. Huang X, Shi L, Suykens JA (2014) Support vector machine classifier with pinball loss. IEEE Trans Pattern Analys Mach Intell 36(5):984–97

    Article  Google Scholar 

  10. Gupta U, Gupta D (2019) An improved regularization based Lagrangian asymmetric -twin support vector regression using pinball loss function. Appl Intell 49(10):3606–3627

    Article  Google Scholar 

  11. Gupta D, Gupta U (2021) On robust asymmetric Lagrangian -twin support vector regression using pinball loss function. Appl Soft Comput 102:107099

    Article  Google Scholar 

  12. Feng R, Xu Y (2022) Support matrix machine with pinball loss for classification. Neural Comput Appl 34(21):18643–18661

    Article  Google Scholar 

  13. Reshma R, Khemchandani R, Pal A, Chandra S (2018) Generalized pinball loss svms. Neurocomputing 36(5):322

    Google Scholar 

  14. Huang X, Shi L, Johan AK (2015) SuykensSequential minimal optimization for SVM with pinball loss. Neurocomputing 149:1596–1603. https://doi.org/10.1016/j.neucom.2014.08.033

    Article  Google Scholar 

  15. Wang J, Wonka P, Ye J (2014) Scaling SVM and least absolute deviations via exact data reduction. In: Proceedings of the 31st International Conference on Machine Learning, PMLR. 32(2), 523-531

  16. Vaněk J, Michálek J, Psutka J (2017) A GPU-architecture optimized hierarchical decomposition algorithm for support vector machine training. IEEE Trans Parallel Distrib Syst 28(12):3330–3343. https://doi.org/10.1109/TPDS.2017.2731764

    Article  Google Scholar 

  17. Shalev-Shwartz S, Singer Y, Srebro N (2011) Pegasos: primal estimated sub-gradient solver for SVM. Math Program 127:3–30. https://doi.org/10.1007/s10107-010-0420-4

    Article  MathSciNet  MATH  Google Scholar 

  18. Wang Z, Shao YH, Bai L, Li C, Liu L, Denge N (2018) Insensitive stochastic gradient twin support vector machines for large scale problems. Inf Sci 462(1):14–31

    MathSciNet  Google Scholar 

  19. Panup W, Wangkeeree R (2021) Stochastic subgradient for large-scale support vector machine using the generalized pinball loss function. Symmetry 13(9):1652. https://doi.org/10.3390/sym13091652

    Article  Google Scholar 

  20. Arun Kumar M, Gopal M (2008) Application of smoothing technique on twin support vector machines. Pattern Recognit Lett 29(13):1842–1848. https://doi.org/10.1016/j.patrec.2008.05.016

    Article  Google Scholar 

  21. Li K, Lv Z (2021) Smooth twin bounded support vector machine with pinball loss. Appl Intell 51:5489–5505

    Article  Google Scholar 

  22. Xiong J, Hu T, Li G, Peng H (2006) A comparative study of three smooth SVM classifiers. In: 2006 6th World Congress on Intelligent Control and Automation. 5962–5966, doi: https://doi.org/10.1109/WCICA.2006.1714223

  23. Qin C, Liu S (2012) Fuzzy smooth support vector machine with different smooth functions. J Syst Eng Electron 23(3):460–466. https://doi.org/10.1109/JSEE.2012.00058

    Article  Google Scholar 

  24. Sharma S, Rastogi R, Chandra S (2021) Large-scale twin parametric support vector machine using pinball loss function. IEEE Trans Syst, Man, and Cyberne: Syst 51(2):987–1003

    Article  Google Scholar 

  25. Liang ZL, Zhang L (2021) Support vector machines with the \(\epsilon\)-insensitive pinball loss function for uncertain data classification. Neurocomputing 457:117–127. https://doi.org/10.1016/j.neucom.2021.06.044

    Article  Google Scholar 

  26. Lee Y, Mangasarian OL (2001) SSVM: a smooth support vector machine for classification. Comput Optim Appl 20(1):5–22

    Article  MathSciNet  MATH  Google Scholar 

  27. Ding S, Huang H, Xu X, Wang J (2014) Polynomial smooth twin support vector machines. Appl Math, Inf Sci 8:2063–2071

    Article  MathSciNet  Google Scholar 

  28. Tanveer M (2017) Smooth twin support vector machines via unconstrained convex minimization. Filomat 31:2195–2210

    Article  MathSciNet  MATH  Google Scholar 

  29. Liu MZ, Shao YH, Li CN, Chen WJ (2021) Smooth pinball loss nonparallel support vector machine for robust classification. Appl Soft Comput 98:106840

    Article  Google Scholar 

  30. Ratiphaphongthon W, Panup W, Wangkeeree R (2022) An improved technique for pneumonia infected patients image recognition based on combination algorithm of smooth generalized pinball SVM and variational autoencoders. IEEE Access 10:107431–107445. https://doi.org/10.1109/ACCESS.2022.3212535

    Article  Google Scholar 

  31. Panup W, Ratipapongton W, Wangkeeree R (2022) A novel twin support vector machine with generalized pinball loss function for pattern classification. Symmetry 14(2):289

    Article  Google Scholar 

  32. Ralph Tyrell Rockafellar, Roger J-B Wets.: Variational analysis. Springer. (1998)

  33. Proximite et dualtité dans un espace Hilbertien (1965) Jean J. Moreau. Búlletin de la Société Mathematique de Francé. 93:273–299

    Google Scholar 

  34. Boyd S, Vandenberghe L (2004) Convex optimization. Cambridge University Press. http://stanford.edu/boyd/cvxbook/

  35. Nocedal J, Wright SJ (1999) Numerical optimization, 2nd edn. Springer-Verlag, New York, NY

    Book  MATH  Google Scholar 

  36. Bache K, Lichman M (2013) UCI machine learning repository. [Online]. Available: http://archive.ics.uci.edu/ml (1990):92

  37. Pedregosa et al (2011) Scikit-learn: machine learning in Python. J Mach Learn Res 12:2825–2830

    MathSciNet  MATH  Google Scholar 

  38. Hsu CW, Chang CC, Lin CJ (2003) A practical guide to support vector classification. Nat. Taiwan Univ, Taipei, Taiwan, pp 1–12

    Google Scholar 

  39. Garcia S, Fernandez A, Luengo J, Herrera F (2010) Advanced non- parametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining experimental analysis of power. Inf Sci 180:2044–2064

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the Office of National Higher Education Science Research and Innovation Policy and Naresuan University, Thailand, with Grant Number B05F640180.

Funding

This research was funded by Office of National Higher Education Science Research and Innovation Policy and Naresuan University, Thailand, with Grant Number B05F640180.

Author information

Author notes

  1. Wachiraphong Ratiphaphongthon and Rabian Wangkeeree are contributed equally to this work.

Authors and Affiliations

  1. Department of Mathematics, Faculty of Science, Naresuan University, Phitsanulok, 65000, Thailand

    Dawrawee Makmuang & Rabian Wangkeeree

  2. Department of Mathematics, University of York, Heslington, York, YO10 5DD, UK

    Wachiraphong Ratiphaphongthon

  3. Research center for Academic Excellence in Mathematics, Naresuan University, Phitsanulok, 65000, Thailand

    Rabian Wangkeeree

Authors
  1. Dawrawee Makmuang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wachiraphong Ratiphaphongthon
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rabian Wangkeeree
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Authors’ contributions Dawrawee Makmuang constructed the model, analyzed its properties, and wrote the code. WR tested the model and performed the numerical experiments. RW reviewed and edited the original draft. The published version of the manuscript has been read and approved by all authors.

Corresponding author

Correspondence to Rabian Wangkeeree.

Ethics declarations

Conflict of interest

The authors have no conflicts of interest to declare that are relevant to the content of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Makmuang, D., Ratiphaphongthon, W. & Wangkeeree, R. Smooth support vector machine with generalized pinball loss for Pattern Classification. J Supercomput 79, 11684–11706 (2023). https://doi.org/10.1007/s11227-023-05082-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-023-05082-w

Keywords

Search

Navigation