Skip to main content
SpringerLink
Log in

Robust kernel-based multiclass support vector machines via second-order cone programming

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Kernel methods are very important in pattern analysis due to their ability to capture nonlinear relationships in datasets. The best known kernel-based technique is Support Vector Machine (SVM), which can be used for several pattern recognition tasks, including multiclass classification. In this paper, we focus on maximum margin classifiers for nonlinear multiclass learning, based on second-order cone programming (SOCP), proposing three novel formulations that extend the most common strategies for this task: One-vs.-The-Rest, One-vs.-One, and All-Together optimization. The proposed SOCP formulations achieved superior performance compared to their traditional SVM counterparts on benchmark datasets, demonstrating the virtues of robust optimization.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Alizadeh F, Goldfarb D (2003) Second-order cone programming. Math Progr 95:3–51

    Article  MathSciNet  MATH  Google Scholar 

  2. Alvarez F, López J, Ramírez CH (2010) Interior proximal algorithm with variable metric for second-order cone programming: applications to structural optimization and support vector machines. Optim Meth Soft 25 (6):859–881

    Article  MathSciNet  MATH  Google Scholar 

  3. Asuncion A, Newman D (2007) UCI machine learning repository. http://archive.ics.uci.edu/ml/

  4. Bhattacharyya C (2004) Second order cone programming formulations for feature selection. J Mach Learn Res 5:1417–1433

    MathSciNet  MATH  Google Scholar 

  5. Bosch P, López J, Ramírez H, Robotham H (2013) Support vector machine under uncertainty: An application for hydroacoustic classification of fish-schools in Chile. Expert Syst Appl 40(10):4029–4034

    Article  Google Scholar 

  6. Bottou L, Cortes C, Denker J, Drucker H, Guyon I, Jackel L, LeCun Y, Muller U, Sackinger E, Simard P, Vapnik V (1994) Comparison of classifier methods: a case study in handwritten digit recognition. In: Proceedings of international conference on pattern recognition, vol 2, pp 77-82

  7. Bravo C, Thomas L, Weber R (2014) Improving credit scoring by differentiating defaulter behaviour. J Oper Res Soc 66:771–781

    Article  Google Scholar 

  8. Bredensteiner EJ, Bennett KP (1999) Multicategory classification by support vector machines. Comp Optim Appl 12:53–79

    Article  MathSciNet  MATH  Google Scholar 

  9. Debnath R, Muramatsu M, Takahashi H (2005) An efficient support vector machine learning method with second-order cone programming for large-scale problems. Appl Intell 23(3):219–239

    Article  MATH  Google Scholar 

  10. Djuric N, Lan L, Vucetic S, Wang Z (2013) Budgetedsvm: a toolbox for scalable svm approximations. J Mach Learn Res 14:3813–3817

    MathSciNet  MATH  Google Scholar 

  11. Friedman J (1996) Another approach to polychotomous classification. Tech. rep., Department of Statistics, Stanford University, http://www-stat.stanford.edu/~jhf/ftp/poly.ps.Z

  12. Geng X, Zhan DC, Zhou ZH (2005) Supervised nonlinear dimensionality reduction for visualization and classification. IEEE Transactions on Systems, Man, and Cybernetics. Part B Cybernetics 35(6):1098–1107

    Article  Google Scholar 

  13. Hao PY, Chiang JH, Lin YH (2009) A new maximal-margin spherical-structured multi-class support vector machine. Appl Intell 30:98–111

    Article  Google Scholar 

  14. Hastie T, Tibshirani R, Friedman J (2008) The elements of statistical learning, Springer, chap 4.3

  15. Hsu C, Lin C (2002) A comparison of methods for multiclass support vector machines. IEEE Trans Neural Netw 13(2):415–425

    Article  Google Scholar 

  16. Jayadeva, Khemchandani R, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Transactions on Pattern Analysis and Machine Intelligence 29(5):905–910

  17. Kressel UG (1999) Advances in kernel methods, MIT Press, Cambridge, MA, USA, chap Pairwise classification and support vector machines:255–268

  18. Lanckriet G, Ghaoui L, Bhattacharyya C, Jordan M (2003) A robust minimax approach to classification. J Mach Learn Res 3:555–582

    MathSciNet  MATH  Google Scholar 

  19. Lee LH, Wan C, Rajkumar R, Isa D (2012) An enhanced support vector machine classification framework by using euclidean distance function for text document categorization. Appl Intell 37(1):80–99

    Article  Google Scholar 

  20. Lin T, Liu R, Chen C, Chao Y, Chen S (2006) Pattern classification in DNA microarray data of multiple tumor types. Pattern Recogn 39(12):2426–2438

    Article  MATH  Google Scholar 

  21. López J, Maldonado S (2015) Robust feature selection for multi-class support vector machines using second-order cone programming. Intell Data Anal 19(S1):S117–S133

    Article  Google Scholar 

  22. López J, Maldonado S (2016) Multi-class second-order cone programming support vector machines. Inform Sci 330:328–341

    Article  MATH  Google Scholar 

  23. López J, Maldonado S, Carrasco M (2016) A novel multi-class svm model using second-order cone constraints. Appl Intell 44(2):457–469

    Article  Google Scholar 

  24. Maldonado S, López J (2014) Imbalanced data classification using second-order cone programming support vector machines. Pattern Recogn 47:2070–2079

    Article  MATH  Google Scholar 

  25. Maldonado S, Weber R, Basak J (2011) Kernel-penalized SVM for feature selection. Inform Sci 181 (1):115–128

    Article  Google Scholar 

  26. Mangasarian O, Musicant D (1999) Successive overrelaxation for support vector machines. IEEE Trans Neural Netw 10(5):1032–1037

    Article  Google Scholar 

  27. Mercer J (1909) Functions of positive and negative type, and their connection with the theory of integral equations. Phil Trans R Soc Lond 209:415–446

    Article  MATH  Google Scholar 

  28. Michailidis P, Margaritis K (2013) Accelerating kernel density estimation on the gpu using the cuda framework. Appl Math Sci 7(30):1447–1476

    Article  MathSciNet  Google Scholar 

  29. Nath S, Bhattacharyya C (2007) Maximum margin classifiers with specified false positive and false negative error rates. Proceedings of the SIAM International Conference on Data mining

  30. Platt J (1999) Advances in kernel Methods-Support vector learning, MIT press, cambridge, MA, chap Fast training of support vector machines using sequential minimal optimization

  31. Rifkin R, Klautau A (2004) In defense of one-vs-all classification. J Mach Learn Res 5:101–141

    MathSciNet  MATH  Google Scholar 

  32. Schölkopf B, Smola AJ (2002) Learning with Kernels. MIT Press

  33. Shivaswamy PK, Bhattacharyya C, Smola AJ (2006) Second order cone programming approaches for handling missing and uncertain data. J Mach Learn Res 7:1283–1314

    MathSciNet  MATH  Google Scholar 

  34. Sturm J (1999) Using sedumi 1.02, a matlab toolbox for optimization over symmetric cones. Optim Meth Softw 11(12):625–653

    Article  MathSciNet  MATH  Google Scholar 

  35. Tomar D, Agarwal S (2015) A comparison on multi-class classification methods based on least squares twin support vector machines. Knowl-Based Syst 81:131–147

    Article  Google Scholar 

  36. Vapnik V (1998) Statistical Learning Theory. John Wiley and Sons

  37. Wald V (1971) Statistical decision functions. Chelsea scientific books, Chelsea Pub. Co

  38. Weston J, Watkins C (1999) Multi-class support vector machines. In: Proceedings of the Seventh European Symposium on Artificial Neural Networks

  39. Weston J, Elisseeff A, BakIr G, Sinz F (2005) The spider machine learning toolbox. Software available at http://www.kyb.tuebingen.mpg.de/bs/people/spider/

  40. Xie J, Hone K, Xie W, Gao X, Shi Y, Liu X (2013) Extending twin support vector machine classifier for multi-category classification problems. Intell Data Anal 17(4):649–664

    Google Scholar 

  41. Zhong P, Fukushima M (2007) Second-order cone programming formulations for robust multiclass classification. Neural Comp 19:258–282

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgments

The first author was funded by FONDECYT project 1140831, while the second was supported by FONDECYT projects 1130905 and 1160894. The work reported in this paper has been partially funded by the Complex Engineering Systems Institute (ICM: P-05-004-F, CONICYT: FB016, www.sistemasdeingenieria.cl).

Author information

Authors and Affiliations

  1. Facultad de Ingeniería y Ciencias Aplicadas, Universidad de los Andes, Monseñor Álvaro del Portillo, Las Condes, Santiago, 12455, Chile

    Sebastián Maldonado

  2. Facultad de Ingeniería, Universidad Diego Portales, Ejército 441, Santiago, Chile

    Julio López

Authors
  1. Sebastián Maldonado
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Julio López
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sebastián Maldonado.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Maldonado, S., López, J. Robust kernel-based multiclass support vector machines via second-order cone programming. Appl Intell 46, 983–992 (2017). https://doi.org/10.1007/s10489-016-0881-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-016-0881-0

Keywords

Search

Navigation