Skip to main content

Advertisement

SpringerLink
Log in

Effective Parameter Tuning of SVMs Using Radius/Margin Bound Through Data Envelopment Analysis

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

This paper presents parameter tuning of multi-class support vector machines (SVMs) using radius–margin bound through data envelopment analysis (DEA). Relative efficiencies of the binary SVMs in one-vs-one multi-class SVM are obtained using radius as the input and margin as the output by employing Charnes, Cooper and Rhodes model (CCR model). The objective is to maximize the sum of the relative efficiencies of individual binary SVMs in one-vs-one multi-class SVM. The proposed formulation is empirically found to be outperforming Keerthi’s radius–margin based parameter tuning. In the best case, the radius obtained through the proposed method is 1,110 times lower than Keerthi’s method. Margin is 4.4 times higher, the number of support vectors are 5.8 times lower and test error rates are competitive. Proposed method is also compared with (i) Widely employed grid search method and (ii) Parameter tuning method proposed in (IEEE Trans Mach Intell 32:1888–1898, 2010. The merit of the proposed method over these tuning methods is shown through experimentation along the four identified dimensions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. All the input and outputs are assumed to be positive.

  2. We thank the authors of the paper [8] for sharing their code through their webpage. The code can be downloaded at:www.dsp.rice.edu/software.

References

  1. Allwein EL, Schapire RE, Singer Y (2000) Reducing multiclass to binary: a unifying approach for margin classifiers. J Mach Learn Res 1:113–141

    MathSciNet  Google Scholar 

  2. Asuncion A, Newman D (2007) UCI machine learning repository. http://www.ics.uci.edu/mlearn/MLRepository.html

  3. Burges CJC (1998) A tutorial on support vector machines for pattern recognition. Data Min Knowl Discov 2(2):121–167

    Article  Google Scholar 

  4. Chapelle O, Vapnik V, Bousquet O, Mukherjee S (2002) Choosing multiple parameters for support vector machines. Mach Learn 46:131–159

    Article  MATH  Google Scholar 

  5. Charnes A, Cooper W, Rhodes E (1978) Measuring the efficiency of decision making units. Eur J Oper Res 2(6):429–444. doi:10.1016/0377-2217(78)90138-8

    Google Scholar 

  6. Cooper WW, Seiford LM, Tone K (2000) Data envelopment analysis: a comprehensive text with models, applications, references and DEA-solver softwares. Kluwer Academic, New York

    Google Scholar 

  7. Cortes C, Vapnik VN (1995) Support vector networks. Mach Learn 20:273–297

    MATH  Google Scholar 

  8. Davenport MA, Baraniuk RG, Scott CD (2010) Tuning support vector machines for minimax and Neyman–Pearson classification. IEEE Trans Pattern Anal Mach Intell 32(10):1888–1898

    Article  Google Scholar 

  9. Dietterich TG, Bakiri G (1995) Solving multiclass learning problems via error-correcting output codes. J Artif Intell Res 2:263–286

    MATH  Google Scholar 

  10. Friedman J (1996) Another approach to polychotomous classification. Online technical report. Stanford University, Stanford

    Google Scholar 

  11. Friedrichs F, Igel C (2004) Evolutionary tuning of multiple SVM parameters. Neurocomputing 64:3326–3334

    Google Scholar 

  12. Fürnkranz J (2002) Round robin classification. J Mach Learn Res 2:721–747

    MATH  MathSciNet  Google Scholar 

  13. Goldberg DE (1989) Genetic algorithms in search optimization and machine learning. Addison-Wesley, Bosto

    MATH  Google Scholar 

  14. Hsu CW, Lin CJ (2002) A comparison of methods for multi-class support vector machines. IEEE Trans Neural Netw 13:415–425

    Article  Google Scholar 

  15. Hsu CW, Chang CC, Lin CJ (2010) A practical guide to support vector classification. Bioinformatics 1:1–16

    Google Scholar 

  16. Joachims T (2000) The maximum margin approach to learning text classifiers: methods, theory, and algorithms. PhD thesis, Universität Dortmund, Informatik, LS VIII.

  17. Keerthi S (2002) Efficient tuning of SVM hyperparameters using radius/margin bound and iterative algorithms. Neural Netw IEEE Trans 13(5):1225–1229. doi:10.1109/TNN.2002.1031955

    Article  Google Scholar 

  18. Lanckriet GRG, Cristianini N, Bartlett P, Ghaoui LE, Jordan MI (2004) Learning the kernel matrix with semidefinite programming. J Mach Learn Res 5:27–72

    MATH  Google Scholar 

  19. Lee Y, Lin Y, Wahba G (2001a) Multicategory support vector machines. Tech. rep., Department of Statistics, University of Wisconsin.

  20. Lee Y, Lin Y, Wahba G (2001b) Multicategory support vector machines. In The 33rd symposium on the Interface.

  21. Lins MPE, Gomes EG, de Mello JCCS, de Mello AJRS (2003) Olympic ranking based on a zero sum gains DEA model. Eur J Oper Res 148:312–322

    Article  MATH  Google Scholar 

  22. Lorena AC, de Carvalho AC (2008) Evolutionary tuning of SVM parameter values in multiclass problems. Neurocomputing 71:3326–3334

    Article  Google Scholar 

  23. Murphy PM, Aha DW (1993) UCI repository of machine learning databases (machine-readable data repository). Tech. rep., Department of Information and Computer Science, University of California, Irvine. http://www.ics.uci.edu/~mlearn/MLRepository.html

  24. Saha A, Ravisankar TS (2000) Rating of Indian commercial banks: a DEA approach. Eur J Oper Res 124:187–203

    Article  MATH  Google Scholar 

  25. Schölkopf B, Burges C, Vapnik V (1995) Extracting support data for a given task. In First international conference on knowledge discovery and data mining, New York.

  26. Shanno DF, Phu KH (1980) Minimization of unconstrained multivariate functions. ACM Trans Math Softwar 2(1):87–94

    Article  Google Scholar 

  27. Staub RB, da Silva e Souza G, Tabak BM (2010) Evolution of bank efficiency in Brazil: a DEA approach. Eur J Oper Res 202:204–213

    Article  MATH  Google Scholar 

  28. Stensrud E, Myrtveit I (2003) Identifying high performance ERP projects. IEEE Trans Softw Eng 29(5):398–416

    Article  Google Scholar 

  29. Valentini G, Dietterich TG (2006) Bias-variance analysis of support vector machines for the development of SVM-based ensemble methods. J Mach Learn Res 5:725–775

    MathSciNet  Google Scholar 

  30. Vapnik V, Chapell O (2000) Bounds on error expectation for support vector machines. Neural Comput 12:2013–2036. doi:10.1162/089976600300015042

    Article  Google Scholar 

  31. Vapnik VN (1998) Stat Learn Theory. Wiley, New York

    Google Scholar 

  32. Weston J (1999) Multi-class support vector machines. In Proceedings of ESANN, Brussels D. Facto Press.

  33. Wu J, Liang L, Yang F (2009) Achievement and benchmarking of countries at the summer olympics using cross efficiency evaluation method. Eur J Oper Res 197:722–730

    Article  MATH  Google Scholar 

  34. Zhao X, Wang S, Lai KK (2011) Mutual funds performance evaluation based on endogenous benchmarks. Exp Syst Appl 38:3663–3670

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Indian Institute of Technology Guwahati, Guwahati, Assam, India

    V. Vijaya Saradhi

  2. Dell R & D, Enterprise Solutions Group Bangalore, Bangalore, India

    K. R. Girish

Authors
  1. V. Vijaya Saradhi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. K. R. Girish
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to V. Vijaya Saradhi.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Vijaya Saradhi, V., Girish, K.R. Effective Parameter Tuning of SVMs Using Radius/Margin Bound Through Data Envelopment Analysis. Neural Process Lett 41, 125–138 (2015). https://doi.org/10.1007/s11063-014-9338-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-014-9338-9

Keywords

Search

Navigation