Skip to main content

Advertisement

SpringerLink
Log in

A multi-objective genetic algorithm for simultaneous model and feature selection for support vector machines

  • Published:
Artificial Intelligence Review Aims and scope Submit manuscript

Abstract

The Support Vector Machines (SVM) constitute a very powerful technique for pattern classification problems. However, its efficiency in practice depends highly on the selection of the kernel function type and relevant parameter values. Selecting relevant features is another factor that can also impact the performance of SVM. The identification of the best set of parameters values for a classification model such as SVM is considered as an optimization problem. Thus, in this paper, we aim to simultaneously optimize SVMs parameters and feature subset using different kernel functions. We cast this problem as a multi-objective optimization problem, where the classification accuracy, the number of support vectors, the margin and the number of selected features define our objective functions. To solve this optimization problem, a method based on multi-objective genetic algorithm NSGA-II is suggested. A multi-criteria selection operator for our NSGA-II is also introduced. The proposed method is tested on some benchmark data-sets. The experimental results show the efficiency of the proposed method where features were reduced and the classification accuracy has been improved.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. available at http://www.csie.ntu.edu.tw/~cjlin/.

  2. available at http://archive.ics.uci.edu/ml/.

References

  • Andy, Fernando M, Halim K, Sanjaya G (2015) Optimization features using GA-SVM approach. Int J Sci Res 4(9):193–197

  • Balakrishnan S, Narayanaswamy R (2009) Feature selection using fcbf in type ii diabetes databases. In: International conference on IT to celebrate S Charmonman’s 72nd Birthday, Thailand

  • Chapelle O, Vapnik V, Bousquet O, Mukherjee S (2002) Choosing multiple parameters for support vector machines. Mach Learn 46(1):131–159

    Article  MATH  Google Scholar 

  • Chen Z, Lin T, Tang N, Xia X (2016) A parallel genetic algorithm based feature selection and parameter optimization for support vector machines. Sci Program 2016:2739621

  • Chung K, Kao W, Sun C, Lin C (2003) Radius margin bounds for support vector machines with RBF kernel. Neural Comput 15(11):2643–2681

    Article  MATH  Google Scholar 

  • Deb K, Pratap A, Agarwal S, Meyarivan T (2002) A fast and elitist multi-objective genetic algorithm: NSGA-II. IEEE Trans Evol Comput 6(2):182–197

    Article  Google Scholar 

  • Deb K, Beyer H (1999) Self-adaptive genetic algorithms with simulated binary crossover. Complex Syst 9:431–454

    Google Scholar 

  • Deb K, Tiwari S (2008) Omni-optimizer: a generic evolutionary algorithm for single and multiobjective optimization. Eur J Oper Res 185(3):1062–1087

    Article  MATH  Google Scholar 

  • Frohlich H, Chapelle O (2003) Feature selection for support vector machines by means of genetic algorithms. In: Proceedings of the 15th IEEE international conference on tools with artificial intelligence, Sacramento, CA, USA, pp 142–148

  • Guyon I, Weston J, Barnhill S, Vapnik V (2002) Gene selection for cancer classification using support vector machines. Mach Learn 46(1–3):389–422

    Article  MATH  Google Scholar 

  • Guyon I, Gunn S, Nikravesh M, Zadeh L (2006) Feature extraction, foundations and applications. Springer, Berlin

    Book  MATH  Google Scholar 

  • Guyon I, Elisseeff A (2003) An introduction to variable and feature selection. J Mach Learn Res 3:1157–1182

    MATH  Google Scholar 

  • Hsu C, Lin C (2002) A comparison of methods for multiclass support vector machines. IEEE Trans Neural Netw 13(2):415–425

    Article  Google Scholar 

  • Huang C, Wang C (2006) A ga-based feature selection and parameters optimization for support vector machine. Expert Syst Appl 31(2):231–240

    Article  Google Scholar 

  • Keerthi S, Lin C (2003) Asymptotic behaviors of support vector machines with gaussian kernel. Neural Comput 15:1667–1689

    Article  MATH  Google Scholar 

  • Kotsia I, Pitas I (2007) Facial expression recognition in image sequences using geometric deformation features and support vector machines. IEEE Trans Image Process 16(1):172–187

    Article  MathSciNet  Google Scholar 

  • Lee M (2009) Using support vector machine with a hybrid feature selection method to the stock trend prediction. Expert Syst Appl 36:10896–10904

    Article  Google Scholar 

  • Liang X, Liu F (2002) Choosing multiple parameters for svm based on genetic algorithm. In: 6th international conference on signal processing, vol 1, pp 117–119

  • Liao P, Zhang X, Li K, Fu Y, Wang M, Wang S (2015) Parameter optimization for support vector machine based on nested genetic algorithms. J Autom Control Eng 3(6):507–511

    Article  Google Scholar 

  • Lin K-C, Huang Y-H, Hung J-C, Lin Y-T (2015) Feature selection and parameter optimization of support vector machines based on modified cat swarm optimization. Int J Distrib Sens Netw 2015:365869

  • Lin S, Ying K, Chen S, Lee Z (2008) Particle swarm optimization for parameter determination and feature selection of support vector machines. Expert Syst Appl 35(4):1817–1824

    Article  Google Scholar 

  • Liu H, Wang Y, Lu X (2005a) A method to choose kernel function and its parameters for support vector machines. Proc IEEE Int Conf Mach Learn Cybern 7:4277–4280

    Google Scholar 

  • Liu S, Jia C, Ma H (2005b) A new weighted support vector machine with ga-based parameter selection. In: Proceedings of 2005 international conference on machine learning and cybernetics, vol 7, pp 4351–4355

  • Ma J, Nguyen M, Rajapakse J (2009) Gene classification using codon usage and support vector machines. ACM Trans Comput Biol Bioinf 6(1):134–143

    Article  Google Scholar 

  • Mao K (2004) Feature subset selection for support vector machines through discriminative function pruning analysis. IEEE Trans Syst Man Cybern 34(1):60–67

    Article  Google Scholar 

  • Melgani F, Bazi Y (2008) Classification of electrocardiogram signals with support vector machines and swarm optimization. IEEE Trans Inf Technol Biomed 12(5):667–677

    Article  Google Scholar 

  • Pardo M, Sberveglieri G (2005) Classification of electronic nose data with support vector machines. Sens Actuators B Chem 107:730–737

  • Plat J (1999) Fast training of support vector machine using sequential minimal optimization. In: Smola A (ed) Advances in kernel methods? Support vector learning. MIT Press, Cambridge

    Google Scholar 

  • Quang A, Zhang Q, Li X (2002) Evolving support vector machine parameters. In: Proceedings of 2002 international conference on machine learning and cybernetics, pp 548–551

  • Raymer M, Punch W, Goodman E, Kuhn L, Jain A (2000) Dimensionality reduction using genetic algorithms. IEEE Trans Evol Comput 4(2):164–171

    Article  Google Scholar 

  • Rinaldia F, Sciandronea M (2010) Feature selection combining linear support vector machines and concave optimization. Optim Methods Softw 25(1):117–128

    Article  MathSciNet  Google Scholar 

  • Rojas S, Fernandez-Reyes D (2005) Adapting multiple kernel parameters for support vector machines using genetic algorithms. In: The 2005 IEEE Congress on Evolutionary Computation, vol 1, pp 626–631

  • Thadani K, Jayaraman V, Sundararajan V (2006) Evolutionary selection of kernels in support vector machines. In: Proceedings of IEEE international conference on advanced computing and communications, pp 19–24

  • Vapnik V (1995) The nature of statistical learning theory. Springer, New York

    Book  MATH  Google Scholar 

  • Wu CH, Tzeng GH, Lin RH (2009) A Novel hybrid genetic algorithm for kernel function and parameter optimization in support vector regression. Expert Syst Appl 36:4725–4735

    Article  Google Scholar 

  • Xue B, Zhang M, Browne W-N, Yao X (2016) A survey on evolutionary computation approaches to feature selection. IEEE Trans Evol Comput 20(4):606–626

  • Yang J, Honavar V (1998) Feature subset selection using a genetic algorithm. IEEE Intell Syst 13(2):44–49

    Article  Google Scholar 

  • Yu L, Chen H, Wang S, Lai K (2009) Evolving least squares support vector machines for stock market trend mining. IEEE Trans Evol Comput 13(1):87–102

    Article  Google Scholar 

  • Yuan FC (2012) Parameters optimization using genetic algorithms in support vector regression for sales volume forecasting. Appl Math 3:1480–1486

    Article  Google Scholar 

  • Yuxia H, Hongtao Z (2012) Chaos optimization method of SVM parameters selection for chaotic time series forecasting. Phys Proc 25(2012):588–594 2012 international conference on solid state devices and materials science

    Article  Google Scholar 

  • Zhang G (2000) Neural network for classification: a survey. IEEE Trans Syst Man Cybern Part C Appl Rev 30(4):451–462

    Article  Google Scholar 

  • Zhao M, Fu C, Ji L, Tang K, Zhou M (2011) Feature selection and parameter optimization for support vector machines: a new approach based on genetic algorithm with feature chromosomes. Expert Syst Appl 38(5):5197–5204

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Multimedia, InfoRmation systems and Advanced Computing Laboratory MIRACL-Sfax University, ISIMS, Sfax-Tunisia Technopole of Sfax: Av.Tunis Km 10, B.P. 242, 3021, Sfax, Tunisia

    Amal Bouraoui, Salma Jamoussi & Yassine BenAyed

Authors
  1. Amal Bouraoui
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Salma Jamoussi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yassine BenAyed
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Amal Bouraoui.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bouraoui, A., Jamoussi, S. & BenAyed, Y. A multi-objective genetic algorithm for simultaneous model and feature selection for support vector machines. Artif Intell Rev 50, 261–281 (2018). https://doi.org/10.1007/s10462-017-9543-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10462-017-9543-9

Keywords

Search

Navigation