Jonghyok Jang
ABSTRACT
The performance of support vector machine(SVM) highly depends on selection of hyperparameters(kernel function parameters and penalty parameter). Currently selection methods of hyperparameters in nonlinear SVMs tend to fall into local optimized solution or rather is time consuming. Taking account of this respect, we propose a new automatic selection method of hyperparameters of nonlinear support vector machines. The theoretical analysis for the appropriate selection of hyperparameters is conducted by Bayesian inference. On the basis of this analysis, we determine the parameter by using Markov chain Monte Carlo(MCMC) algorithm. By using the determined parameters, we construct a classifier with low complexity of recognizing computation. Experimental results from various databases and the evaluation of toxicity in water samples demonstrate that the proposed method provides high classification performance.
- V. Vapnik, The Nature of Statistical Learning Theory, Springer Verlag, New York, 1995.Google Scholar
Cross Ref
- Jianfeng Wang,Lei Zhang, “A parameter optimization method for an SVM based on improved grid search algorithm,” Applied Science and Technology, Vol.39,No.3 ,2012.Google Scholar
- S. Keerthi, V. Sindhwani, O. Chapelle, An efficient method for gradient-based adaptation of hyperparameters in SVM models, Adv. Neural Inf. Process. Syst.19 (2007) 673–680.Google Scholar
- S.T. Li, M.K. Tan, S.V.M. Tuning, parameters by using a hybrid CLPSO–BFGS algorithm, Neurocomputing 73 (2010) 2089–2096.Google ScholarDigital Library
- C.-L. Huang, ACO-based hybrid classification system with feature subset selection and model parameters optimization, Neurocomputing 73 (2009) 438–448.Google ScholarDigital Library
- X.L. Zhang, X.F. Chen, Z.J. He, An ACO-based algorithm for parameter optimization of support vector machines, Expert Syst. Appl. 37 (2010) 6618–6628.Google ScholarDigital Library
- İ. İlhan, G. Tezel, A genetic algorithm-support vector machine method with parameter optimization for selecting the tag SNPs, J. Biomed. Inf. 46 (2013) 328–340.Google ScholarDigital Library
- S.W. Lin, K.C. Ying, S.C. Chen, Z.J. Lee, Particle swarm optimization for parameter determination and feature selection of support vector machines, Expert Syst. Appl. 35 (2008) 1817–1824.Google ScholarDigital Library
- X.-l. Li, L.-h. Li, B.-l. Zhang, Q.-j. Guo, Hybrid self-adaptive learning based particle swarm optimization and support vector regression model for grade estimation, Neurocomputing 118 (2013) 179–190.Google ScholarDigital Library
- Y. Bao, Z. Hu, T. Xiong, A PSO and pattern search based memetic algorithm for SVMs parameters optimization, Neurocomputing 117 (2013) 98–106.Google ScholarCross Ref
- S. W. Lin, "Parameter determination of support vector machine and f eature selection using simulated annealing approach", Applied Soft Computin g, 8(4):1505-1512, 2008.Google ScholarDigital Library
- T.Van Gestel Bayesian Framework for Least-Squares Support Vector Machine Classifiers, Gaussian Processes, and Kernel Fisher Discriminant Analysis [J]. Neural Computation 14 1115-1147. 2002.Google ScholarDigital Library
- Polson, N.G., Scott, S.L.: Data augmentation for support vector machines. Bayesian Anal, 2011.Google Scholar
- Shounak Datta, Swagatam Das, "Near-Bayesian Support Vector Machines for imbalanced data classification with equal or unequal misclassification costs", Neural Networks, vol.70, 39–52, 2015.Google ScholarDigital Library
- L. Sun, K.-A. Toh and Z. Lin, “A center sliding Bayesian binary classifier adopting orthogonal polynomials”, Pattern Recognition, vol. 48, no. 6, pp. 2013-2028, 2015.Google ScholarDigital Library
- Henao, R., Yuan, X., Carin, L.: Bayesian Nonlinear Support Vector Machines and Discriminative Factor Modeling. NIPS (2017)Google Scholar
- Polson, N.G., Scott, S.L.: Data augmentation for support vector machines. B ayesian Anal. 2011.Google Scholar
- Carl Edward Rasmussen, " Adaptive stellar spectral subclass classification based on Bayesian SVMs", New Astronomy 51 (2017) 51–58.Google ScholarCross Ref
- Henao, R., Yuan, X., Carin, L.: Bayesian Nonlinear Support Vector Machine s and Discriminative Factor Modeling. NIPS 2014.Google Scholar
- Marin, J. M., and C. P. Robert. Bayesian Core: A Practical Approach to Computati onal Bayesian Statistics. New York: Springer Science+Business Media, LLC, 2007.Google Scholar
- Gareth O. Roberts, "General state space Markov chains and MCMC algorithms" [J]. Probability Surveys 20, No 7, 20–71. 2004.Google Scholar
- I. Murray and R. P. Adams. "Slice sampling covariance hyperparameters of latent Gaussian models" NIPS, 23, pp 1723–1731, 2010.Google Scholar
- Kevin P. Mur phy, "Machine Learning", 864∼866, MIT press 2012.Google Scholar
- Adam Bownik, “Daphnia swimming behaviour as a biomarker in toxicity assessment”, Science of the Total Environment 601 –602 (2017) 194 –205.Google ScholarCross Ref
Index Terms
- Selecting Hyperparameters of Nonlinear Support Vector Machine Using Bayesian Inference
Recommendations
Mean field variational Bayesian inference for support vector machine classification
A mean field variational Bayes approach to support vector machines (SVMs) using the latent variable representation on Polson and Scott (2012) is presented. This representation allows circumvention of many of the shortcomings associated with classical ...
Simulation-based Bayesian inference for epidemic models
A powerful and flexible method for fitting dynamic models to missing and censored data is to use the Bayesian paradigm via data-augmented Markov chain Monte Carlo (DA-MCMC). This samples from the joint posterior for the parameters and missing data, but ...
Bayesian learning of Bayesian networks with informative priors
This paper presents and evaluates an approach to Bayesian model averaging where the models are Bayesian nets (BNs). A comprehensive study of the literature on structural priors for BNs is conducted. A number of prior distributions are defined using ...
Comments