Abstract
The generalization capacity of support vector machine (SVM) depends largely on the selection of kernel function and its parameters, and penalty factor, which is regarded as model selection of SVM. When various forms of differentiable and loose generalization bounds are considered as the objective functions, the traditional optimization algorithms easily fall into the local optimal solutions, whereas the modern techniques difficultly find out really optimal ones. Recently the empirical error criterion on a validation set is used as a new objective function which is optimized by the classical optimization methods. In this paper, we propose a new SVM model selection based on hybrid genetic algorithm and empirical error minimization criterion. The hybrid genetic method integrates the gradient descent method into the genetic algorithm to search for a better parameter of RBF kernel. The experiments on 13 benchmark datasets demonstrate that our method can work well on some real applications.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Vapnik, V.N.: The Nature of Statistical Learning Theory. Springer, New York (1995)
Cortes, C., Vapnik, V.N.: Support Vector Networks. Machine Learning 20(3), 273–297 (1995)
Ratsch, G., Onoda, T., Muller, K.R.: Soft Margins for AdaBoost. Machine Learning 42, 287–320 (2001)
Lee, J.H., Lin, C.J.: Automatic Model Selection for Support Vector Machines. Technical Report, Department of Computer Science and Information Engineering, National Taiwan University (2000)
Chapelle, O., Vapnik, V.N., Bousquet, O., Mukherjee, S.: Choosing Multiple Parameters for Support Vector Machines. Machine Learning 46(1-3), 131–159 (2002)
Keerthi, S.S.: Efficient Tuning of SVM Hyperparameters Using Radius Margin Bound and Iterative Algorithms. IEEE Trans. on Neural Networks 13, 1225–1229 (2002)
Duan, K., Keerthi, S.S., Poo, A.N.: Evaluation of Simple Performance Measures for Tuning SVM Hyperparameters. Neurocomputing 51, 41–59 (2003)
Chung, K.M., Kao, W.C., Sun, C.L., Wang, L.L., Lin, C.J.: Radius Margin Bounds for Support Vector Machines with the RBF Kernel. Neural Computation 15, 2643–2681 (2003)
Ayat, N.E., Cheriet, M., Suen, C.Y.: Optimization of the SVM Kernels Using an Empirical Error Minimization Scheme. In: Lee, S.W., Verri, A. (eds.) SVM 2002. LNCS, vol. 2388, pp. 354–369. Springer, Heidelberg (2002)
Adankon, M.M., Cheriet, M., Ayat, N.E.: Optimizing Resources in Model Selection for Support Vector Machines. In: 2005 International Joint Conference on Neural Networks, pp. 925–930. IEEE Press, Montreal (2005)
Ayat, N.E., Cheriet, M., Suen, C.Y.: Automatic Model Selection for the Optimization of the SVM Kernels. Pattern Recognition 38, 1733–1745 (2005)
Adankon, M.M., Cheriet, M.: New Formulation of SVM for Model Selection. In: 2006 International Joint Conference on Neural Networks, pp. 1900–1907. IEEE Press, Vancouver (2006)
Zheng, C.H., Li, C.J.: Automatic Parameters Selection for SVM Based on GA. In: 5th World Congress on Intelligent Control and Automation, pp. 1869–1872. IEEE Press, Hangzhou (2004)
Javier, A., Saturnino, M., Philip, S.: Tuning L1-SVM Hyperparameters with Modified Radius Margin Bounds and Simulated Annealing. In: Sandoval, F., Prieto, A.G., Cabestany, J., Graña, M. (eds.) IWANN 2007. LNCS, vol. 4507, pp. 284–291. Springer, Heidelberg (2007)
Platt, J.: Probabilistic Outputs for Support Vector Machines and Comparisons to Regula-rized Likelihood Methods. In: Bartlett, P.J., Scholkopf, B., Schuurmans, D., Smola, A.J. (eds.) Advances in large margin classifiers, pp. 67–74. MIT Press, Cambridge (1999)
Ratsch, G.: Benchmark data sets, http://ida.first.fhg.de/projects/bench/benchmarks.htm
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Zhou, X., Xu, J. (2009). A SVM Model Selection Method Based on Hybrid Genetic Algorithm and Empirical Error Minimization Criterion. In: Wang, H., Shen, Y., Huang, T., Zeng, Z. (eds) The Sixth International Symposium on Neural Networks (ISNN 2009). Advances in Intelligent and Soft Computing, vol 56. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-01216-7_26
Download citation
DOI: https://doi.org/10.1007/978-3-642-01216-7_26
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-01215-0
Online ISBN: 978-3-642-01216-7
eBook Packages: EngineeringEngineering (R0)