Abstract
Support vector machine (SVM) is a popular classifier that has been used to solve a broad range of problems. Unfortunately, its applications are limited by computational complexity of training which is \(O(t^3)\), where t is the number of vectors in the training set. This limitation makes it difficult to find a proper model, especially for non-linear SVMs, where optimization of hyperparameters is needed. Nowadays, when datasets are getting bigger in terms of their size and the number of features, this issue is becoming a relevant limitation. Furthermore, with a growing number of features, there is often a problem that a lot of them may be redundant and noisy which brings down the performance of a classifier. In this paper, we address both of these issues by combining a recursive feature elimination algorithm with our evolutionary method for model and training set selection. With all of these steps, we reduce both the training and classification times of a trained classifier. We also show that the model obtained using this procedure has similar performance to that determined with other algorithms, including grid search. The results are presented over a set of well-known benchmark sets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bao, Y., Hu, Z., Xiong, T.: A PSO and pattern search based memetic algorithm for SVMS parameters optimization. Neurocomputing 117, 98–106 (2013)
Cervantes, J., Lamont, F.G., López-Chau, A., Mazahua, L.R., RuÃz, J.S.: Data selection based on decision tree for SVM classification on large data sets. Appl. Soft Comput. 37, 787–798 (2015)
Chou, J.S., Cheng, M.Y., Wu, Y.W., Pham, A.D.: Optimizing parameters of support vector machine using fast messy genetic algorithm for dispute classification. Expert Syst. Appl. 41(8), 3955–3964 (2014)
Fawcett, T.: An introduction to ROC analysis. Pattern Recogn. Lett. 27(8), 861–874 (2006)
Geurts, P., Ernst, D., Wehenkel, L.: Extremely randomized trees. Mach. Learn. 63(1), 3–42 (2006)
Ghamisi, P., Benediktsson, J.A.: Feature selection based on hybridization of genetic algorithm and particle SWARM optimization. IEEE Geosci. Remote Sens. Lett. 12(2), 309–313 (2015)
Ghamisi, P., Couceiro, M.S., Benediktsson, J.A.: A novel feature selection approach based on FODPSO and SVM. IEEE Trans. Geosci. Remote Sens. 53(5), 2935–2947 (2015)
Guo, L., Boukir, S.: Fast data selection for SVM training using ensemble margin. Pattern Recogn. Lett. 51, 112–119 (2015)
He, Q., Xie, Z., Hu, Q., Wu, C.: Neighborhood based sample and feature selection for SVM classification learning. Neurocomputing 74(10), 1585–1594 (2011)
Huang, C.L., Wang, C.J.: A GA-based feature selection and parameters optimizationfor support vector machines. Expert Syst. Appl. 31(2), 231–240 (2006)
Kawulok, M., Nalepa, J., Dudzik, W.: An alternating genetic algorithm for selecting SVM model and training set. In: Mexican Conference on Pattern Recognition, pp. 94–104. Springer (2017)
Klein, A., Falkner, S., Bartels, S., Hennig, P., Hutter, F.: Fast bayesian optimization of machine learning hyperparameters on large datasets. CoRR abs/1605.07079 (2016)
Lin, H.T., Lin, C.J., Weng, R.C.: A note on platt’s probabilistic outputs for support vector machines. Mach. Learn. 68(3), 267–276 (2007)
Lin, S.W., Ying, K.C., Chen, S.C., Lee, Z.J.: Particle swarm optimization for parameter determination and feature selection of support vector machines. Expert Syst. Appl. 35(4), 1817–1824 (2008)
Nalepa, J., Kawulok, M.: A memetic algorithm to select training data for support vector machines. In: Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation, GECCO 2014, pp. 573–580. ACM, New York (2014)
Nalepa, J., Kawulok, M.: Selecting training sets for support vector machines: a review. Artif. Intell. Rev. 1–44 (2018)
Nalepa, J., Siminski, K., Kawulok, M.: Towards parameter-less support vector machines. In: Proceedings of the ACPR, pp. 211–215 (2015)
Neumann, J., Schnörr, C., Steidl, G.: Combined SVM-based feature selection and classification. Mach. Learn. 61(1), 129–150 (2005)
Shen, X.J., Mu, L., Li, Z., Wu, H.X., Gou, J.P., Chen, X.: Large-scale support vector machine classification with redundant data reduction. Neurocomputing 172, 189–197 (2016)
Tharwat, A., Hassanien, A.E., Elnaghi, B.E.: A BA-based algorithm for parameter optimization of support vector machine. Pattern Recogn. Lett. 93, 13–22 (2017). Pattern Recognition Techniques in Data Mining
Vanek, J., Michalek, J., Psutka, J.: A GPU-architecture optimized hierarchical decomposition algorithm for support vector machine training. IEEE Trans. Parallel Distrib. Syst. 28(12), 3330–3343 (2017)
Wang, D., Shi, L.: Selecting valuable training samples for SVMs via data structure analysis. Neurocomputing 71, 2772–2781 (2008)
Wen, Z., Shi, J., He, B., Li, Q., Chen, J.: ThunderSVM: A fast SVM library on GPUs and CPUs. To appear in arxiv (2018)
Zhang, X., Qiu, D., Chen, F.: Support vector machine with parameter optimization by a novel hybrid method and its application to fault diagnosis. Neurocomputing 149, 641–651 (2015)
Acknowledgement
This work was supported by the National Science Centre under Grant DEC-2017/25/B/ST6/00474, and by the Silesian University of Technology, Poland, funds no. BKM-509/RAu2/2017.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Dudzik, W., Nalepa, J., Kawulok, M. (2019). Automated Optimization of Non-linear Support Vector Machines for Binary Classification. In: Xhafa, F., Barolli, L., Greguš, M. (eds) Advances in Intelligent Networking and Collaborative Systems. INCoS 2018. Lecture Notes on Data Engineering and Communications Technologies, vol 23. Springer, Cham. https://doi.org/10.1007/978-3-319-98557-2_47
Download citation
DOI: https://doi.org/10.1007/978-3-319-98557-2_47
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-98556-5
Online ISBN: 978-3-319-98557-2
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)