Abstract
Feature selection methods are generally divided into three categories: filter, wrapper and embedded approaches. In terms of learning performance, the filter approach is typically inferior compared to the other two because it does not use the target learning algorithm. The embedded and wrapper approaches are both considered high-performing. In this paper we compare the embedded and the wrapper approaches in the context of Support Vector Machines (SVMs). In the wrapper category, we compare well-known algorithms such as Genetic Algorithm (GA), Forward and Backward selection, and a new binary Particle Swarm Optimization (PSO) algorithm. For an embedded approach we devise a new heuristic algorithm based on Multiple Kernel Learning.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Saeys, Y., Inza, I., Larrañaga, P.: A review of feature selection techniques in bioinformatics. Bioinformatics 23(19), 2507–2517 (2007)
Pudil, P., Novovičová, J., Kittler, J.: Floating search methods in feature selection. Pattern Recogn. Lett. 15(11), 1119–1125 (1994)
Xue, B., Zhang, M., Browne, W.N., Yao, X.: A survey on evolutionary computation approaches to feature selection. IEEE Trans. Evol. Comput. 20(4), 606–626 (2016)
Tran, B., Xue, B., Zhang, M.: Overview of particle swarm optimisation for feature selection in classification. In: Dick, G., et al. (eds.) SEAL 2014. LNCS, vol. 8886, pp. 605–617. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-13563-2_51
Tran, B., Xue, B., Zhang, M.: A new representation in PSO for discretization-based feature selection. IEEE Trans. Cybern. 48(6), 1733–1746 (2018)
Xue, B., Zhang, M., Browne, W.N.: Particle swarm optimization for feature selection in classification: a multi-objective approach. IEEE Trans. Cybern. 43(6), 1656–1671 (2013)
Guyon, I., Weston, J., Barnhill, S., Vapnik, V.: Gene selection for cancer classification using support vector machines. Mach. Learn. 46(1), 389–422 (2002)
Tibshirani, R.: Regression shrinkage and selection via the lasso. J. R. Stat. Soc. (Series B) 58, 267–288 (1996)
Friedman, J., Hastie, T., Tibshirani, R.: Regularization paths for generalized linear models via coordinate descent. J. Stat. Softw. 33(1), 1–22 (2010)
Kloft, M.: lp-Norm Multiple Kernel Learning. Ph.D. thesis, Berlin Institute of Technology (2011)
Kennedy, J., Eberhart, R.C.: Particle swarm optimization. In: Proceedings of the 1995 IEEE International Conference on Neural Networks, Perth, Australia, vol. 4, pp. 1942–1948. IEEE Service Center, Piscataway (1995)
Shi, Y., Eberhart, R.C.: A modified particle swarm optimizer. In: Proceedings of IEEE International Conference on Evolutionary Computation, pp. 69–73. IEEE Computer Society, Washington, DC, May 1998
Zhen, L., Wang, L., Wang, X., Huang, Z.: A novel PSO-inspired probability-based binary optimization algorithm. In: 2008 International Symposium on Information Science and Engineering, vol. 2, pp. 248–251, December 2008
Bonyadi, M.R., Michalewicz, Z.: Stability analysis of the particle swarm optimization without stagnation assumption. IEEE Trans. Evol. Comput. 20(5), 814–819 (2016)
Yamada, S., Neshatian, K.: Multiple kernel learning with one-level optimization of radius and margin. In: Advances in Artificial Intelligence - 30th Australasian Joint Conference, AI 2017, Melbourne, VIC, Australia, 19–20 August 2017, Proceedings, pp. 52–63 (2017)
Dua, D., Graff, C.: UCI machine learning repository (2017)
Mitchell, M.: An Introduction to Genetic Algorithms. MIT Press, Cambridge (1998)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Yamada, S., Neshatian, K. (2019). Comparison of Embedded and Wrapper Approaches for Feature Selection in Support Vector Machines. In: Nayak, A., Sharma, A. (eds) PRICAI 2019: Trends in Artificial Intelligence. PRICAI 2019. Lecture Notes in Computer Science(), vol 11671. Springer, Cham. https://doi.org/10.1007/978-3-030-29911-8_12
Download citation
DOI: https://doi.org/10.1007/978-3-030-29911-8_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-29910-1
Online ISBN: 978-3-030-29911-8
eBook Packages: Computer ScienceComputer Science (R0)