Abstract
We explore in this paper the use of metaheuristics to select features from a dataset in order to improve the prediction performance of models build with different machine learning methods. To this end, we compare the performances of 5 learning methods: Logistic Regression (LR), K-Nearest Neighbors (KNN), Gaussian Naive Bayes (GNB), Support Vector Machine (SVM) and Random Forest (RF) on 4 heterogeneous datasets in the number of data and features, for different feature selection methods (metaheuristics or statistical filters).
The results obtained show that feature selection by improving a metaheuristic derived from the genetic algorithm leads to much better performances no matter the learning method used compared to without feature selection on the same dataset.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Motoda, H., Liu, H.: Feature Selection Extraction and Construction (2002)
Chipperfield, A.J., et al.: A. Carreira-perpiñán. A Review of Dimension Reduction Techniques. Technical
Grollemund, V., et al.: Development and validation of a 1-year survival prognosis estimation model for amyotrophic lateral sclerosis using manifold learning algorithm UMAP. Sci. Rep. 10(1), 13378 (2020). https://doi.org/10.1038/s41598-020-70125-8
Grollemund, V., Chat, G.L., Pradat-Peyre, J.-F., Delbot, F.: Manifold learning for innovation funding: identification of potential funding recipients. In: Maglogiannis, I., Iliadis, L., Pimenidis, E. (eds.) AIAI 2020. IAICT, vol. 583, pp. 119–127. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-49161-1_11
Kohavi, R., John, G.H.: Wrappers for feature subset selection. Artif. Intell. Relev. 97(1), 273–324 (1997)
Grollemund, V., et al.: Machine learning in amyotrophic lateral sclerosis: achievements, pitfalls, and future directions. Front. Neurosci. 13, 135 (2019). https://doi.org/10.3389/fnins.2019.00135
Zhang, H., Sun, G.: Feature selection using tabu search method. Pattern Recognit. 35, 701–711 (2002)
Mafarja, M.M., Mirjalili, S.M.: Hybrid whale optimization algorithm with simulated annealing for feature selection. Neurocomputing 260, 302–312 (2017)
Baluja, S.: Population-Based Incremental Learning: A Method for Integrating Genetic Search Based Function Optimization and Competitive Learning. Tech. rep. CMU-CS-94-163. Carnegie Mellon University, Pittsburgh (1994)
Marandi, A., et al.: Boolean Particle Swarm Optimization and Its Application to the Design of a Dual-Band Dual-Polarized Planar Antenna, Jan 2006, pp. 3212–3218 (2006). https://doi.org/10.1109/CEC.2006.1688716
Chakravarty, K., et al.: Feature selection by differential evolution algorithm - a case study in personnel identification. In: 2013 IEEE Congress on Evolutionary Computation, pp. 892–899 (2013)
Zhang, J., Sanderson, A.C.: JADE: adaptive differential evolution with optional external archive. IEEE Trans. Evolution. Comput. 13(5), 945–958 (2009)
Rowland, L.P.: How amyotrophic lateral sclerosis got its name: the clinical-pathologic genius of Jean-Martin Charcot. Archiv. Neurol. 58(3), 512–515 (2001)
Atassi, N., et al.: The PRO-ACT database. Neurology 83, 1719–1725 (2014)
Boutell, M.R., et al.: Learning Multi-label Scene Classification (2004)
Gravier, E., et al.: A prognostic DNA signature for T1T2 nodenegative breast cancer patients. Genes Chromos. Cancer 49(12), 1125–1125 (2010)
Tian, E., et al.: The role of the wnt-signaling antagonist DKK1 in the development of osteolytic lesions in multiple myeloma. New Engl. J. Med. 349(26), 2483–2494 (2003)
Bommert, A., et al.: Benchmark for filter methods for feature selection in high-dimensional classification data. Comput. Statist. Data Anal. 143, 106839 (2020). https://doi.org/10.1016/j.csda.2019.106839
Robnik-Sikonja, M., Kononenko, I.: Theoretical and empirical analysis of ReliefF and RReliefF. Mach. Learn. 53(1), 23–69 (2003). https://doi.org/10.1023/A:1025667309714
Peng, H., Long, F., Ding, C.: Feature selection based on mutual information criteria of max-dependency, max-relevance, and minredundancy. IEEE Trans. Pattern Anal. Mach. Intell. 27(8), 1226–1238 (2005). https://doi.org/10.1109/TPAMI.2005.159
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 IFIP International Federation for Information Processing
About this paper
Cite this paper
Anani, T., Delbot, F., Pradat-Peyre, JF. (2022). Experimental Comparison of Metaheuristics for Feature Selection in Machine Learning in the Medical Context. In: Maglogiannis, I., Iliadis, L., Macintyre, J., Cortez, P. (eds) Artificial Intelligence Applications and Innovations. AIAI 2022. IFIP Advances in Information and Communication Technology, vol 647. Springer, Cham. https://doi.org/10.1007/978-3-031-08337-2_17
Download citation
DOI: https://doi.org/10.1007/978-3-031-08337-2_17
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-08336-5
Online ISBN: 978-3-031-08337-2
eBook Packages: Computer ScienceComputer Science (R0)
