Abstract
Selecting highly discriminative features from a whole feature set has become an important research area. Not only can this improve the performance of classification, but it can also decrease the cost of system diagnoses when a large number of noisy, redundant features are excluded. Binary nature-inspired algorithms have been used as a feature selection procedure. Each of these algorithms requires an initial population to be set, and the appropriateness of the initialization plays a key role in the final result. At the stage of population initialization, the positions are initialized randomly by uniform distribution which leads to a high variability of the classification results. To avoid the randomness of the population generated and to take into account the relation between each feature and the class variable, parametric and non-parametric methods, such as the t-test and Wilcoxon rank sum test are proposed as an initial population in the binary nature-inspired algorithms. This modification can help these binary algorithms to enhance global exploration and local exploitation or exhibit a slow convergence speed compared with the standard procedure. The binary bat, gray wolf, and whale algorithms are considered. The performance of our proposed methods is evaluated on ten publicly available datasets with high-dimensional and low-dimensional data. The experimental results and statistical analysis confirm that the performance of our proposed methods compared with the standard algorithms is better in terms of classification accuracy, the number of selected features, running time, and feature selection stability.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abdel-Basset M, El-Shahat D, Sangaiah AK (2017) A modified nature inspired meta-heuristic whale optimization algorithm for solving 0–1 knapsack problem. Int J Mach Learn Cybern 10:495–514. https://doi.org/10.1007/s13042-017-0731-3
Abdulrazaq Kahya M, Abduljabbar Altamir S, Yahya Algamal Z (2019) Improving whale optimization algorithm for feature selection with a time-varying transfer function. Numer Algebra Control Optim. https://doi.org/10.3934/naco.2020017
Al-Fakih AM, Algamal ZY, Lee MH, Aziz M, Ali HTM (2019) QSAR classification model for diverse series of antifungal agents based on improved binary differential search algorithm. SAR QSAR Environ Res 30:131–143. https://doi.org/10.1080/1062936X.2019.1568298
Algamal ZY, Lee MH (2018) A two-stage sparse logistic regression for optimal gene selection in high-dimensional microarray data classification. Adv Data Anal Classif 13:753–771. https://doi.org/10.1007/s11634-018-0334-1
Algamal ZY, Qasim MK, Lee MH, Ali HTM (2020a) QSAR model for predicting neuraminidase inhibitors of influenza A viruses (H1N1) based on adaptive grasshopper optimization algorithm. SAR QSAR Environ Res 31:803–814. https://doi.org/10.1080/1062936X.2020.1818616
Algamal ZY, Qasim MK, Lee MH, Mohammad Ali HT (2020b) High-dimensional QSAR/QSPR classification modeling based on improving pigeon optimization algorithm. Chemom Intell Lab Syst. https://doi.org/10.1016/j.chemolab.2020.104170
Alomari OA, Khader AT, Al-Betar MA, Awadallah MA (2018) A novel gene selection method using modified MRMR and hybrid bat-inspired algorithm with β-hill climbing. Appl Intell 48:4429–4447. https://doi.org/10.1007/s10489-018-1207-1
Al-Tashi Q, Abdul Kadir SJ, Rais HM, Mirjalili S, Alhussian H (2019a) Binary optimization using hybrid grey wolf optimization for feature selection. IEEE Access 7:39496–39508. https://doi.org/10.1109/access.2019.2906757
Al-Tashi Q, Kadir SJA, Rais HM, Mirjalili S, Alhussian H (2019b) Binary optimization using hybrid grey wolf optimization for feature selection. IEEE Access 7:39496–39508
Arora S, Anand P (2019) Binary butterfly optimization approaches for feature selection. Expert Syst Appl 116:147–160. https://doi.org/10.1016/j.eswa.2018.08.051
Arora S, Kaur G (2018) Chaotic whale optimization algorithm. J Comput Des Eng 5:275–284. https://doi.org/10.1016/j.jcde.2017.12.006
Arora S, Kohli M (2018) Chaotic grey wolf optimization algorithm for constrained optimization problems. J Comput Des Eng 5:458–472. https://doi.org/10.1016/j.jcde.2017.02.005
Barshandeh S, Haghzadeh M (2020) A new hybrid chaotic atom search optimization based on tree-seed algorithm and Levy flight for solving optimization problems. Eng Comput. https://doi.org/10.1007/s00366-020-00994-0
Dhal KG, Ray S, Das A, Das S (2018) A survey on nature-inspired optimization algorithms and their application in image enhancement domain. Arch Comput Methods Eng 26:1607–1638. https://doi.org/10.1007/s11831-018-9289-9
Emary E, Zawbaa HM, Ghany KKA, Hassanien AE, Parv B (2015) Firefly optimization algorithm for feature selection. In: Proceedings of the 7th Balkan conference on informatics conference. ACM, pp 26
Emary E, Zawbaa HM, Hassanien AE (2016) Binary grey wolf optimization approaches for feature selection. Neurocomputing 172:371–381. https://doi.org/10.1016/j.neucom.2015.06.083
Feng Y-H, Wang G-G (2018) Binary moth search algorithm for discounted 0–1 knapsack problem. IEEE Access 6:10708–10719. https://doi.org/10.1109/access.2018.2809445
Guyon I, Elisseeff A (2003) An introduction to variable and feature selection. J Mach Learn Res 3:1157–1182
Hameed SS, Hassan R, Muhammad FF (2017) Selection and classification of gene expression in autism disorder: use of a combination of statistical filters and a GBPSO-SVM algorithm. PLoS ONE 12:e0187371. https://doi.org/10.1371/journal.pone.0187371
He Z, Yu W (2010) Stable feature selection for biomarker discovery. Comput Biol Chem 34:215–225. https://doi.org/10.1016/j.compbiolchem.2010.07.002
Hichem H, Elkamel M, Rafik M, Mesaaoud MT, Ouahiba C (2019) A new binary grasshopper optimization algorithm for feature selection problem. J King Saud Univ Comput Inf Sci. https://doi.org/10.1016/j.jksuci.2019.11.007
Hu P, Pan J-S, Chu S-C (2020) Improved binary grey wolf optimizer and its application for feature selection. Knowl Based Syst 195:105746
Kalina J (2014) Classification methods for high-dimensional genetic data. Biocybern Biomed Eng 34:10–18. https://doi.org/10.1016/j.bbe.2013.09.007
Karthikeyan S, Asokan P, Nickolas S (2014) A hybrid discrete firefly algorithm for multi-objective flexible job shop scheduling problem with limited resource constraints. Int J Adv Manuf Technol 72:1567–1579
Kathiravan K, Rathina Prabha N (2018) Assessment of ramping cost for independent power producers using firefly algorithm and gray wolf optimization. Clust Comput 22:4479–4490. https://doi.org/10.1007/s10586-018-2045-y
Kumar V, Kumar D (2018) Binary whale optimization algorithm and its application to unit commitment problem. Neural Comput Appl 32:2095–2123. https://doi.org/10.1007/s00521-018-3796-3
Li Q et al (2017) An enhanced grey wolf optimization based feature selection wrapped kernel extreme learning machine for medical diagnosis. Comput Math Methods Med 2017:9512741. https://doi.org/10.1155/2017/9512741
Luo K, Zhao Q (2019) A binary grey wolf optimizer for the multidimensional knapsack problem. Appl Soft Comput 83:105645. https://doi.org/10.1016/j.asoc.2019.105645
Mafarja M, Aljarah I, Faris H, Hammouri AI, Al-Zoubi AM, Mirjalili S (2019) Binary grasshopper optimisation algorithm approaches for feature selection problems. Expert Syst Appl 117:267–286. https://doi.org/10.1016/j.eswa.2018.09.015
Mafarja M, Qasem A, Heidari AA, Aljarah I, Faris H, Mirjalili S (2020) Efficient hybrid nature-inspired binary optimizers for feature selection. Cogn Comput 12:150–175
Mirjalili S, Lewis A (2013) S-shaped versus V-shaped transfer functions for binary particle swarm optimization. Swarm Evol Comput 9:1–14. https://doi.org/10.1016/j.swevo.2012.09.002
Mirjalili S, Lewis A (2016) The whale optimization algorithm. Adv Eng Softw 95:51–67
Mirjalili S, Mirjalili SM, Lewis A (2014a) Grey wolf optimizer. Adv Eng Softw 69:46–61. https://doi.org/10.1016/j.advengsoft.2013.12.007
Mirjalili S, Mirjalili SM, Yang X-S (2014b) Binary bat algorithm. Neural Comput Appl 25:663–681. https://doi.org/10.1007/s00521-013-1525-5
Oreski S, Oreski G (2014) Genetic algorithm-based heuristic for feature selection in credit risk assessment. Expert Syst Appl 41:2052–2064
Qasim OS, Algamal ZY (2018) Feature selection using particle swarm optimization-based logistic regression model. Chemom Intell Lab Syst 182:41–46. https://doi.org/10.1016/j.chemolab.2018.08.016
Qasim OS, Algamal ZY (2020) Feature selection using different transfer functions for binary bat algorithm. Int J Math Eng Manag Sci 5:697–706. https://doi.org/10.33889/ijmems.2020.5.4.056
Qasim OS, Al-Thanoon NA, Algamal ZY (2020) Feature selection based on chaotic binary black hole algorithm for data classification. Chemom Intell Lab Syst 204:104104. https://doi.org/10.1016/j.chemolab.2020.104104
Qiu C (2019) A novel multi-swarm particle swarm optimization for feature selection. Genet Program Evolvable Mach 20:503–529. https://doi.org/10.1007/s10710-019-09358-0
Rizk-Allah RM, Hassanien AE (2017) New binary bat algorithm for solving 0–1 knapsack problem. Complex Intell Syst 4:31–53. https://doi.org/10.1007/s40747-017-0050-z
Sayed GI, Darwish A, Hassanien AE (2018) A new chaotic whale optimization algorithm for features selection. J Classif 35:300–344. https://doi.org/10.1007/s00357-018-9261-2
Shang R, Wang W, Stolkin R, Jiao L (2018) Non-negative spectral learning and sparse regression-based dual-graph regularized feature selection. IEEE Trans Cybern 48:793–806. https://doi.org/10.1109/TCYB.2017.2657007
Shang R, Meng Y, Wang W, Shang F, Jiao L (2019) Local discriminative based sparse subspace learning for feature selection. Pattern Recogn 92:219–230. https://doi.org/10.1016/j.patcog.2019.03.026
Shang R, Xu K, Shang F, Jiao L (2020) Sparse and low-redundant subspace learning-based dual-graph regularized robust feature selection. Knowl Based Syst 187:104830. https://doi.org/10.1016/j.knosys.2019.07.001
Sharma M, Kaur P (2020) A comprehensive analysis of nature-inspired meta-heuristic techniques for feature selection problem. Arch Comput Methods Eng. https://doi.org/10.1007/s11831-020-09412-6
Shrivastava P, Shukla A, Vepakomma P, Bhansali N, Verma K (2017) A survey of nature-inspired algorithms for feature selection to identify Parkinson’s disease. Comput Methods Programs Biomed 139:171–179. https://doi.org/10.1016/j.cmpb.2016.07.029
Tawhid MA, Ibrahim AM (2019) Feature selection based on rough set approach, wrapper approach, and binary whale optimization algorithm. Int J Mach Learn Cybern 11:573–602. https://doi.org/10.1007/s13042-019-00996-5
UCI machine learning repository (2013) Irvine, CA. http://archive.ics.uci.edu/ml. Accessed 8 Jan 2020
Yan C, Ma J, Luo H, Zhang G, Luo J (2019) A Novel Feature Selection Method for High-Dimensional Biomedical Data Based on an Improved Binary Clonal Flower Pollination Algorithm Hum Hered 84:34–46. https://doi.org/10.1159/000501652
Yang X-S (2010) A new metaheuristic bat-inspired algorithm. In: González JR, Pelta DA, Cruz C, Terrazas G, Krasnogor N (eds) Nature inspired cooperative strategies for optimization (NICSO 2010). Springer, pp 65–74
Zhao D, He Y (2015) Chaotic binary bat algorithm for analog test point selection. Analog Integr Circ Sig Process 84:201–214. https://doi.org/10.1007/s10470-015-0548-5
Zhao X et al (2019) Chaos enhanced grey wolf optimization wrapped ELM for diagnosis of paraquat-poisoned patients. Comput Biol Chem 78:481–490. https://doi.org/10.1016/j.compbiolchem.2018.11.017
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Al-Thanoon, N.A., Qasim, O.S. & Algamal, Z.Y. Improving nature-inspired algorithms for feature selection. J Ambient Intell Human Comput 13, 3025–3035 (2022). https://doi.org/10.1007/s12652-021-03136-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12652-021-03136-6