Skip to main content
SpringerLink
Log in

Spatial bound whale optimization algorithm: an efficient high-dimensional feature selection approach

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Selecting a subset of candidate features is one of the important steps in the data mining process. The ultimate goal of feature selection is to select an optimal number of high-quality features that can maximize the performance of the learning algorithm. However, this problem becomes challenging when the number of features increases in a dataset. Hence, advanced optimization techniques are used these days to search for the optimal feature combinations. Whale Optimization Algorithm (WOA) is a recent metaheuristic that has successfully applied to different optimization problems. In this work, we propose a new variant of WOA (SBWOA) based on spatial bounding strategy to play the role of finding the potential features from the high-dimensional feature space. Also, a simplified version of SBWOA is introduced in an attempt to maintain a low computational complexity. The effectiveness of the proposed approach was validated on 16 high-dimensional datasets gathered from Arizona State University, and the results are compared with the other eight state-of-the-art feature selection methods. Among the competitors, SBWOA has achieved the highest accuracy for most datasets such as TOX_171, Colon, and Prostate_GE. The results obtained demonstrate the supremacy of the proposed approaches over the comparison methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data availability

Available at http://featureselection.asu.edu/datasets.php.

Notes

  1. https://seyedalimirjalili.com/woa

References

  1. Nguyen BH, Xue B, Zhang M (2020) A survey on swarm intelligence approaches to feature selection in data mining. Swarm Evol Comput 54:100663. https://doi.org/10.1016/j.swevo.2020.100663

    Article  Google Scholar 

  2. Zhang L, Shan L, Wang J (2017) Optimal feature selection using distance-based discrete firefly algorithm with mutual information criterion. Neural Comput Appl 28:2795–2808. https://doi.org/10.1007/s00521-016-2204-0

    Article  Google Scholar 

  3. Ma B, Xia Y (2017) A tribe competition-based genetic algorithm for feature selection in pattern classification. Appl Soft Comput 58:328–338. https://doi.org/10.1016/j.asoc.2017.04.042

    Article  Google Scholar 

  4. Jiang S, Chin K-S, Wang L et al (2017) Modified genetic algorithm-based feature selection combined with pre-trained deep neural network for demand forecasting in outpatient department. Expert Syst Appl 82:216–230. https://doi.org/10.1016/j.eswa.2017.04.017

    Article  Google Scholar 

  5. Labani M, Moradi P, Ahmadizar F, Jalili M (2018) A novel multivariate filter method for feature selection in text classification problems. Eng Appl Artif Intell 70:25–37. https://doi.org/10.1016/j.engappai.2017.12.014

    Article  Google Scholar 

  6. Wang D, Zhang H, Liu R et al (2014) t-Test feature selection approach based on term frequency for text categorization. Pattern Recognit Lett 45:1–10. https://doi.org/10.1016/j.patrec.2014.02.013

    Article  Google Scholar 

  7. Gao W, Hu L, Zhang P, He J (2018) Feature selection considering the composition of feature relevancy. Pattern Recognit Lett 112:70–74. https://doi.org/10.1016/j.patrec.2018.06.005

    Article  Google Scholar 

  8. Peng H, Long F, Ding C (2005) Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans Pattern Anal Mach Intell 27:1226–1238. https://doi.org/10.1109/TPAMI.2005.159

    Article  Google Scholar 

  9. Hammouri AI, Mafarja M, Al-Betar MA et al (2020) An improved dragonfly algorithm for feature selection. Knowl-Based Syst 203:106131. https://doi.org/10.1016/j.knosys.2020.106131

    Article  Google Scholar 

  10. Faris H, Hassonah MA, Al-Zoubi AM et al (2017) A multi-verse optimizer approach for feature selection and optimizing SVM parameters based on a robust system architecture. Neural Comput Appl. https://doi.org/10.1007/s00521-016-2818-2

    Article  Google Scholar 

  11. Kennedy J (2011) Particle swarm optimization. Encyclopedia of machine learning. Springer, Boston, MA, pp 760–766

    Google Scholar 

  12. Storn R, Price K (1997) Differential evolution – a simple and efficient heuristic for global optimization over continuous spaces. J Glob Optim 11:341–359. https://doi.org/10.1023/A:1008202821328

    Article  MathSciNet  MATH  Google Scholar 

  13. Holland JH (1992) Genetic algorithms. Sci Am 267:66–73

    Article  Google Scholar 

  14. Yang X-S (2010) A new metaheuristic bat-inspired algorithm. Nature inspired cooperative strategies for optimization (NICSO 2010). Springer, Berlin, Heidelberg, pp 65–74

    Chapter  Google Scholar 

  15. Dorigo M, Birattari M (2011) Ant colony optimization. Encyclopedia of machine learning. Springer, Boston, MA, pp 36–39

    Google Scholar 

  16. Karaboga D, Basturk B (2007) A powerful and efficient algorithm for numerical function optimization: artificial bee colony (ABC) algorithm. J Glob Optim 39:459–471. https://doi.org/10.1007/s10898-007-9149-x

    Article  MathSciNet  MATH  Google Scholar 

  17. Rashedi E, Nezamabadi-pour H, Saryazdi S (2009) GSA: a gravitational search algorithm. Inf Sci 179:2232–2248. https://doi.org/10.1016/j.ins.2009.03.004

    Article  MATH  Google Scholar 

  18. Mirjalili S, Lewis A (2016) The whale optimization algorithm. Adv Eng Softw 95:51–67. https://doi.org/10.1016/j.advengsoft.2016.01.008

    Article  Google Scholar 

  19. Mafarja MM, Mirjalili S (2017) Hybrid whale optimization algorithm with simulated annealing for feature selection. Neurocomputing 260:302–312. https://doi.org/10.1016/j.neucom.2017.04.053

    Article  Google Scholar 

  20. Abd Elaziz M, Oliva D (2018) Parameter estimation of solar cells diode models by an improved opposition-based whale optimization algorithm. Energy Convers Manag 171:1843–1859. https://doi.org/10.1016/j.enconman.2018.05.062

    Article  Google Scholar 

  21. Aljarah I, Faris H, Mirjalili S (2018) Optimizing connection weights in neural networks using the whale optimization algorithm. Soft Comput 22:1–15. https://doi.org/10.1007/s00500-016-2442-1

    Article  Google Scholar 

  22. Sun W, Zhang C (2018) Analysis and forecasting of the carbon price using multi—resolution singular value decomposition and extreme learning machine optimized by adaptive whale optimization algorithm. Appl Energy 231:1354–1371. https://doi.org/10.1016/j.apenergy.2018.09.118

    Article  Google Scholar 

  23. Abdel-Basset M, Abdle-Fatah L, Sangaiah AK (2018) An improved Lévy based whale optimization algorithm for bandwidth-efficient virtual machine placement in cloud computing environment. Clust Comput. https://doi.org/10.1007/s10586-018-1769-z

    Article  Google Scholar 

  24. Sharawi M, Zawbaa HM, Emary E, et al (2017) Feature selection approach based on whale optimization algorithm. In: 2017 Ninth International Conference on Advanced Computational Intelligence (ICACI). pp 163–168

  25. Faramarzi A, Heidarinejad M, Mirjalili S, Gandomi AH (2020) Marine predators algorithm: a nature-inspired metaheuristic. Expert Syst Appl 152:113377. https://doi.org/10.1016/j.eswa.2020.113377

    Article  Google Scholar 

  26. Barani F, Mirhosseini M, Nezamabadi-pour H (2017) Application of binary quantum-inspired gravitational search algorithm in feature subset selection. Appl Intell 47:304–318. https://doi.org/10.1007/s10489-017-0894-3

    Article  Google Scholar 

  27. Neggaz N, Houssein EH, Hussain K (2020) An efficient henry gas solubility optimization for feature selection. Expert Syst Appl 152:113364. https://doi.org/10.1016/j.eswa.2020.113364

    Article  Google Scholar 

  28. Siedlecki W, Sklansky J (1989) A note on genetic algorithms for large-scale feature selection. Pattern Recognit Lett 10:335–347. https://doi.org/10.1016/0167-8655(89)90037-8

    Article  MATH  Google Scholar 

  29. Huang C-L, Wang C-J (2006) A GA-based feature selection and parameters optimizationfor support vector machines. Expert Syst Appl 31:231–240. https://doi.org/10.1016/j.eswa.2005.09.024

    Article  Google Scholar 

  30. Nemati S, Basiri ME, Ghasem-Aghaee N, Aghdam MH (2009) A novel ACO–GA hybrid algorithm for feature selection in protein function prediction. Expert Syst Appl 36:12086–12094. https://doi.org/10.1016/j.eswa.2009.04.023

    Article  Google Scholar 

  31. De Stefano C, Fontanella F, Marrocco C, Scotto di Freca A (2014) A GA-based feature selection approach with an application to handwritten character recognition. Pattern Recognit Lett 35:130–141. https://doi.org/10.1016/j.patrec.2013.01.026

    Article  Google Scholar 

  32. Rejer I (2015) Genetic algorithm with aggressive mutation for feature selection in BCI feature space. Pattern Anal Appl 18:485–492. https://doi.org/10.1007/s10044-014-0425-3

    Article  MathSciNet  Google Scholar 

  33. Chuang L-Y, Chang H-W, Tu C-J, Yang C-H (2008) Improved binary PSO for feature selection using gene expression data. Comput Biol Chem 32:29–38. https://doi.org/10.1016/j.compbiolchem.2007.09.005

    Article  MATH  Google Scholar 

  34. Unler A, Murat A (2010) A discrete particle swarm optimization method for feature selection in binary classification problems. Eur J Oper Res 206:528–539. https://doi.org/10.1016/j.ejor.2010.02.032

    Article  MATH  Google Scholar 

  35. Chuang L-Y, Tsai S-W, Yang C-H (2011) Improved binary particle swarm optimization using catfish effect for feature selection. Expert Syst Appl 38:12699–12707. https://doi.org/10.1016/j.eswa.2011.04.057

    Article  Google Scholar 

  36. Tan TY, Zhang L, Neoh SC, Lim CP (2018) Intelligent skin cancer detection using enhanced particle swarm optimization. Knowl-Based Syst 158:118–135. https://doi.org/10.1016/j.knosys.2018.05.042

    Article  Google Scholar 

  37. Too J, Abdullah AR, Mohd Saad N (2019) A new co-evolution binary particle swarm optimization with multiple inertia weight strategy for feature selection. Informatics 6:21. https://doi.org/10.3390/informatics6020021

    Article  Google Scholar 

  38. Banka H, Dara S (2015) A Hamming distance based binary particle swarm optimization (HDBPSO) algorithm for high dimensional feature selection, classification and validation. Pattern Recognit Lett 52:94–100. https://doi.org/10.1016/j.patrec.2014.10.007

    Article  Google Scholar 

  39. Ji B, Lu X, Sun G et al (2020) Bio-inspired feature selection: an improved binary particle swarm optimization approach. IEEE Access 8:85989–86002. https://doi.org/10.1109/ACCESS.2020.2992752

    Article  Google Scholar 

  40. Xue Y, Tang T, Pang W, Liu AX (2020) Self-adaptive parameter and strategy based particle swarm optimization for large-scale feature selection problems with multiple classifiers. Appl Soft Comput 88:106031. https://doi.org/10.1016/j.asoc.2019.106031

    Article  Google Scholar 

  41. Mirjalili S, Gandomi AH, Mirjalili SZ et al (2017) Salp swarm algorithm: a bio-inspired optimizer for engineering design problems. Adv Eng Softw 114:163–191. https://doi.org/10.1016/j.advengsoft.2017.07.002

    Article  Google Scholar 

  42. Sayed GI, Khoriba G, Haggag MH (2018) A novel chaotic salp swarm algorithm for global optimization and feature selection. Appl Intell. https://doi.org/10.1007/s10489-018-1158-6

    Article  Google Scholar 

  43. Kaur T, Saini BS, Gupta S (2018) A novel feature selection method for brain tumor MR image classification based on the Fisher criterion and parameter-free Bat optimization. Neural Comput Appl 29:193–206. https://doi.org/10.1007/s00521-017-2869-z

    Article  Google Scholar 

  44. Atashpaz-Gargari E, Reis MS, Braga-Neto UM et al (2018) A fast branch-and-bound algorithm for u-curve feature selection. Pattern Recognit 73:172–188. https://doi.org/10.1016/j.patcog.2017.08.013

    Article  Google Scholar 

  45. Sindhu R, Ngadiran R, Yacob YM et al (2017) Sine–cosine algorithm for feature selection with elitism strategy and new updating mechanism. Neural Comput Appl 28:2947–2958. https://doi.org/10.1007/s00521-017-2837-7

    Article  Google Scholar 

  46. Gu S, Cheng R, Jin Y (2018) Feature selection for high-dimensional classification using a competitive swarm optimizer. Soft Comput 22:811–822. https://doi.org/10.1007/s00500-016-2385-6

    Article  Google Scholar 

  47. Zawbaa HM, Emary E, Grosan C (2016) feature selection via chaotic antlion optimization. PLoS ONE 11:e0150652. https://doi.org/10.1371/journal.pone.0150652

    Article  Google Scholar 

  48. Mirjalili S (2015) The ant lion optimizer. Adv Eng Softw 83:80–98. https://doi.org/10.1016/j.advengsoft.2015.01.010

    Article  Google Scholar 

  49. Mafarja M, Aljarah I, Heidari AA et al (2018) Binary dragonfly optimization for feature selection using time-varying transfer functions. Knowl-Based Syst 161:185–204. https://doi.org/10.1016/j.knosys.2018.08.003

    Article  Google Scholar 

  50. Tu Q, Chen X, Liu X (2019) Multi-strategy ensemble grey wolf optimizer and its application to feature selection. Appl Soft Comput 76:16–30. https://doi.org/10.1016/j.asoc.2018.11.047

    Article  Google Scholar 

  51. Too J, Abdullah AR (2020) Opposition based competitive grey wolf optimizer for EMG feature selection. Evol Intell. https://doi.org/10.1007/s12065-020-00441-5

    Article  Google Scholar 

  52. Gholami J, Pourpanah F, Wang X (2020) Feature selection based on improved binary global harmony search for data classification. Appl Soft Comput 93:106402. https://doi.org/10.1016/j.asoc.2020.106402

    Article  Google Scholar 

  53. Saremi S, Mirjalili S, Lewis A (2017) Grasshopper optimisation algorithm: theory and application. Adv Eng Softw 105:30–47. https://doi.org/10.1016/j.advengsoft.2017.01.004

    Article  Google Scholar 

  54. Zakeri A, Hokmabadi A (2019) Efficient feature selection method using real-valued grasshopper optimization algorithm. Expert Syst Appl 119:61–72. https://doi.org/10.1016/j.eswa.2018.10.021

    Article  Google Scholar 

  55. Sayed GI, Hassanien AE (2017) Moth-flame swarm optimization with neutrosophic sets for automatic mitosis detection in breast cancer histology images. Appl Intell 47:397–408. https://doi.org/10.1007/s10489-017-0897-0

    Article  Google Scholar 

  56. AbdEl-Fattah Sayed S, Nabil E, Badr A (2016) A binary clonal flower pollination algorithm for feature selection. Pattern Recognit Lett 77:21–27. https://doi.org/10.1016/j.patrec.2016.03.014

    Article  Google Scholar 

  57. Chen Y-P, Li Y, Wang G et al (2017) A novel bacterial foraging optimization algorithm for feature selection. Expert Syst Appl 83:1–17. https://doi.org/10.1016/j.eswa.2017.04.019

    Article  Google Scholar 

  58. Tubishat M, Abushariah MAM, Idris N, Aljarah I (2019) Improved whale optimization algorithm for feature selection in Arabic sentiment analysis. Appl Intell 49:1688–1707. https://doi.org/10.1007/s10489-018-1334-8

    Article  Google Scholar 

  59. Sun Y, Wang X, Chen Y, Liu Z (2018) A modified whale optimization algorithm for large-scale global optimization problems. Expert Syst Appl 114:563–577. https://doi.org/10.1016/j.eswa.2018.08.027

    Article  Google Scholar 

  60. Ling Y, Zhou Y, Luo Q (2017) Levy flight trajectory-based whale optimization algorithm for global optimization. IEEE Access 5:6168–6186. https://doi.org/10.1109/ACCESS.2017.2695498

    Article  Google Scholar 

  61. Kılıç H, Yüzgeç U (2019) Tournament selection based antlion optimization algorithm for solving quadratic assignment problem. Eng Sci Technol Int J 22:673–691. https://doi.org/10.1016/j.jestch.2018.11.013

    Article  Google Scholar 

  62. Gunasundari S, Janakiraman S, Meenambal S (2016) Velocity bounded boolean particle swarm optimization for improved feature selection in liver and kidney disease diagnosis. Expert Syst Appl 56:28–47. https://doi.org/10.1016/j.eswa.2016.02.042

    Article  Google Scholar 

  63. Zheng Y, Zhang B (2015) A simplified water wave optimization algorithm. In: 2015 IEEE Congress on Evolutionary Computation (CEC). pp 807–813

  64. Xue B, Zhang M, Browne WN (2013) Particle swarm optimization for feature selection in classification: a multi-objective approach. IEEE Trans Cybern 43:1656–1671. https://doi.org/10.1109/TSMCB.2012.2227469

    Article  Google Scholar 

  65. Emary E, Zawbaa HM (2018) Feature selection via Lèvy Antlion optimization. Pattern Anal Appl. https://doi.org/10.1007/s10044-018-0695-2

    Article  Google Scholar 

  66. Xue B, Zhang M, Browne WN (2014) Particle swarm optimisation for feature selection in classification: novel initialisation and updating mechanisms. Appl Soft Comput 18:261–276. https://doi.org/10.1016/j.asoc.2013.09.018

    Article  Google Scholar 

  67. Alweshah M, Khalaileh SA, Gupta BB et al (2020) The monarch butterfly optimization algorithm for solving feature selection problems. Neural Comput Appl. https://doi.org/10.1007/s00521-020-05210-0

    Article  Google Scholar 

  68. Hegazy AhE, Makhlouf MA, El-Tawel GhS (2020) Improved salp swarm algorithm for feature selection. J King Saud Univ Comput Inf Sci 32:335–344. https://doi.org/10.1016/j.jksuci.2018.06.003

    Article  Google Scholar 

  69. El-Hasnony IM, Barakat SI, Elhoseny M, Mostafa RR (2020) Improved feature selection model for big data analytics. IEEE Access 8:66989–67004. https://doi.org/10.1109/ACCESS.2020.2986232

    Article  Google Scholar 

  70. Datasets | Feature Selection @ ASU. http://featureselection.asu.edu/datasets.php. Accessed from 3 Oct 2019

  71. Rao R (2016) Jaya: A simple and new optimization algorithm for solving constrained and unconstrained optimization problems. Int J Ind Eng Comput 7:19–34

    Google Scholar 

  72. Mirjalili S (2015) Moth-flame optimization algorithm: a novel nature-inspired heuristic paradigm. Knowl-Based Syst 89:228–249. https://doi.org/10.1016/j.knosys.2015.07.006

    Article  Google Scholar 

  73. Askarzadeh A (2016) A novel metaheuristic method for solving constrained engineering optimization problems: crow search algorithm. Comput Struct 169:1–12. https://doi.org/10.1016/j.compstruc.2016.03.001

    Article  Google Scholar 

  74. Zhang Y, Jin Z, Mirjalili S (2020) Generalized normal distribution optimization and its applications in parameter extraction of photovoltaic models. Energy Convers Manag 224:113301. https://doi.org/10.1016/j.enconman.2020.113301

    Article  Google Scholar 

  75. Cheng R, Jin Y (2015) A competitive swarm optimizer for large scale optimization. IEEE Trans Cybern 45:191–204. https://doi.org/10.1109/TCYB.2014.2322602

    Article  Google Scholar 

Download references

Funding

No funding was received for conducting this study.

Author information

Authors and Affiliations

  1. Faculty of Electrical Engineering, Universiti Teknikal Malaysia Melaka, Hang Tuah Jaya, Durian Tunggal, 76100, Melaka, Malaysia

    Jingwei Too

  2. Department of Computer Science, Faculty of Engineering and Technology, Birzeit University, Birzeit, Palestine

    Majdi Mafarja

  3. Center for Artificial Intelligence Research and Optimization, Torrens University Australia, Fortitude Valley, Brisbane, QLD, 4006, Australia

    Seyedali Mirjalili

  4. Yonsei Frontier Lab, Yonsei University, Seoul, Korea

    Seyedali Mirjalili

  5. King Abdulaziz University, Jeddah, Saudi Arabia

    Seyedali Mirjalili

Authors
  1. Jingwei Too
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Majdi Mafarja
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Seyedali Mirjalili
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jingwei Too.

Ethics declarations

Conflict of interest

Authors declare they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Too, J., Mafarja, M. & Mirjalili, S. Spatial bound whale optimization algorithm: an efficient high-dimensional feature selection approach. Neural Comput & Applic 33, 16229–16250 (2021). https://doi.org/10.1007/s00521-021-06224-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-06224-y

Keywords

Search

Navigation