Skip to main content
SpringerLink
Log in

S-shaped and V-shaped gaining-sharing knowledge-based algorithm for feature selection

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

In machine learning, searching for the optimal feature subset from the original datasets is a very challenging and prominent task. The metaheuristic algorithms are used in finding out the relevant, important features, that enhance the classification accuracy and save the resource time. Most of the algorithms have shown excellent performance in solving feature selection problems. A recently developed metaheuristic algorithm, gaining-sharing knowledge-based optimization algorithm (GSK), is considered for finding out the optimal feature subset. GSK algorithm was proposed over continuous search space; therefore, a total of eight S-shaped and V-shaped transfer functions are employed to solve the problems into binary search space. Additionally, a population reduction scheme is also employed with the transfer functions to enhance the performance of proposed approaches. It explores the search space efficiently and deletes the worst solutions from the search space, due to the updation of population size in every iteration. The proposed approaches are tested over twenty-one benchmark datasets from UCI repository. The obtained results are compared with state-of-the-art metaheuristic algorithms including binary differential evolution algorithm, binary particle swarm optimization, binary bat algorithm, binary grey wolf optimizer, binary ant lion optimizer, binary dragonfly algorithm, binary salp swarm algorithm. Among eight transfer functions, V4 transfer function with population reduction on binary GSK algorithm outperforms other optimizers in terms of accuracy, fitness values and the minimal number of features. To investigate the results statistically, two non-parametric statistical tests are conducted that concludes the superiority of the proposed approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Abd Elaziz M, Heidari AA, Fujita H, Moayedi H (2020) A competitive chain-based harris hawks optimizer for global optimization and multi-level image thresholding problems. Appl Soft Comput 106347

  2. Abe S (2005) Modified backward feature selection by cross validation. In: ESANN. Citeseer, pp 163–168

  3. Agrawal P, Ganesh T, Mohamed AW (2020) A novel binary gaining–sharing knowledge-based optimization algorithm for feature selection. Neural Comput Applic 1–20

  4. Al-Madi N, Faris H, Mirjalili S (2019) Binary multi-verse optimization algorithm for global optimization and discrete problems. Int J Mach Learn Cybern 10(12):3445–3465

    Article  Google Scholar 

  5. Allam M, Nandhini M (2018) Optimal feature selection using binary teaching learning based optimization algorithm. J King Saud University-Comput Inf Sci

  6. Brest J, Maučec MS (2011) Self-adaptive differential evolution algorithm using population size reduction and three strategies. Soft Comput 15(11):2157–2174

    Article  Google Scholar 

  7. Cheng J, Zhang G, Neri F (2013) Enhancing distributed differential evolution with multicultural migration for global numerical optimization. Inf Sci 247:72–93

    Article  MathSciNet  MATH  Google Scholar 

  8. Chuang LY, Chang HW, Tu CJ, Yang CH (2008) Improved binary pso for feature selection using gene expression data. Comput Biol Chem 32(1):29–38

    Article  MATH  Google Scholar 

  9. Dash M, Liu H (1997) Feature selection for classification. Intell Data Anal 1(3):131–156

    Article  Google Scholar 

  10. Ding C, Peng H (2005) Minimum redundancy feature selection from microarray gene expression data. J Bioinf Comput Biol 3(02):185–205

    Article  Google Scholar 

  11. Emary E, Zawbaa HM, Hassanien AE (2016) Binary ant lion approaches for feature selection. Neurocomputing 213:54–65

    Article  Google Scholar 

  12. Emary E, Zawbaa HM, Hassanien AE (2016) Binary grey wolf optimization approaches for feature selection. Neurocomputing 172:371–381

    Article  Google Scholar 

  13. Faris H, Ala’M AZ, Heidari AA, Aljarah I, Mafarja M, Hassonah MA, Fujita H (2019) An intelligent system for spam detection and identification of the most relevant features based on evolutionary random weight networks. Inf Fus 48:67–83

    Article  Google Scholar 

  14. Faris H, Mafarja MM, Heidari AA, Aljarah I, Ala’M AZ, Mirjalili S, Fujita H (2018) An efficient binary salp swarm algorithm with crossover scheme for feature selection problems. Knowl Based Sys 154:43–67

    Article  Google Scholar 

  15. Firpi HA, Goodman E (2004) Swarmed feature selection. In: 33Rd applied imagery pattern recognition workshop (AIPR’04). IEEE, pp 112–118

  16. Frank A, Asuncion A et al (2011) Uci machine learning repository, 2010. URL http://archive.ics.uci.edu/ml 15, 22

  17. Gao WF, Yen GG, Liu SY (2014) A dual-population differential evolution with coevolution for constrained optimization. IEEE Trans Cybern 45(5):1108–1121

    Article  Google Scholar 

  18. Garcìa S, Molina D, Lozano M, Herrera F (2009) A study on the use of non-parametric tests for analyzing the evolutionary algorithms’ behaviour: a case study on the cec’2005 special session on real parameter optimization. J Heuristics 15(6):617

    Article  MATH  Google Scholar 

  19. Ghimatgar H, Kazemi K, Helfroush MS, Aarabi A (2018) An improved feature selection algorithm based on graph clustering and ant colony optimization. Knowl-Based Syst 159:270–285

    Article  Google Scholar 

  20. Guan SU, Liu J, Qi Y (2004) An incremental approach to contribution-based feature selection. J Intell Syst 13(1):15– 42

    Google Scholar 

  21. Hammouri AI, Mafarja M, Al-Betar MA, Awadallah MA, Abu-Doush I (2020) An improved Dragonfly Algorithm for feature selection, Knowl-Based Syst, 203

  22. He X, Zhang Q, Sun N, Dong Y (2009) Feature selection with discrete binary differential evolution. In: 2009 International conference on artificial intelligence and computational intelligence, vol 4. IEEE, pp 327–330

  23. Hsu CN, Huang HJ, Dietrich S (2002) The annigma-wrapper approach to fast feature selection for neural nets. IEEE Trans Sys Man Cybern Part B (Cybernetics) 32(2):207–212

    Article  Google Scholar 

  24. Hu P, Pan JS, Chu SC (2020) Improved binary grey wolf optimizer and its application for feature selection. Knowl Based Sys 105746

  25. Huang CL, Tsai CY (2009) A hybrid sofm-svr with a filter-based feature selection for stock market forecasting. Expert Syst Appl 36(2):1529–1539

    Article  MathSciNet  Google Scholar 

  26. Huang J, Cai Y, Xu X (2007) A hybrid genetic algorithm for feature selection wrapper based on mutual information. Pattern Recogn Lett 28(13):1825–1844

    Article  Google Scholar 

  27. John H (1975) Holland, adaptation in natural and artificial systems. University of Michigan Press, Ann Arbor MI

    Google Scholar 

  28. Kanan HR, Faez K (2008) An improved feature selection method based on ant colony optimization (aco) evaluated on face recognition system. Appl Math Comput 205(2):716–725

    MATH  Google Scholar 

  29. Kennedy J, Eberhart RC (1997) A discrete binary version of the particle swarm algorithm. In: 1997 IEEE International conference on systems, man, and cybernetics. Computational cybernetics and simulation, vol 5. IEEE, pp 4104–4108

  30. Lai C, Reinders MJ, Wessels L (2006) Random subspace method for multivariate feature selection. Patt Recogn Lett 27(10):1067–1076

    Article  Google Scholar 

  31. Leardi R (1994) Application of a genetic algorithm to feature selection under full validation conditions and to outlier detection. J Chemom 8(1):65–79

    Article  Google Scholar 

  32. Liu H, Motoda H (1998) Feature extraction, construction and selection: A data mining perspective, vol 453. Springer Science & Business Media, New York

    Book  MATH  Google Scholar 

  33. Mafarja M, Aljarah I, Heidari AA, Faris H, Fournier-Viger P, Li X, Mirjalili S (2018) Binary dragonfly optimization for feature selection using time-varying transfer functions. Knowl-Based Syst 161:185–204

    Article  Google Scholar 

  34. Mafarja M, Aljarah I, Heidari AA, Hammouri AI, Faris H, Ala’M AZ, Mirjalili S (2018) Evolutionary population dynamics and grasshopper optimization approaches for feature selection problems. Knowl-Based Sys 145:25–45

    Article  Google Scholar 

  35. Mafarja MM, Eleyan D, Jaber I, Hammouri A, Mirjalili S (2017) Binary dragonfly algorithm for feature selection. In: 2017 International conference on new trends in computing sciences (ICTCS). IEEE, pp 12–17

  36. Mirjalili S, Lewis A (2013) S-shaped versus v-shaped transfer functions for binary particle swarm optimization. Swarm Evol Comput 9:1–14

    Article  Google Scholar 

  37. Mohamed AK, Mohamed AW, Elfeky EZ, Saleh M (2018) Enhancing agde algorithm using population size reduction for global numerical optimization. In: International conference on advanced machine learning technologies and applications. Springer, pp 62–72

  38. Mohamed AW, Hadi AA, Mohamed AK (2020) Gaining-sharing knowledge based algorithm for solving optimization problems: a novel nature-inspired algorithm. Int J Mach Learn Cybern 11:1501–1529

    Article  Google Scholar 

  39. Mohamed AW, Sabry HZ (2012) Constrained optimization based on modified differential evolution algorithm. Inf Sci 194:171–208

    Article  Google Scholar 

  40. Moradi P, Gholampour M (2016) A hybrid particle swarm optimization for feature subset selection by integrating a novel local search strategy. Appl Soft Comput 43:117–130

    Article  Google Scholar 

  41. Nakamura RY, Pereira LA, Costa KA, Rodrigues D, Papa JP, Yang XS (2012) Bba: a binary bat algorithm for feature selection. In: 2012 25Th SIBGRAPI conference on graphics, patterns and images. IEEE, pp 291–297

  42. Pashaei E, Aydin N (2017) Binary black hole algorithm for feature selection and classification on biological data. Appl Soft Comput 56:94–106

    Article  Google Scholar 

  43. Rashedi E, Nezamabadi-Pour H (2014) Feature subset selection using improved binary gravitational search algorithm. J Intell Fuzz Sys 26(3):1211–1221

    Article  Google Scholar 

  44. Rashedi E, Nezamabadi-Pour H, Saryazdi S (2010) Bgsa: binary gravitational search algorithm. Nat Comput 9(3):727–745

    Article  MathSciNet  MATH  Google Scholar 

  45. Rodrigues D, Pereira LA, Almeida T, Papa JP, Souza A, Ramos CC, Yang XS (2013) Bcs: a binary cuckoo search algorithm for feature selection. In: 2013 IEEE International symposium on circuits and systems (ISCAS2013). IEEE, pp 465–468

  46. Sayed GI, Khoriba G, Haggag MH (2018) A novel chaotic salp swarm algorithm for global optimization and feature selection. Appl Intell 48(10):3462–3481

    Article  Google Scholar 

  47. Sayed GI, Tharwat A, Hassanien AE (2019) Chaotic dragonfly algorithm: an improved metaheuristic algorithm for feature selection. Appl Intell 49(1):188–205

    Article  Google Scholar 

  48. Schiezaro M, Pedrini H (2013) Data feature selection based on artificial bee colony algorithm. EURASIP J Image Video Process 2013(1):47

    Article  Google Scholar 

  49. Sivagaminathan RK, Ramakrishnan S (2007) A hybrid approach for feature subset selection using neural networks and ant colony optimization. Expert Sys Appl 33(1):49–60

    Article  Google Scholar 

  50. Sun Y. (2007) Iterative relief for feature weighting: algorithms, theories, and applications. IEEE Trans Patt Anal Mach Intell 29(6):1035–1051

    Article  Google Scholar 

  51. Taradeh M, Mafarja M, Heidari AA, Faris H, Aljarah I, Mirjalili S, Fujita H (2019) An evolutionary gravitational search-based feature selection. Inf Sci 497:219–239

    Article  Google Scholar 

  52. Tawhid MA, Dsouza KB (2018) Hybrid binary bat enhanced particle swarm optimization algorithm for solving feature selection problems. Appl Comput Inf

  53. Tubishat M, Abushariah MA, Idris N, Aljarah I (2019) Improved whale optimization algorithm for feature selection in arabic sentiment analysis. Appl Intell 49(5):1688–1707

    Article  Google Scholar 

  54. Wan Y, Wang M, Ye Z, Lai X (2016) A feature selection method based on modified binary coded ant colony optimization algorithm. Appl Soft Comput 49:248–258

    Article  Google Scholar 

  55. Wang H, Jing X, Niu B (2017) A discrete bacterial algorithm for feature selection in classification of microarray gene expression cancer data. Knowl-Based Syst 126:8–19

    Article  Google Scholar 

  56. Xue B, Zhang M, Browne WN (2012) Particle swarm optimization for feature selection in classification: a multi-objective approach. IEEE Trans Cybern 43(6):1656–1671

    Article  Google Scholar 

  57. Yan C, Ma J, Luo H, Patel A (2019) Hybrid binary coral reefs optimization algorithm with simulated annealing for feature selection in high-dimensional biomedical datasets. Chemometr Intell Lab Syst 184:102–111

    Article  Google Scholar 

  58. Yan Z, Yuan C (2004) Ant colony optimization for feature selection in face recognition. In: International conference on biometric authentication. Springer, pp 221–226

  59. Yang J, Honavar V (1998) Feature subset selection using a genetic algorithm. In: Feature extraction, construction and selection. Springer, pp 117–136

  60. Yu H, Gu G, Liu H, Shen J, Zhao J (2009) A modified ant colony optimization algorithm for tumor marker gene selection. Genom Proteom Bioinf 7(4):200–208

    Article  Google Scholar 

  61. Zawbaa HM, Emary E, Parv B, Sharawi M (2016) Feature selection approach based on moth-flame optimization algorithm. In: 2016 IEEE Congress on evolutionary computation (CEC). IEEE, pp 4612–4617

  62. Zhang H, Liang Z, Liu H, Wang R, Liu Y (2020) Ensemble framework by using nature inspired algorithms for the early-stage forest fire rescue—a case study of dynamic optimization problems. Eng Appl Artif Intell 90:103517

    Article  Google Scholar 

  63. Zhang H, Wang R, Liu H, Luo H, Liu Y (2019) Mcdmsr: multicriteria decision making selection/replacement based on agility strategy for real optimization problems. Appl Intell 49 (8):2918–2941

    Article  Google Scholar 

  64. Zhang WQ, Zhang Y, Peng C (2019) Brain storm optimization for feature selection using new individual clustering and updating mechanism. Appl Intell 49(12):4294–4302

    Article  Google Scholar 

  65. Zhang Y, Li HG, Wang Q, Peng C (2019) A filter-based bare-bone particle swarm optimization algorithm for unsupervised feature selection. Appl Intell 49(8):2889–2898

    Article  Google Scholar 

  66. Zhang Y, Song XF, Gong DW (2017) A return-cost-based binary firefly algorithm for feature selection. Inf Sci 418:561–574

    Article  Google Scholar 

  67. Zhu Y, Liang J, Chen J, Ming Z (2017) An improved nsga-iii algorithm for feature selection used in intrusion detection. Knowl-Based Syst 116:74–85

    Article  Google Scholar 

Download references

Acknowledgment

The authors would like to acknowledge the Editors and anonymous reviewers for providing their valuable comments and suggestions.

Author information

Authors and Affiliations

  1. Department of Mathematics and Scientific Computing, National Institute of Technology Hamirpur, Himachal Pradesh, 177005, India

    Prachi Agrawal & Talari Ganesh

  2. IN3 - Computer Science Department, Universitat Oberta de Catalunya, Castelldefels, Spain

    Diego Oliva

  3. Departamento de Ciencias Computacionales, Universidad de Guadalajara, CUCEI, Guadalajara, Jal, Mexico

    Diego Oliva

  4. Operations Research Department, Faculty of Graduate Studies for Statistical Research, Cairo University, 12613, Giza, Egypt

    Ali Wagdy Mohamed

  5. Wireless Intelligent Networks Center (WINC), School of Engineering and Applied Sciences, Nile University, Giza, Egypt

    Ali Wagdy Mohamed

Authors
  1. Prachi Agrawal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Talari Ganesh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Diego Oliva
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ali Wagdy Mohamed
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ali Wagdy Mohamed.

Ethics declarations

Conflict of interests

The authors have declared no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

Table 11 Average fitness value of PbGSK-V4 against with other metaheuristic algorithms
Full size table
Table 12 Average classification accuracy values of the proposed approach PbGSK-V4 against with other metaheuristic algorithms
Full size table
Table 13 Average number of selected features of the proposed approach PbGSK-V4 against with other metaheuristic algorithms
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Agrawal, P., Ganesh, T., Oliva, D. et al. S-shaped and V-shaped gaining-sharing knowledge-based algorithm for feature selection. Appl Intell 52, 81–112 (2022). https://doi.org/10.1007/s10489-021-02233-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02233-5

Keywords

Search

Navigation