Skip to main content
SpringerLink
Log in

Binary \(\beta\)-hill climbing optimizer with S-shape transfer function for feature selection

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

Feature selection is an essential stage in many data mining and machine learning and applications that find the proper subset of features from a set of irrelevant, redundant, noisy and high dimensional data. This dimensional reduction is a vital task to increase classification accuracy and thus reduce the processing time. An optimization algorithm can be applied to tackle the feature selection problem. In this paper, a \(\beta\)-hill climbing optimizer is applied to solve the feature selection problem. \(\beta\)-hill climbing is recently introduced as a local-search based algorithm that can obtain pleasing solutions for different optimization problems. In order to tailor \(\beta\)-hill climbing for feature selection, it has to be adapted to work in a binary context. The S-shaped transfer function is used to transform the data into the binary representation. A set of 22 de facto benchmark real-world datasets are used to evaluate the proposed algorithm. The effect of the \(\beta\)-hill climbing parameters on the convergence rate is studied in terms of accuracy, the number of features, fitness values, and computational time. Furthermore, the proposed method is compared against three local search methods and ten metaheuristics methods. The obtained results show that the proposed binary \(\beta\)-hill climbing optimizer outperforms other comparative local search methods in terms of classification accuracy on 16 out of 22 datasets. Furthermore, it overcomes other comparative metaheuristics approaches in terms of classification accuracy in 7 out of 22 datasets. The obtained results prove the efficiency of the proposed binary \(\beta\)-hill climbing optimizer.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2

Similar content being viewed by others

References

  • Abed-alguni B, Klaib A (2018) Hybrid whale optimization and \(\beta\)-hill climbing algorithm. Int J CompuT Sci Math, pp 1–13

  • Abed-alguni BH, Alkhateeb F (2018) Intelligent hybrid cuckoo search and \(\beta\)-hill climbing algorithm. J King Saud Univ Comput Inf Sci 32(2):159–173

    Google Scholar 

  • Abualigah LM, Khader AT, Al-Betar MA (2017a) \(\beta\)-hill climbing technique for the text document clustering. New Trends in Information Technology NTIT2017 Conference, Amman, Jordan, IEEE, pp 1–6

  • Abualigah LM, Khadery AT, Al-Betar MA, Alyasseri ZAA, Alomari OA, Hanandehk ES (2017b) Feature selection with \(\beta\)-hill climbing search for text clustering application. Second Palestinian International Conference on Information and Communication Technology (PICICT 2017), Gaza, Palestine, IEEE, pp 22–27

  • Al-Abdallah RZ, Jaradat AS, Doush IA, Jaradat YA (2017) Abinary classifier based on firefly algorithm. Jordan J Comput Inf Technol (JJCIT) 3(3)

  • Al-Betar MA (2017) \(\beta\)-hill climbing: an exploratory local search. Neural Comput Appl 28(1):153–168. https://doi.org/10.1007/s00521-016-2328-2

    Article  Google Scholar 

  • Al-Betar MA, Awadallah MA, Bolaji AL, Alijla BO (2017) \(\beta\)-hill climbing algorithm for sudoku game. Second Palestinian International Conference on Information and Communication Technology (PICICT 2017), Gaza, Palestine, IEEE, pp 84–88

  • Al-Betar MA, Awadallah MA, Abu Doush I, Alsukhni E, ALkhraisat H (2018) A non-convex economic dispatch problem with valve loading effect using a new modified \(\beta\)-hill climbing local search algorithm. Arab J Sci Eng. https://doi.org/10.1007/s13369-018-3098-1

    Article  Google Scholar 

  • Aljarah I, Mafarja M, Heidari AA, Faris H, Zhang Y, Mirjalili S (2018) Asynchronous accelerating multi-leader salp chains for feature selection. Appl Soft Comput 71:964–979

    Google Scholar 

  • Alomari OA, Khader AT, Al-Betar MA, Alyasseri ZAA (2018a) A hybrid filter-wrapper gene selection method for cancer classification. 2018 2nd International Conference on BioSignal Analysis. Processing and Systems (ICBAPS), IEEE, pp 113–118

  • Alomari OA, Khader AT, Al-Betar MA, Awadallah MA (2018b) A novel gene selection method using modified mrmr and hybrid bat-inspired algorithm with \(\beta\)-hill climbing. Appl Intell 48(11):4429–4447

    Google Scholar 

  • Alsaafin A, Elnagar A (2017) A minimal subset of features using feature selection for handwritten digit recognition. J Intell Learn Syst Appl 9(4):55–68

    Google Scholar 

  • Alsukni E, Arabeyyat OS, Awadallah MA, Alsamarraie L, Abu-Doush I, Al-Betar MA (2017) Multiple-reservoir scheduling using \(\beta\)-hill climbing algorithm. J Intell Syst 28(4):559–570

    Google Scholar 

  • Alyasseri ZAA, Khader AT, Al-Betar MA (2017a) Optimal eeg signals denoising using hybrid \(\beta\)-hill climbing algorithm and wavelet transform. ICISPC ’17. Penang, Malaysia, ACM, pp 5–11

  • Alyasseri ZAA, Khader AT, Al-Betar MA (2017b) Optimal electroencephalogram signals denoising using hybrid \(\beta\)-hill climbing algorithm and wavelet transform. In: Proceedings of the International Conference on Imaging, Signal Processing and Communication, ACM, pp 106–112

  • Alyasseri ZAA, Khader AT, Al-Betar MA, Awadallah MA (2018) Hybridizing \(\beta\)-hill climbing with wavelet transform for denoising ecg signals. Inf Sci 429:229–246

    MathSciNet  Google Scholar 

  • Alzaidi AA, Ahmad M, Doja MN, Al Solami E, Beg MS (2018) A new 1d chaotic map and \(beta\)-hill climbing for generating substitution-boxes. IEEE Access 6:55405–55418

    Google Scholar 

  • Bermejo P, Gámez JA, Puerta JM (2011) A grasp algorithm for fast hybrid (filter-wrapper) feature subset selection in high-dimensional datasets. Pattern Recognit Lett 32(5):701–711

    Google Scholar 

  • Bermejo P, Gámez JA, Puerta JM (2014) Speeding up incremental wrapper feature subset selection with naive bayes classifier. Knowl-Based Syst 55(Supplement C):14–147, https://doi.org/10.1016/j.knosys.2013.10.016, http://www.sciencedirect.com/science/article/pii/S0950705113003274

  • Blum C, Roli A (2003) Metaheuristics in combinatorial optimization: overview and conceptual comparison. ACM Comput Surv (CSUR) 35(3):268–308

    Google Scholar 

  • Bolón-Canedo V, Alonso-Betanzos A (2019) Ensembles for feature selection: a review and future trends. Inf Fusion 52:1–12

    Google Scholar 

  • Boughaci D, Alkhawaldeh AAs (2018) Three local search-based methods for feature selection in credit scoring. Vietnam J Comput Sci 5(2):107–121

    Google Scholar 

  • Chen Y, Garcia EK, Gupta MR, Rahimi A, Cazzanti L (2009) Similarity-based classification: concepts and algorithms. J Mach Learn Res 10(Mar):747–776

    MathSciNet  MATH  Google Scholar 

  • Corana A, Marchesi M, Martini C, Ridella S (1987) Minimizing multimodal functions of continuous variables with the “simulated annealing” algorithm corrigenda for this article is available here. ACM Trans Math Softw (TOMS) 13(3):262–280

    MATH  Google Scholar 

  • Cover TM, Thomas JA (2012) Elements of information theory. Wiley, Hoboken

    MATH  Google Scholar 

  • Delen D, Walker G, Kadam A (2005) Predicting breast cancer survivability: a comparison of three data mining methods. Artif Intell Med 34(2):113–127

    Google Scholar 

  • Deng X, Li Y, Weng J, Zhang J (2019) Feature selection for text classification: a review. Multimed Tools Appl 78(3):3797–3816

    Google Scholar 

  • Doush IA, Sahar AB (2017) Currency recognition using a smartphone: comparison between color sift and gray scale sift algorithms. J King Saud Univ Comput Inf Sci 29(4):484–492

    Google Scholar 

  • Dubey SR, Singh SK, Singh RK (2015) Local wavelet pattern: a new feature descriptor for image retrieval in medical ct databases. IEEE Trans Image Process 24(12):5892–5903

    MathSciNet  MATH  Google Scholar 

  • Duda RO, Hart PE, Stork DG (2012) Pattern classification. Wiley, Hoboken

    MATH  Google Scholar 

  • Emary E, Zawbaa HM, Hassanien AE (2016) Binary ant lion approaches for feature selection. Neurocomputing 213:54–65

    Google Scholar 

  • Forman G (2003) An extensive empirical study of feature selection metrics for text classification. J Mach Learn Res 3(Mar):1289–1305

    MATH  Google Scholar 

  • García S, Fernández A, Luengo J, Herrera F (2010) Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: Experimental analysis of power. Inf Sci 180(10):2044–2064

    Google Scholar 

  • Ghareb AS, Bakar AA, Hamdan AR (2016) Hybrid feature selection based on enhanced genetic algorithm for text categorization. Expert Syst Appl 49:31–47

    Google Scholar 

  • Gheyas IA, Smith LS (2010) Feature subset selection in large dimensionality domains. Pattern Recognit 43(1):5–13. https://doi.org/10.1016/j.patcog.2009.06.009, http://www.sciencedirect.com/science/article/pii/S0031320309002520

  • Goltsev A, Gritsenko V (2012) Investigation of efficient features for image recognition by neural networks. Neural Netw 28:15–23

    Google Scholar 

  • Gu S, Cheng R, Jin Y (2018) Feature selection for high-dimensional classification using a competitive swarm optimizer. Soft Comput 22(3):811–822

    Google Scholar 

  • Hall MA, Smith LA (1999) Feature selection for machine learning: comparing a correlation-based filter approach to the wrapper. FLAIRS Conf 1999:235–239

    Google Scholar 

  • Hu Q, Yu D, Xie Z (2006) Information-preserving hybrid data reduction based on fuzzy-rough techniques. Pattern Recognit Lett 27(5):414–423

    Google Scholar 

  • Jing LP, Huang HK, Shi HB (2002) Improved feature selection approach tfidf in text mining. In: Proceedings. International Conference on Machine Learning and Cybernetics, IEEE, vol 2, pp 944–946

  • Kabir MM, Shahjahan M, Murase K (2012) A new hybrid ant colony optimization algorithm for feature selection. Expert Syst Appl 39(3):3747–3763

    Google Scholar 

  • Kashef S, Nezamabadi-pour H (2015) An advanced aco algorithm for feature subset selection. Neurocomputing 147:271–279

    Google Scholar 

  • Kennedy J, Eberhart RC (1997) A discrete binary version of the particle swarm algorithm. In: 1997 IEEE International conference on systems, man, and cybernetics. Computational cybernetics and simulation, IEEE, vol 5, pp 4104–4108

  • Lai C, Reinders MJ, Wessels L (2006) Random subspace method for multivariate feature selection. Pattern Recognit Lett 27(10):1067–1076

    Google Scholar 

  • Lee C, Lee GG (2006) Information gain and divergence-based feature selection for machine learning-based text categorization. Inf Process Manag 42(1):155–165

    Google Scholar 

  • Li P, Shrivastava A, Moore JL, König AC (2011) Hashing algorithms for large-scale learning. In: Advances in neural information processing systems, pp 2672–2680

  • Li Y, Li T, Liu H (2017) Recent advances in feature selection and its applications. Knowl Inf Syst 53(3):551–577

    Google Scholar 

  • Liao Y, Vemuri VR (2002) Use of k-nearest neighbor classifier for intrusion detection. Comput Secur 21(5):439–448

    Google Scholar 

  • Liu H, Setiono R (1995) Chi2: Feature selection and discretization of numeric attributes, pp 388–391

  • Ma B, Xia Y (2017) A tribe competition-based genetic algorithm for feature selection in pattern classification. Appl Soft Comput 58(Supplement C):328–338, https://doi.org/10.1016/j.asoc.2017.04.042, http://www.sciencedirect.com/science/article/pii/S1568494617302247

  • Mafarja M, Mirjalili S (2018) Whale optimization approaches for wrapper feature selection. Appl Soft Comput 62:441–453

    Google Scholar 

  • Mafarja M, Aljarah I, Heidari AA, Faris H, Fournier-Viger P, Li X, Mirjalili S (2018a) Binary dragonfly optimization for feature selection using time-varying transfer functions. Knowl-Based Syst 161:185–204

    Google Scholar 

  • Mafarja M, Aljarah I, Heidari AA, Hammouri AI, Faris H, Ala’M AZ, Mirjalili S (2018b) Evolutionary population dynamics and grasshopper optimization approaches for feature selection problems. Knowl-Based Syst 145:25–45

    Google Scholar 

  • Mafarja M, Aljarah I, Faris H, Hammouri AI, Ala’M AZ, Mirjalili S (2019) Binary grasshopper optimisation algorithm approaches for feature selection problems. Expert Syst Appl 117:267–286

    Google Scholar 

  • Marinaki M, Marinakis Y (2015) A hybridization of clonal selection algorithm with iterated local search and variable neighborhood search for the feature selection problem. Memetic Comput 7(3):181–201

    Google Scholar 

  • Mashrgy MA, Bdiri T, Bouguila N (2014) Robust simultaneous positive data clustering and unsupervised feature selection using generalized inverted dirichlet mixture models. Knowl-Based Syst 59(Supplement C):182–195, https://doi.org/10.1016/j.knosys.2014.01.007, http://www.sciencedirect.com/science/article/pii/S0950705114000185

  • Mlakar U, Fister I, Brest J, Potočnik B (2017) Multi-objective differential evolution for feature selection in facial expression recognition systems. Expert Syst Appl 89:129–137

    Google Scholar 

  • Moayedikia A, Ong KL, Boo YL, Yeoh WG, Jensen R (2017) Feature selection for high dimensional imbalanced class data using harmony search. Eng Appl Artif Intell 57:38–49

    Google Scholar 

  • Park CH, Kim SB (2015) Sequential random k-nearest neighbor feature selection for high-dimensional data. Expert Syst Appl 42(5):2336–2342. https://doi.org/10.1016/j.eswa.2014.10.044, http://www.sciencedirect.com/science/article/pii/S095741741400668X

  • Rashedi E, Nezamabadi-Pour H, Saryazdi S (2013) A simultaneous feature adaptation and feature selection method for content-based image retrieval systems. Knowl-Based Syst 39:85–94

    Google Scholar 

  • Ravisankar P, Ravi V, Rao GR, Bose I (2011) Detection of financial statement fraud and feature selection using data mining techniques. Decision Support Syst 50(2):491–500

    Google Scholar 

  • Robnik-Šikonja M, Kononenko I (2003) Theoretical and empirical analysis of relieff and rrelieff. Mach Learn 53(1):23–69

    MATH  Google Scholar 

  • Saeys Y, Inza I, Larrañaga P (2007) A review of feature selection techniques in bioinformatics. Bioinformatics 23(19):2507–2517

    Google Scholar 

  • Sawalha R, Doush IA (2012) Face recognition using harmony search-based selected features. Int J Hybrid Inf Technol 5(2):1–16

    Google Scholar 

  • Sayed GI, Hassanien AE, Azar AT (2019) Feature selection via a novel chaotic crow search algorithm. Neural Comput Appl 31(1):171–188

    Google Scholar 

  • Shang C, Li M, Feng S, Jiang Q, Fan J (2013) Feature selection via maximizing global information gain for text classification. Knowl-Based Syst 54(Supplement C):298 – 309, https://doi.org/10.1016/j.knosys.2013.09.019, http://www.sciencedirect.com/science/article/pii/S0950705113003067

  • Shao C, Paynabar K, Kim TH, Jin JJ, Hu SJ, Spicer JP, Wang H, Abell JA (2013) Feature selection for manufacturing process monitoring using cross-validation. J Manuf Syst 32(4):550–555

    Google Scholar 

  • Sindhu SSS, Geetha S, Kannan A (2012) Decision tree based light weight intrusion detection using a wrapper approach. Expert Syst Appl 39(1):129–141. https://doi.org/10.1016/j.eswa.2011.06.013, http://www.sciencedirect.com/science/article/pii/S0957417411009080

  • Talbi EG (2009) Metaheuristics: from design to implementation, vol 74. Wiley, Hoboken

    MATH  Google Scholar 

  • Taradeh M, Mafarja M, Heidari AA, Faris H, Aljarah I, Mirjalili S, Fujita H (2019) An evolutionary gravitational search-based feature selection. Inf Sci 497:219–239

    Google Scholar 

  • Urbanowicz RJ, Olson RS, Schmitt P, Meeker M, Moore JH (2018) Benchmarking relief-based feature selection methods for bioinformatics data mining. J Biomed Inform 85:168–188

    Google Scholar 

  • Weinberger KQ, Saul LK (2009) Distance metric learning for large margin nearest neighbor classification. J Mach Learn Res 10(Feb):207–244

    MATH  Google Scholar 

  • Wieland M, Pittore M (2014) Performance evaluation of machine learning algorithms for urban pattern recognition from multi-spectral satellite images. Remote Sens 6(4):2912–2939

    Google Scholar 

  • Wolpert DH, Macready WG et al (1997) No free lunch theorems for optimization. IEEE Trans Evol Comput 1(1):67–82

    Google Scholar 

  • Yu L, Liu H (2003) Feature selection for high-dimensional data: a fast correlation-based filter solution. In: Proceedings of the 20th international conference on machine learning (ICML-03), pp 856–863

  • Zhang H, Sun G (2002) Feature selection using tabu search method. Pattern Recognit 35(3):701–711

    MATH  Google Scholar 

  • Zhang L, Mistry K, Lim CP, Neoh SC (2018) Feature selection using firefly optimization for classification and regression models. Decision Support Syst 106:64–85

    Google Scholar 

  • Zhang Y, Wang S, Phillips P, Ji G (2014) Binary pso with mutation operator for feature selection using decision tree applied to spam detection. Knowl-Based Syst 64:22–31

    Google Scholar 

  • Zhao Z, Liu H (2007) Spectral feature selection for supervised and unsupervised learning. In: Proceedings of the 24th international conference on Machine learning, ACM, pp 1151–1157

  • Zhong N, Dong J, Ohsuga S (2001) Using rough sets with heuristics for feature selection. J Intell Inf Syst 16(3):199–214

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Information Technology - MSAI, College of Engineering and Information Technology, Ajman University, Ajman, UAE

    Mohammed Azmi Al-Betar

  2. Department of Information Technology, Al-Huson University College, Al-Balqa Applied University, P.O. Box 50, Al-Huson, Irbid, Jordan

    Mohammed Azmi Al-Betar

  3. Department of Computer Information Systems, Al-Balqa Applied University, Al-Salt, Jordan

    Abdelaziz I. Hammouri

  4. Department of Computer Science, Al-Aqsa University, P. O. Box 4051, Gaza, Palestine

    Mohammed A. Awadallah

  5. Computer Science Department, American University of Kuwait, Salmiya, Kuwait

    Iyad Abu Doush

  6. Computer Science Department, Yarmouk University, Irbid, Jordan

    Iyad Abu Doush

Authors
  1. Mohammed Azmi Al-Betar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Abdelaziz I. Hammouri
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohammed A. Awadallah
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Iyad Abu Doush
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohammed A. Awadallah.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Al-Betar, M.A., Hammouri, A.I., Awadallah, M.A. et al. Binary \(\beta\)-hill climbing optimizer with S-shape transfer function for feature selection. J Ambient Intell Human Comput 12, 7637–7665 (2021). https://doi.org/10.1007/s12652-020-02484-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-020-02484-z

Keywords

Search

Navigation