Skip to main content

Advertisement

SpringerLink
Log in

A novel hybrid wrapper–filter approach based on genetic algorithm, particle swarm optimization for feature subset selection

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

The classification is one of the main technique of machine learning science. In many problems, the data sets have a high dimensionality that the existence of all features is not important to the purpose of the problem, and this will decrease the accuracy and performance of the algorithm. In this situation, the feature selection will play a significant role, and by eliminating unrelated features, the efficiency of the algorithm will be increased. A hybrid filter-wrapper method is proposed in the present study for feature subset selection established with integration of evolutionary based genetic algorithms (GA) and particle swarm optimization (PSO). The presented method mainly aims to reduce the complication of calculation and the search time expended to achieve an optimum solution to the high dimensional datasets feature selection problem. The proposed method, named smart HGP-FS, utilizes artificial neural network (ANN) in the fitness function. The filter and wrapper methods are integrated in order to take the benefit of filter technique acceleration and the wrapper technique vigor for selection of dataset efficacious characteristics. Some dataset characteristics are eliminated through the filter phase, which in turn reduces complex computations and search time in the wrapper phase. Comparisons have been made for the effectiveness of the proposed hybrid algorithm with the usability of three hybrid filter-wrapper methods, two pure wrapper algorithms, two pure filter procedures, and two traditional wrapper feature selection techniques. The findings obtained over real-world datasets show the efficiency of the presented algorithm. The outcomes of algorithm examination on five datasets reveal that the developed method is able to obtain a more accurate classification and to remove unsuitable and unessential characteristics more effectively relative to the other approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  • Apolloni J, Leguizamón G, Alba E (2016) Two hybrid wrapper-filter feature selection algorithms applied to high-dimensional microarray experiments. Appl Soft Comput 38:922–932

    Article  Google Scholar 

  • Bennasar M, Hicks Y, Setchi R (2015) Feature selection using joint mutual information maximisation. Expert Syst Appl 42(22):8520–8532

    Article  Google Scholar 

  • Bostani H, Sheikhan M (2017) Hybrid of binary gravitational search algorithm and mutual information for feature selection in intrusion detection systems. Soft Comput 21(9):2307–2324

    Article  Google Scholar 

  • Butler-Yeoman T, Xue B, Zhang M (2015) Particle swarm optimisation for feature selection: a hybrid filter-wrapper approach. In: CEC, pp 2428–2435

  • Cervante L, Xue B, Zhang M, Shang L (2012) Binary particle swarm optimisation for feature selection: a filter based approach. In: Evolutionary computation (CEC), 2012 IEEE congress. IEEE, pp 1–8

  • Chandrashekar G, Sahin F (2014) A survey on feature selection methods. Comput Electr Eng 40(1):16–28

    Article  Google Scholar 

  • Chen KJ, Ji P (2007) A genetic algorithm for dynamic advanced planning and scheduling (DAPS) with a frozen interval. Expert Syst Appl 33(4):1004–1010

    Article  MathSciNet  Google Scholar 

  • Chinnaswamy A, Srinivasan R (2017) Hybrid information gain based fuzzy roughset feature selection in cancer microarray data. In: Power and advanced computing technologies (i-PACT), 2017 Innovations. IEEE, pp 1–6

  • Chuang LY, Ke CH, Yang CH (2016) A hybrid both filter and wrapper feature selection method for microarray classification. arXiv:1612.08669

  • Dash M, Liu H, Yao J (1997) Dimensionality reduction of unsupervised data. In: Tools with artificial intelligence, 1997. Proceedings, 9th IEEE international conference. IEEE, pp 532–539

  • Ebrahimpour MK, Eftekhari M (2017) Ensemble of feature selection methods: a hesitant fuzzy sets approach. Appl Soft Comput 50:300–312

    Article  Google Scholar 

  • Caruana R, Freitag, D (1994) Greedy attribute selection. In: Proceedings of the 11th international conference on machine learning, New Brunswick

  • Frohlich H, Chapelle O, Scholkopf B (2003) Feature selection for support vector machines by means of genetic algorithm. In: Tools with artificial intelligence, 2003. Proceedings. 15th IEEE international conference. IEEE, pp 142–148

  • Goldberg DE, Holland JH (1988) Genetic algorithms and machine learning. Mach Learn 3(2):95–99

    Article  Google Scholar 

  • Gu Q, Li Z, Han J (2011) Generalized fisher score for feature selection. In: Proceedings of the international conference on uncertainty in artificial intelligence

  • Gu S, Cheng R, Jin Y (2018) Feature selection for high-dimensional classification using a competitive swarm optimizer. Soft Comput 22(3):811–822

    Article  Google Scholar 

  • Guo H, Jack LB, Nandi AK (2005) Feature generation using genetic programming with application to fault classification. IEEE Trans Syst Man Cybern Part B (Cybernetics) 35(1):89–99

    Article  Google Scholar 

  • Gutlein M, Frank E, Hall M, Karwath A (2009) Large-scale attribute selection using wrappers. In: Computational intelligence and data mining, 2009. CIDM’09. IEEE symposium. IEEE, pp 332–339

  • Haeri A, Tavakkoli-Moghaddam R (2012) Developing a hybrid data mining approach based on multi-objective particle swarm optimization for solving a traveling salesman problem. J Bus Econ Manag 13(5):951–967

    Article  Google Scholar 

  • Hammami M, Bechikh S, Hung CC, Said LB (2018) A multi-objective hybrid filter-wrapper evolutionary approach for feature selection. Memet Comput 11(2):193–208

    Article  Google Scholar 

  • Hancer E (2018) Differential evolution for feature selection: a fuzzy wrapper–filter approach. Soft Comput. https://doi.org/10.1007/s00500-018-3545-7

    Article  Google Scholar 

  • Hancer E, Xue B, Karaboga D, Zhang M (2015) A binary ABC algorithm based on advanced similarity scheme for feature selection. Appl Soft Comput 36:334–348

    Article  Google Scholar 

  • He X, Cai D, Niyogi P (2006) Laplacian score for feature selection. In: Advances in neural information processing systems, pp 507–514

  • Hoque N, Ahmed HA, Bhattacharyya DK, Kalita JK (2016) A fuzzy mutual information-based feature selection method for classification. Fuzzy Inf Eng 8(3):355–384

    Article  MathSciNet  Google Scholar 

  • Hsieh Y, Lee P, You P (2018) Immune based evolutionary algorithm for determining the optimal sequence of multiple disinfection operations. Sci Iran. https://doi.org/10.24200/sci.2018.20324

    Article  Google Scholar 

  • Hsu HH, Hsieh CW, Lu MD (2011) Hybrid feature selection by combining filters and wrappers. Expert Syst Appl 38(7):8144–8150

    Article  Google Scholar 

  • Huang CL, Wang CJ (2006) A GA-based feature selection and parameters optimization for support vector machines. Expert Syst Appl 31(2):231–240

    Article  Google Scholar 

  • Huang H, Xie HB, Guo JY, Chen HJ (2011) Ant colony optimization–based feature selection for surface electromyography signals classification. Comput Biol Med 42(1):30–38

    Article  Google Scholar 

  • Ibrahim RA, Ewees AA, Oliva D, Elaziz MA, Lu S (2018) Improved salp swarm algorithm based on particle swarm optimization for feature selection. J Ambient Intell Humaniz Comput. https://doi.org/10.1007/s12652-018-1031-9

    Article  Google Scholar 

  • Indira K, Kanmani S (2015) Mining association rules using hybrid genetic algorithm and particle swarm optimisation algorithm. Int J Data Anal Tech Strateg 7(1):59–76

    Article  Google Scholar 

  • Jayaraman V, Sultana HP (2019) Artificial gravitational cuckoo search algorithm along with particle bee optimized associative memory neural network for feature selection in heart disease classification. J Ambient Intell Humaniz Comput. https://doi.org/10.1007/s12652-019-01193-6

    Article  Google Scholar 

  • Jiang Y, Liu X, Yan G, Xiao J (2017) Modified binary cuckoo search for feature selection: a hybrid filter–wrapper approach. In: 2017 13th International conference on computational intelligence and security (CIS). IEEE, pp 488–491

  • Jović A, Brkić K, Bogunović N (2015) A review of feature selection methods with applications. In: Information and communication technology, electronics and microelectronics (MIPRO) 2015 38th international convention. IEEE, pp 1200–1205

  • Kanan HR, Faez K (2008) An improved feature selection method based on ant colony optimization (ACO) evaluated on face recognition system. Appl Math Comput 205(2):716–725

    MATH  Google Scholar 

  • Lefkovits S, Lefkovits L (2017) Gabor feature selection based on information gain. Proc Eng 181:892–898

    Article  Google Scholar 

  • Leung Y, Hung Y (2010) A multiple-filter-multiple-wrapper approach to gene selection and microarray data classification. IEEE/ACM Trans Comput Biol Bioinf 7(1):108–117

    Article  Google Scholar 

  • Lin JY, Ke HR, Chien BC, Yang WP (2008) Classifier design with feature selection and feature extraction using layered genetic programming. Expert Syst Appl 34(2):1384–1393

    Article  Google Scholar 

  • Liu H, Hussain F, Tan CL, Dash M (2002) Discretization: an enabling technique. Data Min Knowl Discov 6(4):393–423

    Article  MathSciNet  Google Scholar 

  • Liu Y, Wang G, Chen H, Dong H, Zhu X, Wang S (2011) An improved particle swarm optimization for feature selection. J Bionic Eng 8(2):191–200

    Article  Google Scholar 

  • Liu S, Zhang DG, Liu XH, Zhang T, Gao JX, Cui YY (2019) Dynamic analysis for the average shortest path length of mobile ad hoc networks under random failure scenarios. IEEE Access 7:21343–21358

    Article  Google Scholar 

  • Mafarja MM, Mirjalili S (2017) Hybrid whale optimization algorithm with simulated annealing for feature selection. Neurocomputing 260:302–312

    Article  Google Scholar 

  • Moradi P, Gholampour M (2016) A hybrid particle swarm optimization for feature subset selection by integrating a novel local search strategy. Appl Soft Comput 43:117–130

    Article  Google Scholar 

  • Moslehi F, Haeri A (2019) A genetic algorithm based framework for mining quantitative association rules without specifying minimum support and minimum confidence. Sci Iran. https://doi.org/10.24200/SCI.2019.51030.1969

    Article  Google Scholar 

  • Moslehi F, Haeri A, Gholamian M (2019a) A novel selective clustering framework for appropriate labeling of the clusters based on K-means algorithm. Sci Iran. https://doi.org/10.24200/SCI.2019.51110.2010

    Article  Google Scholar 

  • Moslehi F, Haeri A, Gholamian M (2019b) Investigation of effective factors in expanding electronic payment in Iran using datamining techniques. J Ind Syst Eng 12(2):61–94

    Google Scholar 

  • Mukhopadhyay A, Maulik U (2013) An SVM-wrapped multiobjective evolutionary feature selection approach for identifying cancer-microRNA markers. IEEE Trans Nanobiosci 12(4):275–281

    Article  Google Scholar 

  • Muni DP, Pal NR, Das J (2006) Genetic programming for simultaneous feature selection and classifier design. IEEE Trans Syst Man Cybern Part B (Cybernetics) 36(1):106–117

    Article  Google Scholar 

  • Narendra PM, Fukunaga K (1977) A branch and bound algorithm for feature subset selection. IEEE Trans Comput 9:917–922

    Article  MATH  Google Scholar 

  • Omar N, bin Othman MS (2013) Particle swarm optimization feature selection for classification of survival analysis in cancer. Int J Innov Comput 2(1):1–7

    Google Scholar 

  • Oreski S, Oreski G (2014) Genetic algorithm-based heuristic for feature selection in credit risk assessment. Expert Syst Appl 41(4):2052–2064

    Article  Google Scholar 

  • Osanaiye O, Cai H, Choo KKR, Dehghantanha A, Xu Z, Dlodlo M (2016) Ensemble-based multi-filter feature selection method for DDoS detection in cloud computing. EURASIP J Wirel Commun Netw 2016(1):130

    Article  Google Scholar 

  • Peng H, Long F, Ding C (2005) Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans Pattern Anal Mach Intell 27(8):1226–1238

    Article  Google Scholar 

  • Purohit A, Chaudhari N S, Tiwari A (2010) Construction of classifier with feature selection based on genetic programming. In: Evolutionary computation (CEC), 2010 IEEE congress. IEEE, pp 1–5

  • Russell SJ, Norvig P (2008) Artificial intelligence: a modern approach

  • Sadeghi H, Zolfaghari M, Heydarizade M (2011) Estimation of electricity demand in residential sector using genetic algorithm approach. J Ind Eng Prod Res 22(1):43–50

    Google Scholar 

  • Saeys Y, Inza I, Larranaga P (2007) A review of feature selection techniques in bioinformatics. Bioinformatics 23(19):2507–2517

    Article  Google Scholar 

  • Sahu B (2018) A combo feature selection method (filter +wrapper) for microarray gene classification. Int J Pure Appl Math 118(16):389–401

    Google Scholar 

  • Salguero AG, Medina J, Delatorre P, Espinilla M (2018) Methodology for improving classification accuracy using ontologies: application in the recognition of activities of daily living. J Ambient Intell Humaniz Comput 10(6):2125–2142

    Article  Google Scholar 

  • Sampath kumar R, Alagumurthi N, Ramesh R (2010) Integrated total cost and tolerance optimization with genetic algorithm. Int J Comput Intell Syst 3(3):325–333

    Article  Google Scholar 

  • Seal A, Ganguly S, Bhattacharjee D, Nasipuri M, Gonzalo-Martin C (2015) Feature selection using particle swarm optimization for thermal face recognition. In: Applied computation and security systems. Springer, New Delhi, pp 25–35

  • Sheikhan M, Mohammadi N (2012) Neural-based electricity load forecasting using hybrid of GA and ACO for feature selection. Neural Comput Appl 21(8):1961–1970

    Article  Google Scholar 

  • Shreem SS, Abdullah S, Nazri MZA, Alzaqebah M (2012) Hybridizing reliefF, MRMR filters and GA wrapper approaches for gene selection. Theor Appl Inf Technol 46(2):1034–1039

    Google Scholar 

  • Shunmugapriya P, Kanmani S (2017) A hybrid algorithm using ant and bee colony optimization for feature selection and classification (AC-ABC Hybrid). Swarm Evolut Comput 36:27–36

    Article  Google Scholar 

  • Su S, Zhao S (2017) A hierarchical hybrid of genetic algorithm and particle swarm optimization for distributed clustering in large-scale wireless sensor networks. J Ambient Intell Humaniz Comput. https://doi.org/10.1007/s12652-017-0619-9

    Article  Google Scholar 

  • Tang J, Alelyani S, Liu H (2014) Feature selection for classification: a review. In: Data classification: algorithms and applications, p 37

  • Tran B, Xue B, Zhang M (2014) Overview of particle swarm optimisation for feature selection in classification. In: Asia-Pacific conference on simulated evolution and learning. Springer, Cham, pp 605–617

  • Uçan F, Altilar DT (2012) Using genetic algorithms for navigation planning in dynamic environments. Appl Comput Intell Soft Comput. https://doi.org/10.1155/2012/560184

    Article  Google Scholar 

  • Uğuz H (2012) A hybrid system based on information gain and principal component analysis for the classification of transcranial Doppler signals. Comput Methods Programs Biomed 107(3):598–609

    Article  Google Scholar 

  • Wah YB, Ibrahim N, Hamid HA, Abdul-Rahman S, Fong S (2018) Feature selection methods: case of filter and wrapper approaches for maximising classification accuracy. Pertanika J Sci Technol 26(1):329–340

    Google Scholar 

  • Wu Y, Liu B, Wu W, Lin Y, Yang C, Wang M (2018) Grading glioma by radiomics with feature selection based on mutual information. J Ambient Intell Humaniz Comput 9(5):1671–1682

    Article  Google Scholar 

  • Xue B, Cervante L, Shang L, Browne WN, Zhang M (2013a) Multi-objective evolutionary algorithms for filter based feature selection in classification. Int J Artif Intell Tools 22(04):1350024

    Article  Google Scholar 

  • Xue B, Zhang M, Browne WN (2013b) Particle swarm optimization for feature selection in classification: a multi-objective approach. IEEE Trans Cybern 43(6):1656–1671

    Article  Google Scholar 

  • Yang CS, Chuang LY, Ke CH, Yang CH (2008a) Boolean binary particle swarm optimization for feature selection. In: Evolutionary computation, 2008. CEC 2008. (IEEE world congress on computational intelligence). IEEE congress. IEEE, pp 2093–2098

  • Yang CS, Chuang LY, Li JC, Yang CH (2008b) Chaotic maps in binary particle swarm optimization for feature selection. In: Soft computing in industrial applications, 2008. SMCia’08. IEEE conference. IEEE, pp 107–112

  • Ye Y, Wu Q, Huang JZ, Ng MK, Li X (2013) Stratified sampling for feature subspace selection in random forests for high dimensional data. Pattern Recogn 46(3):769–787

    Article  Google Scholar 

  • Yongjun P, Minghao P, Kiejung P, Keun HY (2012) An ensemble correlation-based gene selection algorithm for cancer classification with gene expression data. Bioinformatics 28(24):3306–3315

    Article  Google Scholar 

  • Yu L, Liu H (2004) Efficient feature selection via analysis of relevance and redundancy. J Mach Learn Res 5:1205–1224

    MathSciNet  MATH  Google Scholar 

  • Yusta SC (2009) Different metaheuristic strategies to solve the feature selection problem. Pattern Recognit Lett 30(5):525–534

    Article  Google Scholar 

  • Zhang DG (2012) A new approach and system for attentive mobile learning based on seamless migration. Appl Intell 36(1):75–89

    Article  Google Scholar 

  • Zhang DG, Zhang XD (2012) Design and implementation of embedded un-interruptible power supply system (EUPSS) for web-based mobile application. Enterp Inf Syst 6(4):473–489

    Article  Google Scholar 

  • Zhang DG, Zhu YN, Zhao CP, Dai WB (2012a) A new constructing approach for a weighted topology of wireless sensor networks based on local-world theory for the internet of things (IOT). Comput Math Appl 64(5):1044–1055. https://doi.org/10.1016/j.camwa.2012.03.023

    Article  MATH  Google Scholar 

  • Zhang D, Zhao CP, Liang YP, Liu ZJ (2012b) A new medium access control protocol based on perceived data reliability and spatial correlation in wireless sensor network. Comput Electr Eng 38(3):694–702. https://doi.org/10.1016/j.compeleceng.2012.02.016

    Article  Google Scholar 

  • Zhang D, Li G, Zheng K, Ming X, Pan ZH (2013) An energy-balanced routing method based on forward-aware factor for wireless sensor networks. IEEE Trans Ind Inf 10(1):766–773

    Article  Google Scholar 

  • Zhang D, Wang X, Song X, Zhao D (2014) A novel approach to mapped correlation of ID for RFID anti-collision. IEEE Trans Serv Comput 7(4):741–748

    Article  Google Scholar 

  • Zhang DG, Wang X, Song XD (2015a) New medical image fusion approach with coding based on SCD in wireless sensor network. J Electr Eng Technol 10(6):2384–2392. https://doi.org/10.5370/JEET.2015.10.6.2384

    Article  Google Scholar 

  • Zhang DG, Zheng K, Zhang T, Wang X (2015b) A novel multicast routing method with minimum transmission for WSN of cloud computing service. Soft Comput 19(7):1817–1827

    Article  Google Scholar 

  • Zhang DG, Zheng K, Zhao DX, Song XD, Wang X (2016) Novel quick start (QS) method for optimization of TCP. Wirel Netw 22(1):211–222

    Article  Google Scholar 

  • Zhang DG, Liu S, Zhang T, Liang Z (2017a) Novel unequal clustering routing protocol considering energy balancing based on network partition & distance for mobile education. J Netw Comput Appl 88:1–9

    Article  Google Scholar 

  • Zhang DG, Niu HL, Liu S (2017b) Novel PEECR-based clustering routing approach. Soft Comput 21(24):7313–7323

    Article  Google Scholar 

  • Zhang DG, Zhou S, Tang YM (2018a) A low duty cycle efficient MAC protocol based on self-adaption and predictive strategy. Mob Netw Appl 23(4):828–839. https://doi.org/10.1007/s11036-017-0878-x

    Article  Google Scholar 

  • Zhang D, Ge H, Zhang T, Cui YY, Liu X, Mao G (2018b) New multi-hop clustering algorithm for vehicular ad hoc networks. IEEE Trans Intell Transport Syst 99:1–14

    Article  Google Scholar 

  • Zhang DG, Zhang T, Zhang J, Dong Y, Zhang XD (2018c) A kind of effective data aggregating method based on compressive sensing for wireless sensor network. EURASIP J Wirel Commun Netw 2018(1):159. https://doi.org/10.1007/s11036-018-1123-y

    Article  Google Scholar 

  • Zhang DG, Chen C, Cui YY, Zhang T (2018d) New method of energy efficient subcarrier allocation based on evolutionary game theory. Mob Netw Appl. https://doi.org/10.1007/s11036-018-1123-y

    Article  Google Scholar 

  • Zhang DG, Tang YM, Cui YY, Gao JX, Liu XH, Zhang T (2018e) Novel reliable routing method for engineering of internet of vehicles based on graph theory. Eng Comput 36(1):226–247

    Article  Google Scholar 

  • Zhang D, Zhang T, Liu X (2018f) Novel self-adaptive routing service algorithm for application in VANET. Appl Intell 49(5):1866–1879. https://doi.org/10.1007/s10489-018-1368-y

    Article  Google Scholar 

  • Zhang DG, Zhang T, Dong Y, Liu XH, Cui YY, Zhao DX (2018g) Novel optimized link state routing protocol based on quantum genetic strategy for mobile learning. J Netw Comput Appl 122:37–49

    Article  Google Scholar 

  • Zhang DG, Liu S, Liu XH, Zhang T, Cui YY (2018h) Novel dynamic source routing protocol (DSR) based on genetic algorithm-bacterial foraging optimization (GA-BFO). Int J Commun Syst 31(18):e3824

    Article  Google Scholar 

  • Zhang DG, Gao JX, Liu XH, Zhang T, Zhao DX (2019) Novel approach of distributed & adaptive trust metrics for MANET. Wirel Netw 25(16):3587–3603. https://doi.org/10.1007/s11276-019-01955-2

    Article  Google Scholar 

  • Zhao Z, Morstatter F, Sharma S, Alelyani S, Anand A, Liu H (2010) Advancing feature selection research. In: ASU feature selection repository 1–28

  • Zorarpacı E, Özel SA (2016) A hybrid approach of differential evolution and artificial bee colony for feature selection. Expert Syst Appl 62:91–103

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Industrial Engineering, Iran University of Science and Technology, Tehran, Iran

    Fateme Moslehi & Abdorrahman Haeri

Authors
  1. Fateme Moslehi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Abdorrahman Haeri
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Abdorrahman Haeri.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Moslehi, F., Haeri, A. A novel hybrid wrapper–filter approach based on genetic algorithm, particle swarm optimization for feature subset selection. J Ambient Intell Human Comput 11, 1105–1127 (2020). https://doi.org/10.1007/s12652-019-01364-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-019-01364-5

Keywords

Search

Navigation