Skip to main content

Advertisement

SpringerLink
Log in

Utilizing the advantages of both global and local search strategies for finding a small subset of features in a two-stage method

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Feature selection (FS) is one of the pre-processing methods that are widely used in the fields of Data Mining and Pattern Recognition. Elimination of redundant/irrelevant features of large data sets and finding a suitable feature subset are one of the main goals in FS. The utilization of evolutionary algorithms, including global search algorithms e.g. Genetic Algorithm and local search algorithms e.g. hill climbing, is known as the best way to solve a variety of optimization problems such as FS problem. They are never able to find a globally optimal solution because they are often trapped in one of the local optimum solutions and stop. Therefore, the researchers have tried to solve this major problem by escaping from the local solutions. In this article, we propose a two-stage method by applying a global search algorithm and a local search algorithm to find a sub-optimal solution for the FS problem. Here, we define a sub-optimal solution as a solution with the high reduction rate and the similar or even better classification performance. In the suggested two-stage method referred to as BGSA-SA, that is, the binary version of the Gravitational Search Algorithm (BGSA) and Simulated Annealing (SA) are selected as global and local search algorithms, respectively. For evaluating this proposed two-stage method, we utilized several UCI machine learning datasets and both classifiers SVM and K-NN. We compare the accuracy and reduction rate of the proposed two-stage method with three groups of methods, such as: (1) six singular meta-heuristic methods including BGA, BPSO, GSAPSO, CHGSA, BGSA, and SA, (2) the other two-stage methods namely BGA-SA and BPSO-SA, and (3) seven published methods as the state-of-art methods. The obtained results confirm that our BGSA-SA method has the rank 1 in the reduction rate whereas the accuracy of it using both SVM and K-NN classifiers is similar or even, in some cases, better than the other mentioned methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Ashkezari AD, Ma H, Saha TK, Cui Y (2014) Investigation of feature selection techniques for improving efficiency of power transformer condition assessment. IEEE Trans Dielectr Electr Insul 21:836–844

    Article  Google Scholar 

  2. Asuncion A, Newman D (2007) UCI machine learning repository. www.archive.ics.uci.edu/ml

  3. Bajcsy P (2006) An overview of DNA microarray grid alignment and foreground separation approaches. EURASIP J Appl Signal Process 2006:1–13

    Google Scholar 

  4. Barani F, Mirhosseini M, Nezamabadi-pour H (2017) Application of binary quantum-inspired gravitational search algorithm in feature subset selection. Appl Intell 47:304–318

    Article  Google Scholar 

  5. Biricik G, Diri B, Sonmez AC (2012) Abstract feature extraction for text classification. Turk J Electr Eng Comput Sci 20:1137–1159

    Google Scholar 

  6. Chen H, Li S, Tang Z (2011) Hybrid gravitational search algorithm with random-key encoding scheme combined with simulated annealing. IJCSNS Int J Comput Sci Netw Secur 11:208–217

    Google Scholar 

  7. Chengyi T (2014) Gravitational search algorithm based on simulated annealing. J Converg Inf Technol (JCIT) 9:231–237

    Google Scholar 

  8. Cover T, Hart P (1967) Nearest neighbor pattern classification. IEEE Trans Inf Theory 13:21–27

    Article  MATH  Google Scholar 

  9. Demsar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30

    MathSciNet  MATH  Google Scholar 

  10. Fix E, Hodges J (1989) Discriminatory analysis-nonparametric discrimination: consistency properties. Int Stat Rev 57:238–247

    Article  MATH  Google Scholar 

  11. Han X, Chang X (2012) A chaotic digital secure communication based on a modified gravitational search algorithm filter. Inf Sci 208:14–27

    Article  Google Scholar 

  12. Han J, Kamber M (2006) Data Mining: concepts and techniques, 2nd edn. University of Illinois at Urbana-Champaign, printed on Elsevier Inc

  13. Han M, Ren W (2015) Global mutual information-based feature selection approach using single-objective and multi-objective optimization. Neurocomputing 168:47–54

    Article  Google Scholar 

  14. Khadanga R K, Satapathy J K (2015) A new hybrid GA–GSA algorithm for tuning damping controller parameters for a unified power flow controller. Electr Power Energy Syst 73:1060–1069

    Article  Google Scholar 

  15. Kirkpatrick S, Gellat CD, Vechi MP (1983) Optimization by simulated annealing. Pattern Recogn 220:671–680

    MathSciNet  Google Scholar 

  16. Kumar V, Kumar-Chahabra J, Kumar D (2015) Automatic unsupervised feature selection using gravitational search algorithm. IETE J Res 61:22–31

    Article  Google Scholar 

  17. Li C, Zhou J (2011) Parameters identification of hydraulic turbine governing system using improved gravitational search algorithm. Energy Convers Manag 52:374–381

    Article  Google Scholar 

  18. Li C, An X, Li R (2015) A chaos embedded GSA-SVM hybrid system for classification. Neural Comput Appl 26:713–721

  19. Mirjalili S, Hashim S Z M (2010) A new hybrid PSOGSA algorithm for function optimization. In: Proceedings of 2010 international conference on computer and information application (ICCIA 2010). Tianjin, pp 374–377

  20. Nezamabadi-pour H (2015) A quantum-inspired gravitational search algorithm for binary encoded optimization problems. Eng Appl Artif Intell 40:62–75

    Article  Google Scholar 

  21. Papa JP, Pagnin A, Schellini SA, Spadotto A, Guido RC, Chiachia G, Falcao AX (2011) Feature selection through gravitational search algorithm. In: IEEE ICASSP, pp 2052–2055

  22. Pedrajas N G, Rodriguez JP (2012) Multi-selection of instances: a straightforward way to improve evolutionary instance selection. Appl Soft Comput 12:3590–3602

    Article  Google Scholar 

  23. Rashedi E, Nezamabadi-pour H, Saryazd S (2009) GSA: a gravitational search algorithm. Inf Sci 179:2232–2248

    Article  MATH  Google Scholar 

  24. Rashedi E, Nezamabadi-pour H, Saryazdi S (2010) BGSA: binary gravitational search algorithm. Nat Comput 9:727–745

    Article  MathSciNet  MATH  Google Scholar 

  25. Rashedi E, Nezamabadi-pour H, Saryazd S (2013) A simultaneous feature adaptation and feature selection method for content-based image retrieval systems. Knowl-Based Syst 39:85–94

    Article  Google Scholar 

  26. Sarafrazi S, Nezamabadi-pour H, Saryazdi S (2011) Disruption, A new operator in gravitational search algorithm. Sci Iran 18:539–548

    Article  Google Scholar 

  27. Shuang C, Hongnian Y (2012) Mutual information based input feature selection for classification problems. Decis Support Syst 54:691–698

    Article  Google Scholar 

  28. Sikora R, Piramuthu S (2007) Framework for efficient feature selection in genetic algorithm based data mining. Eur J Oper Res 180:723–737

    Article  MATH  Google Scholar 

  29. Tsai CF, Eberle W, Chu CY (2013) Genetic algorithms in feature and instance selection. Knowl-Based Syst 39:240–247

    Article  Google Scholar 

  30. Vapnik VN (1995) The nature of statistical learning theory. Springer, New York

    Book  MATH  Google Scholar 

  31. Vierira SM, Mendonca LF, Farinha GJ, Sousa JMC (2013) Modified binary PSO for feature selection using SVM applied to mortality prediction of septic patients. Appl Soft Comput 13:3494–3504

    Article  Google Scholar 

  32. Wahde M (2008) Biologically inspired optimization methods, 1st edn. WIT Press, Southampton

    MATH  Google Scholar 

  33. Wang CM, Huang YF (2009) Evolutionary-based feature selection approaches with new criteria for data mining: a case study of credit approval data. Expert Syst Appl 36:5900–5908

    Article  Google Scholar 

  34. Wilcoxon F (1945) Individual comparisons by ranking methods. Biometrics 1:80–83

    Article  MathSciNet  Google Scholar 

  35. Xiang J, Han XH, Duan F, Qiang Y, Xiong XY, Lan Y, Chai H (2015) A novel hybrid system for feature selection based on an improved gravitational search algorithm and k-NN method. Appl Soft Comput 31:293–307

    Article  Google Scholar 

  36. Xue B, Zhang M, Browne WN (2012) New fitness functions in binary particle swarm optimization for feature selection. In: IEEE world congress on computational intelligence, pp 10–15

  37. Xue B, Zhang M, Browne WN (2014) Particle swarm optimization for feature selection in classification: novel initialization and updating mechanisms. Appl Soft Comput 18:261–276

    Article  Google Scholar 

  38. Zahiri SH (2012) Fuzzy gravitational search algorithm an approach for data mining. Iran J Fuzzy Syst 9:21–37

    MathSciNet  MATH  Google Scholar 

  39. Zhang H, Qin C, Jiang B, Luo Y (2014) On line adaptive policy learning algorithm for H1 state feedback control of unknown affine on linear discrete-time systems. IEEE Trans Cybern 44:2706–2718

    Article  Google Scholar 

  40. Zhang H, Zhang J, Yang GH, Luo Y (2015) Leader-based optimal coordination control for the consensus problem of multi-agent differential games via fuzzy adaptive dynamic programming. IEEE Trans Fuzzy Syst 23:152–163

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Faculty of Mathematics and Computer, Shahid Bahonar University of Kerman, Kerman, 76169-14111, Iran

    Mohammad Masoud Javidi & Fatemeh Zarisfi Kermani

Authors
  1. Mohammad Masoud Javidi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fatemeh Zarisfi Kermani
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohammad Masoud Javidi.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Javidi, M.M., Zarisfi Kermani, F. Utilizing the advantages of both global and local search strategies for finding a small subset of features in a two-stage method. Appl Intell 48, 3502–3522 (2018). https://doi.org/10.1007/s10489-018-1159-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-018-1159-5

Keywords

Search

Navigation