Skip to main content
SpringerLink
Log in

Hybrid Harmony Search Combined with Stochastic Local Search for Feature Selection

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Feature selection is a challenging task that has been the subject of a large amount of research, especially in relation to classification tasks. It permits to eliminate the redundant attributes and enhance the classification accuracy by keeping only the relevant attributes. In this paper, we propose a hybrid search method based on both harmony search algorithm (HSA) and stochastic local search (SLS) for feature selection in data classification. A novel probabilistic selection strategy is used in HSA–SLS to select the appropriate solutions to undergo stochastic local refinement, keeping a good compromise between exploration and exploitation. In addition, the HSA–SLS is combined with a support vector machine (SVM) classifier with optimized parameters. The proposed HSA–SLS method tries to find a subset of features that maximizes the classification accuracy rate of SVM. Experimental results show good performance in favor of our proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Alia OM, Mandava R, Ramachandram D, Aziz ME (2009) Dynamic fuzzy clustering using harmony search with application to image segmentation. In: IEEE international symposium on signal processing and information technology (ISSPIT), pp 538–543

  2. Awadallah MA, Khader AT, Azmi Al-Betar M, Bolaji AL (2013) Global best Harmony Search with a new pitch adjustment designed for Nurse Rostering. J King Saud Univ Comput Inf Sci 25(2):145–162

    Google Scholar 

  3. Bao Y, Hu Z, Xiong T (2013) A PSO and pattern search based memetic algorithm for SVMs parameters optimization. Neurocomputing 117:98–106

    Article  Google Scholar 

  4. Bermejo P, Gomez JA, Puerta JM (2011) A GRASP algorithm for fast hybrid (filter-wrapper) feature subset selection in high-dimensional datasets. Pattern Recogn Lett 32(5):701–711

    Article  Google Scholar 

  5. Bonilla Huerta EB, Duval B, Hao JK (2006) A hybrid GA/SVM approach for gene selection and classification of microarray data. In: Rothlanf F et al (eds) EvoWorkshops 2006, LNCS, vol 3907, pp 34–44

  6. Boughaci D, Benhamou B, Drias H (2010) Local Search Methods for the optimal winner determination problem. J Math Model Algorithms (Springer) 9(2):165–180

    Article  MathSciNet  Google Scholar 

  7. Breiman L, Friedman J, Olshen R, Stone C (1984) Classification and regression trees. Wadsworth, Belmont

    MATH  Google Scholar 

  8. Campbell C, Ying Y (2011) Learning with support vector machines. Morgan and Claypool, San Rafael

    MATH  Google Scholar 

  9. Chang CC, Lin CJ (2012) LIBSVM: a library for support vector machines. ACM Trans Intell Syst Technol. http://www.csie.ntu.edu.tw/~cjlin/libsvm/oldfiles/index-1.0.html. Accessed 29 March 2014

  10. Dia R, Shen Q (2012) Feature selection with harmony search. IEEE Trans Syst Man Cybern B 42(6):1509–1523

    Article  Google Scholar 

  11. Frank E, Witten IH (1998) Generating accurate rule sets without global optimization. In: Shavlik J (ed) Proceedings of the fifteenth international conference machine learning (ICML 98), pp 144–151

  12. Friedman N, Geiger D, Goldszmidt M (1997) Bayesian network classifiers. Mach Learn 29:131–163

    Article  MATH  Google Scholar 

  13. Gao XZ, Wang X, Zenger K (2013) A memetic-inspired harmony search method in optimal wind generator design. In: International journal of machine learning and cybernetics. Springer, Berlin

  14. Geem ZW (2007) Harmony search algorithm for solving Sudoku. Knowl Based Intell Inf Eng Syst 4692:371–378

    Google Scholar 

  15. Geem ZW (2009) Harmony search algorithms for structural design optimization. Springer, New York

    Book  Google Scholar 

  16. Geem ZW, Choi JY (2007) Music composition using harmony search algorithm. In: Applications of evolutionary computing. Springer, Berlin, pp 593–600

  17. Geem ZW, Kim JH, Loganathan G (2001) A new heuristic optimization algorithm: harmony search. Simulation 76:60–68

    Article  Google Scholar 

  18. Hadwan M, Ayob M, Sabar NR, Qu R (2013) A harmony search algorithm for nurse rostering problems. Inf Sci 233:126–140

    Article  MathSciNet  Google Scholar 

  19. Hamel L (2009) Knowledge discovery with support vector machines. Wiley, Canada

  20. Han J, Kamber M (2006) Data mining concepts and techniques, 2nd edn. Morgan Kaufmann, San Francisco

    MATH  Google Scholar 

  21. Hannah IH, Bagyamathi M, Azar TA (2015) A novel hybrid feature selection method based on rough set and improved harmony search. Neural Comput Appl. doi:10.1007/s00521-015-1840-0

  22. Hertz JA, Krogh A, Palmer RG (1991) Introduction to the theory of neural computation. Addison-Wesley Publishing Company, Inc., Redwood City

    Google Scholar 

  23. Hsu CW, Chang CC, Lin CJ (2003) A practical guide to support vector classification. http://www.csie.ntu.edu.tw/~cjlin/papers/guide/guide.pdf. Accessed 29 March 2014

  24. John GH, Langley P (1995) Estimating continuous distributions in Bayesian classifiers. In: Proceedings of the eleventh conference on uncertainty in artificial intelligence. Morgan Kaufmann, San Mateo, pp 338–345

  25. Kecman V (2001) Learning and soft computing: support vector machines, neural networks, and fuzzy logic models. The MIT Press, London

    MATH  Google Scholar 

  26. Kohavi R, John G (1996) Wrappers for feature subset selection. Artif intell 97(1–2):273–324 (special issue on relevance)

  27. Krishnaveni V, Arumugam G (2013) Harmony search based wrapper feature selection method for 1-nearest neighbour classifier. In: Proceedings of the international conference on pattern recognition, informatics and mobile engineering (PRIME)

  28. Kumar V, Chhabra JK, Kumar D (2014) Parameter adaptive harmony search for unimodal and multimodal optimization problems. J Comput Sci 5(2):144–155. doi:10.1016/j.jocs.2013.12.001

    Article  MathSciNet  Google Scholar 

  29. Lee KS, Geem ZW (2005) A new meta-heuristic algorithm for continuous engineering optimization: harmony search theory and practice. Comput Methods Appl Mech Eng 194:3902–3922

    Article  MATH  Google Scholar 

  30. Lessmann S, Stahlbock R, Crone SF (2006) Genetic algorithms for support vector machine model selection. In: Proceedings of the international joint conference on neural networks, IJCNN 2006, part of the IEEE world congress on computational intelligence, WCCI 2006. IEEE, Vancouver, pp 3063–3069

  31. Li Y, Tong Y, Bai B, Zhang Y (2007) An improved particle swarm optimization for SVM training. In: Third international conference on natural computation (ICNC 2007), pp 611–615

  32. Mahdavi M, Fesanghary M, Damangir E (2007) An improved harmony search algorithm for solving optimization problems. Appl Math Comput 188(2):1567–1579

    MathSciNet  MATH  Google Scholar 

  33. Nekkaa M, Boughaci D (2012) Improving support vector machine using a stochastic local search for classification in data mining. In: Proceedings of ICONIP 2012, Part II, LNCS proceedings, vol 7664, pp 168–176

  34. Nekkaa M, Boughaci D (2014) Stochastic local search versus genetic algorithm for feature selection. In: Proceedings of APMOD CONFERENCE 2014: international conference on applied mathematical optimization and modelling 2014

  35. Nekkaa M, Boughaci D (2015) A memetic algorithm with support vector machine for feature selection and classification. Memetic Comput 7:59–73. doi:10.1007/s12293-015-0153-2

    Article  Google Scholar 

  36. Nekooei K, Farsangi MM, Nezamabadi-Pour H, Lee KY (2013) An improved multi-objective harmony search for optimal placement of DGs in distribution systems. IEEE Trans Smart Grid 4(1):557–567

    Article  Google Scholar 

  37. Panchal A (2009) Harmony search in therapeutic medical physics. In: Geem ZW (ed) Music-inspired harmony search algorithm. Springer, Hiedelberg, pp 189–203

  38. Quinlan JR (1992) C4.5: programs for machine learning. Morgan Kaufmann, San Mateo

    Google Scholar 

  39. Rao R, Savsani V, Vakharia D (2012) Teaching learning-based optimization: an optimization method for continuous non-linear large scale problems. Inf Sci 183:1–15

    Article  MathSciNet  Google Scholar 

  40. Tan KC, Teoh EJ, Yua Q, Goh KC (2009) A hybrid evolutionary algorithm for attribute selection in data mining. Expert Syst Appl 36:8616–8630

    Article  Google Scholar 

  41. Tay FEH, Cao LJ (2001) Application of support vector machines in financial time series forecasting. Omega 29(4):309–317

    Article  Google Scholar 

  42. Vapnik V (1998) Statistical learning theory. Wiley, New York

    MATH  Google Scholar 

  43. Vapnik V (1995) The Natural of Statistical Learning theory. Springer, New York

    Book  MATH  Google Scholar 

  44. Waikato environment for knowledge analysis (WEKA), version 3.6. The University of Waikato, Hamilton. http://www.cs.waikato.ac.nz/ml/weka/downloading.html. Accessed 29 March 2014

  45. Yadav P, Kumar R, Panda S, Chang C (2012) An intelligent tuned harmony search algorithm for optimisation. Inf Sci 196:47–72

    Article  Google Scholar 

  46. Yildiz AR (2012) A comparative study of population-based optimization algorithms for turning operations. Inf Sci 210:81–88

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgments

The authors would like to thank the anonymous reviewers for their valuable comments and suggestions to improve the quality of the paper. They would like also to thank the developers of Waikato Environment for Knowledge Analysis (WEKA) and the Library for Support Vector Machines (LIBSVM) for the provision of the open source code.

Author information

Authors and Affiliations

  1. LIMOSE Laboratory, Computer Science Department, Faculty of Sciences, University M’Hamed Bougara of Boumerdès, Independency Avenue, 35000, Boumerdés, Algeria

    Messaouda Nekkaa

  2. LRIA/Computer Sciences Department, University of Sciences and Technology Houari Boumediene (USTHB), BP 32 El-Alia, Beb-Ezzoaur, 16111, Algiers, Algeria

    Dalila Boughaci

Authors
  1. Messaouda Nekkaa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dalila Boughaci
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Messaouda Nekkaa.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nekkaa, M., Boughaci, D. Hybrid Harmony Search Combined with Stochastic Local Search for Feature Selection. Neural Process Lett 44, 199–220 (2016). https://doi.org/10.1007/s11063-015-9450-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-015-9450-5

Keywords

Search

Navigation