Skip to main content

Advertisement

SpringerLink
Log in

A bumble bees mating optimization algorithm for the feature selection problem

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

The feature selection problem is an interesting and important topic which is relevant for a variety of database applications. This paper utilizes a relatively new bees inspired optimization algorithm, the bumble bees mating optimization algorithm, to implement a feature subset selection procedure while the nearest neighbor classification method is used for the classification task. Several metrics are used in the nearest neighbor classification method, such as the euclidean distance, the standardized euclidean distance, the mahalanobis distance, the city block metric, the cosine distance and the correlation distance, in order to identify the most significant metric for the nearest neighbor classifier. The performance of the proposed algorithm is tested using various benchmark data sets from the UCI machine learning repository. The algorithm is compared with two other bees inspired algorithms, the one is based on the foraging behavior of the bees, the discrete artificial bee colony, and the other is based on the mating behavior of the bees, the honey bees mating optimization algorithm. The algorithm is, also, compared with a particle swarm optimization algorithm, an ant colony optimization algorithm, a genetic algorithm and with a number of algorithms from the literature.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Marinakis Y, Marinaki M, Matsatsinis N (2009a) A hybrid bumble bees mating optimization—GRASP algorithm for clustering. In: Corchado E (eds) HAIS 2009, LN 5572. Springer, Berlin, pp 549–556

  2. Marinakis Y, Marinaki M, Matsatsinis N (2010b) A Bumble bees mating optimization algorithm for global unconstrained optimization problems. In: Gonzalez JR (eds) Nature inspired cooperative strategies for optimization (NICSO’10), studies in computational intelligence, vol 284. Springer, Berlin, pp 305–318

  3. Marinakis Y, Marinaki M (2011) Bumble bees mating optimization algorithm for the vehicle routing problem. In: Panigrahi BK, Shi Y, Lim M-H (eds) Handbook of swarm intelligence—concepts, principles and applications, series on adaptation, learning, and optimization, vol 8. Springer, Berlin, pp 347–369

  4. Marinakis Y, Marinaki M (2014) A bumble bees mating optimization algorithm for the open vehicle routing problem. Swarm Evol Comput 15:80–94

    Article  MATH  Google Scholar 

  5. Feo TA, Resende MGC (1995) Greedy randomized adaptive search procedure. J Glob Optim 6:109–133

    Article  MathSciNet  MATH  Google Scholar 

  6. Abbass HA (2001a) A monogenous MBO approach to satisfiability. In: Proceedings of the international conference on computational intelligence for modelling, control and automation (CIMCA’01), Las Vegas

  7. Abbass HA (2001b) Marriage in honey-bee optimization (MBO): a haplometrosis polygynous swarming approach. In: Proceedings of the congress on evolutionary computation (CEC2001), Seoul, pp 207–214

  8. Karaboga D, Basturk B (2008) On the performance of artificial bee colony (ABC) algorithm. Appl Soft Comput 8:687–697

    Article  Google Scholar 

  9. Yang XS (2005) Engineering optimizations via nature-inspired virtual bee algorithms. In: Yang JM, Alvarez JR (eds) IWINAC 2005, LNCS 3562. Springer, Berlin, pp 317–323

  10. Teodorovic D, Dell’Orco M (2005) Bee colony optimization—a cooperative learning approach to complex transportation problems. In: Advanced OR and AI methods in transportation, pp 51–60

  11. Wedde HF, Farooq M, Zhang Y (2004) BeeHive: an efficient fault-tolerant routing algorithm inspired by honey bee behavior. In: Dorigo M (ed) Ant colony optimization and swarm intelligence, LNCS 3172, Springer, Berlin, pp 83–94

  12. Drias H, Sadeg S, Yahi S (2005) Cooperative bees swarm for solving the maximum weighted satisfiability problem. In: IWAAN international work conference on artificial and natural neural networks, LNCS 3512, pp 318–325

  13. Pham DT, Kog E, Ghanbarzadeh A, Otri S, Rahim S, Zaidi M (2006) The bees algorithm—a novel tool for complex optimization problems. In: Proceedings of 2nd international virtual conference on intelligent production machines and systems (IPROMS’06). Elsevier, Oxford

  14. Baykasoglu A, Ozbakor L, Tapkan P (2007) Artificial bee colony algorithm and its application to generalized assignment problem. In: Chan FTS, Tiwari MK (eds) Swarm intelligence, focus on ant and particle swarm optimization. I-Tech Education and Publishing, pp 113–144

  15. Karaboga D, Akay B (2009) A survey: algorithms simulating bee swarm intelligence. Artif Intell Rev 31:61–85

    Article  Google Scholar 

  16. Xing B, Gao W-J (2014) Bee inspired algorithms (chapter four). Innovative computational intelligence: a rough guide to 134 clever algorithms, intelligent systems reference. Library 62, Springer International Publishing, Switzerland, pp 45-80. doi:10.1007/978-3-319-03404-1

  17. Duda RO, Hart PE (1973) Pattern classification and scene analysis. Wiley, New York

    MATH  Google Scholar 

  18. Duda RO, Hart PE (2001) Pattern classification and scene analysis, 2nd edn. Wiley, New York

    MATH  Google Scholar 

  19. Wei P, Ma P, Hu Q (2014) Comparative analysis on margin based feature selection algorithms. Int J Mach Learn Cybern 5(3):339–367

    Article  Google Scholar 

  20. Gan JQ, Awwad Shiekh Hasan B, Tsui CSL (2014) A filter-dominating hybrid sequential forward floating search method for feature subset selection in high-dimensional space. Int J Mach Learn Cybern 5(3):413–423

    Article  Google Scholar 

  21. Maldonado S, Weber R (2009) A wrapper method for feature selection using support vector machines. Inf Sci 179(13):2208–2217

    Article  Google Scholar 

  22. Uncu O, Turksen IB (2007) A novel feature selection approach: combining feature wrappers and filters. Inf Sci 177(2):449–466

  23. Cotta C, Sloper C (2004) Evolutionary search of thresholds for robust feature set selection: application to the analysis of microarray data. Lect Notes Comput Sci 3005:21–30

    Article  Google Scholar 

  24. Kohavi R, John G (1997) Wrappers for feature subset selection. Artif Intell 97:273–324

    Article  MATH  Google Scholar 

  25. Jain A, Zongker D (1997) Feature selection: evaluation, application, and small sample performance. IEEE Trans Pattern Anal Mach Intell 19:153–158

    Article  Google Scholar 

  26. Narendra PM, Fukunaga K (1977) A branch and bound algorithm for feature subset selection. IEEE Trans Comput 26(9):917–922

    Article  MATH  Google Scholar 

  27. Aha DW, Bankert RL (1996) A comparative evaluation of sequential feature selection algorithms. In: Fisher D, Lenx J-H (eds) Artificial intelligence and statistics. Springer, New York

    Google Scholar 

  28. Cantu-Paz E, Newsam S, Kamath C (2004) Feature selection in scientific application. In: Proceedings of the 2004 ACM SIGKDD international conference on knowledge discovery and data mining, pp 788–793

  29. Pudil P, Novovicova J (1994) Floating search methods in feature selection. Pattern Recognit Lett 15:1119–1125

    Article  Google Scholar 

  30. Cantu-Paz E (2004) Feature subset selection, class separability, and genetic algorithms. In: Proceedings of the genetic and evolutionary computation conference, pp 959–970

  31. Fleuret F (2004) Fast binary feature selection with conditional mutual information. J Mach Learn Res (JMLR) 5:1531–1555

    MathSciNet  MATH  Google Scholar 

  32. Yu L, Liu H (2004) Efficient feature selection via analysis of relevance and redundancy. J Mach Learn Res (JMLR) 5:1205–1224

    MathSciNet  MATH  Google Scholar 

  33. Kira K, Rendell L (1992) A practical approach to feature selection. In: Proceedings of the ninth international conference on machine learning, Aberdeen, pp 249–256

  34. Lin SW, Lee ZJ, Chen SC (2008a) Parameter determination of support vector machine and feature selection using simulated annealing approach. Appl Soft Comput 8:1505–1512

    Article  Google Scholar 

  35. Pourhabibi T, Imania MB (2011) Feature selection on Persian Fonts: A Comparative analysis on GAA, GESA and GA. Procedia Comput Sci 3:1249–1255

    Article  Google Scholar 

  36. Siedlecki W, Sklansky J (1988) On automatic feature selection. Int J Pattern Recognit Artif Intell 2(2):197–220

    Article  MATH  Google Scholar 

  37. Chen SC, Lin SW (2011) Enhancing the classification accuracy by scatter-search-based ensemble approach. Appl Soft Comput 11:1021–1028

    Article  Google Scholar 

  38. Garcia Lopez F, Garcia Torres M, Melian Batista B, Moreno Perez JA (2006) Solving feature subset selection problem by a parallel scatter search. Eur J Oper Res 169:477–489

    Article  MathSciNet  MATH  Google Scholar 

  39. Al-Ani A (2005a) Feature subset selection using ant colony optimization. Int J Comput Intell 2(1):53–58

    Google Scholar 

  40. Al-Ani A (2005b) Ant colony optimization for feature subset selection. Trans Eng Comput Technol 4:35–38

    Google Scholar 

  41. Chen Y, Miao D (2010) A rough set approach to feature selection based on ant colony optimization. Pattern Recognit Lett 31:226–233

    Article  Google Scholar 

  42. Huang CL (2009) ACO-based hybrid classification system with feature subset selection and model parameters optimization. Neurocomputing 73:438–448

    Article  Google Scholar 

  43. Parpinelli RS, Lopes HS, Freitas AA (2002) An ant colony algorithm for classification rule discovery. In: Abbas H, Sarker R, Newton C (eds) Data mining: a heuristic approach. Idea group publishing, London, pp 191–208

  44. Shelokar PS, Jayaraman VK (2004) An ant colony classifier system: application to some process engineering problems. Comput Chem Eng 28:1577–1584

    Article  Google Scholar 

  45. Tabakhi S, Moradi P (2014) An unsupervised feature selection algorithm based on ant colony optimization. Eng Appl Artif Intell 32:112–123

    Article  Google Scholar 

  46. Zhang C, Hu H (2005) Ant colony optimization combining with mutual information for feature selection in support vector machines. In: Zhang S, Jarvis R (eds) AI 2005, LNAI 380, pp 918–921

  47. Casado Yusta S (2009) Different metaheuristic strategies to solve the feature selection problem. Pattern Recognit Lett 30:525–534

    Article  Google Scholar 

  48. Boubezoul A, Paris S (2012) Application of global optimization methods to model and feature selection. Pattern Recognit 45:3676–3686

    Article  MATH  Google Scholar 

  49. Chuang LY, Yang CH (2011) Chaotic maps based on binary particle swarm optimization for feature selection. Appl Soft Comput 11:239–248

    Article  Google Scholar 

  50. Hannah Inbarania H, Taher Azar A (2014) Supervised hybrid feature selection based on PSOand rough sets for medical diagnosis. Comput Methods Programs Biomed 113:175–185

    Article  Google Scholar 

  51. Lin SW, Chen SC (2009) PSOLDA: a particle swarm optimization approach for enhancing classification accurate rate of linear discriminant analysis. Appl Soft Comput 9:1008–1015

    Article  Google Scholar 

  52. Lin SW, Ying KC, Chen SC (2008b) Particle swarm optimization for parameter determination and feature selection of support vector machines. Expert Syst Appl 35:1817–1824

    Article  Google Scholar 

  53. Pedrycz W, Park BJ (2009) Identifying core sets of discriminatory features using particle swarm optimization. Expert Syst Appl 36:4610–4616

    Article  Google Scholar 

  54. Unler A, Murat A (2010) A discrete particle swarm optimization method for feature selection in binary classification problems. Eur J Oper Res 206:528–539

    Article  MATH  Google Scholar 

  55. Wang Y, Feng XY, Huang YX, Pu DB, Zhou WG, Liang YC, Zhou CG (2007) A novel quantum swarm evolutionary algorithm and its applications. Neurocomputing 70(4–6):633–640

    Article  Google Scholar 

  56. Wang X, Yang J, Teng X, Xia W (2007) Feature selection based on rough sets and particle swarm optimization. Pattern Recognit Lett 28:459–471

    Article  Google Scholar 

  57. Xue B, Zhang M (2014) Particle swarm optimisation for feature selection in classification: novel initialisation and updating mechanisms. Appl Soft Comput 18:261–276

    Article  Google Scholar 

  58. Na Tian N, Lai C-H (2014) Parallel quantum-behaved particle swarm optimization. Int J Mach Learn Cybern 5(2):309–318

    Article  Google Scholar 

  59. Wang X-Z, He Y-L, Dong L-C (2011) Particle swarm optimization for determining fuzzy measures from data. Inf Sci 181(19):4230–4252

    Article  MATH  Google Scholar 

  60. Garcia-Pedrajas N, de Haro-Garcia A (2013) A scalable approach to simultaneous evolutionary instance and feature selection. Inf Sci 228:150–174

    Article  MathSciNet  Google Scholar 

  61. Tallon-Ballesteros AJ, Hervas-Martinez C, Riquelme C, Ruiz R (2013) Feature selection to enhance a two-stage evolutionary algorithm in product unit neural networks for complex classification problems. Neurocomputing 114:107–117

    Article  Google Scholar 

  62. Casillas J, Cordon O, Del Jesus MJ (2001) Genetic feature selection in a fuzzy rule-based classification system learning process for high-dimensional problems. Inf Sci 136(1–4):135–157

    Article  MATH  Google Scholar 

  63. Carvalho DR, Freitas AA (2004) A hybrid decision tree/genetic algorithm method for data mining. Inf Sci 163(1–3):13–35

    Article  Google Scholar 

  64. De Stefano C, Fontanella F, Marrocco C (2014) A GA-based feature selection approach with an application to handwritten character recognition. Pattern Recognit Lett 35:130–141

    Article  Google Scholar 

  65. ElAlami ME (2009) A filter model for feature subset selection based on genetic algorithm. Knowl-Based Syst 22:356–362

    Article  Google Scholar 

  66. Hsu WH (2004) Genetic wrappers for feature selection in decision tree induction and variable ordering in Bayesian network structure learning. Inf Sci 163(1–3):103–122

    Article  MathSciNet  Google Scholar 

  67. Huang J, Cai Y (2007) A hybrid genetic algorithm for feature selection wrapper based on mutual information. Pattern Recognit Lett 28:1825–1844

    Article  MathSciNet  Google Scholar 

  68. Pendharkar PC (2013) A maximum-margin genetic algorithm for misclassification cost minimizing feature selection problem. Expert Syst Appl 40:3918–3925

    Article  Google Scholar 

  69. Rokach L (2008) Genetic algorithm-based feature set partitioning for classification problems. Pattern Recognit 41:1676–1700

    Article  MATH  Google Scholar 

  70. Siedlecki W, Sklansky J (1989) A note on genetic algorithms for large-scale feature selection. Pattern Recognit Lett 10:335–347

    Article  MATH  Google Scholar 

  71. Srinivasa KG, Venugopal KR (2007) A self-adaptive migration model genetic algorithm for data mining applications. Inf Sci 177(20):4295–4313

    Article  MATH  Google Scholar 

  72. Al-Ani A, Alsukker A (2013) Feature subset selection using differential evolution and a wheel based search strategy. Swarm Evol Comput 9:15–26

    Article  Google Scholar 

  73. Khushaba RN, Al-Ani A (2011) Feature subset selection using differential evolution and a statistical repair mechanism. Expert Syst Appl 38:11515–11526

    Article  Google Scholar 

  74. Rodrigues D, Pereira LAM, Nakamura RYM, Costa KAP, Yang XS, Souza AN (2014) A wrapper approach for feature selection based on Bat Algorithm and Optimum-Path Forest. Expert Syst Appl 41:2250–2258

    Article  Google Scholar 

  75. Yazdani S, Shanbehzadeh J (2013) Feature subset selection using constrained binary/integer biogeography-based optimization. ISA Trans 52:383–390

    Article  Google Scholar 

  76. Chandrashekar G, Sahin F (2014) A survey on feature selection methods. Comput Electr Eng 40:16–28

    Article  Google Scholar 

  77. Marinaki M, Marinakis Y (2010) Honey bees mating optimization algorithm for financial classification problems. Appl Soft Comput 10:806–812

    Article  MATH  Google Scholar 

  78. Marinakis Y, Marinaki M, Matsatsinis N (2009b) A hybrid discrete artificial bee colony—GRASP algorithm for clustering. In: Proceedings of the 39th international conference on computers and industrial engineering, Troyes

  79. Holland JH (1975) Adaptation in natural and artificial systems. University of Michigan Press, Ann Arbor

    Google Scholar 

  80. Dorigo M, Stutzle T (2004) Ant colony optimization. A Bradford book. The MIT Press, Cambridge

    MATH  Google Scholar 

  81. Kennedy J, Eberhart R (1995). Particle swarm optimization. In: Proceedings of the 1995 IEEE international conference on neural networks, vol 4, pp 1942–1948

  82. Ray S, Turner LF (1992) Mahalanobis distance-based two new feature evaluation criteria. Inf Sci 60(3):217–245

    Article  MATH  Google Scholar 

  83. Abbass HA (2001c) A single queen single worker honey bees approach to 3-SAT. In: Proceedings of the genetic and evolutionary computation conference (GECCO’01), San Francisco

  84. Abbass HA (2001d) An agent based approach to 3-SAT using marriage in honey-bees optimization. Int J Knowl-Based Intell Eng Syst (KES) 6(2):1–8

    Google Scholar 

  85. Afshar A, Bozog Haddad O (2007) Honey-bee mating optimization (HBMO) algorithm for optimal reservoir operation. J Frankl Inst 344:452–462

    Article  MATH  Google Scholar 

  86. Fathian M, Amiri B (2007) Application of honey bee mating optimization algorithm on clustering. Appl Math Comput 190:1502–1513

    MathSciNet  MATH  Google Scholar 

  87. Haddad OB, Afshar A (2006) Honey-bees mating optimization (HBMO) algorithm: a new heuristic approach for water resources optimization. Water Resour Manag 20:661–680

    Article  Google Scholar 

  88. Marinakis Y, Marinaki M (2009) A hybrid honey bees mating optimization algorithm for the probabilistic traveling salesman problem. In: Proceedings of the IEEE congress on evolutionary computation (CEC’09), Trondheim

  89. Marinakis Y, Marinaki M, Dounias G (2008a) Honey bees mating optimization algorithm for the vehicle routing problem. In: Krasnogor N, Nicosia G, Pavone M, Pelta D (eds) Nature inspired cooperative strategies for optimization (NICSO’07). Studies in computational intelligence. Springer, Berlin, pp 129, 139–148

  90. Marinakis Y, Marinaki M (2010a) Honey bees mating optimization algorithm for large scale vehicle routing problems. Nat Comput 9:5–27

    Article  MathSciNet  MATH  Google Scholar 

  91. Marinakis Y, Marinaki M, Matsatsinis N (2008b) A hybrid clustering algorithm based on honey bees mating optimization and greedy randomized adaptive search procedure, learning and intelligence optimization (LION’07). LNCS, vol 5313. Springer, Berlin, pp 138–152

  92. Marinakis Y, Marinaki M, Matsatsinis N (2008) Honey bees mating optimization for the location routing problem. In: Proceedings of the IEEE international engineering management conference (IEMC Europe 2008), Estoril

  93. Teo J, Abbass HA (2003) A true annealing approach to the marriage in honey bees optimization algorithm. Int J Comput Intell Appl 3(2):199–211

    Article  Google Scholar 

  94. Karaboga D, Basturk B (2007) A powerful and efficient algorithm for numerical function optimization: artificial bee colony (ABC) algorithm. J Glob Optim 39:459–471

    Article  MathSciNet  MATH  Google Scholar 

  95. Marinakis Y, Marinaki M, Doumpos M, Matsatsinis N (2008d) Optimization of nearest neighbor classifiers via metaheuristic algorithms for credit risk assessment. J Glob Optim 42:279–293

    Article  MathSciNet  MATH  Google Scholar 

  96. Marinakis Y, Marinaki M, Doumpos M (2009c) Ant colony and particle swarm optimization for financial classification problems. Expert Syst Appl 36(7):10604–10611

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Production Engineering and Management, Technical University of Crete, 73100 , Chania, Greece

    Magdalene Marinaki & Yannis Marinakis

Authors
  1. Magdalene Marinaki
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yannis Marinakis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yannis Marinakis.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Marinaki, M., Marinakis, Y. A bumble bees mating optimization algorithm for the feature selection problem. Int. J. Mach. Learn. & Cyber. 7, 519–538 (2016). https://doi.org/10.1007/s13042-014-0276-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-014-0276-7

Keywords

Search

Navigation