Skip to main content

Advertisement

Springer Nature Link
Log in

Solving feature selection problems by combining mutation and crossover operations with the monarch butterfly optimization algorithm

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Feature selection (FS) is used to solve hard optimization problems in artificial intelligence and data mining. In the FS process, some, rather than all of the features of a dataset are selected in order to both maximize classification accuracy and minimize the time required for computation. In this paper a FS wrapper method that uses K-nearest Neighbor (KSN) classification is subjected to two modifications using a current improvement algorithm, the Monarch Butterfly Optimization (MBO) algorithm. The first modification, named MBOICO, involves the utilization of an enhanced crossover operator to improve FS. The second, named MBOLF, integrates the Lévy flight distribution into the MBO to improve convergence speed. Experiments are carried out on 25 benchmark data sets using the original MBO, MBOICO and MBOLF. The results show that MBOICO is superior, so its performance is also compared against that of four metaheuristic algorithms (PSO, ALO, WOASAT, and GA). The results indicate that it has a high classification accuracy rate of 93% on average for all datasets and significantly reduces the selection size. Hence, the findings demonstrate that the MBOICO outperforms the other algorithms in terms of classification accuracy and number of features chosen (selection size).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Explore related subjects

Discover the latest articles and news from researchers in related subjects, suggested using machine learning.

References

  1. Niu X, Wang J (2019) A combined model based on data preprocessing strategy and multi-objective optimization algorithm for short-term wind speed forecasting. Appl Energy 241:519–539

    Google Scholar 

  2. Masoudi-Sobhanzadeh Y, Motieghader H, Masoudi-Nejad A (2019) FeatureSelect: a software for feature selection based on machine learning approaches. BMC Bioinf 20(1):170

    Google Scholar 

  3. Sayed S, Nassef M, Badr A, Farag I (2019) A nested genetic algorithm for feature selection in high-dimensional cancer microarray datasets. Expert Syst Appl 121:233–243

    Google Scholar 

  4. Alweshah M, Qadoura MA, Hammouri AI, Azmi MS, AlKhalaileh S (2020) Flower Pollination Algorithm for Solving Classification Problems. Int. J. Advance Soft Compu. Appl 12(1)

  5. Al Nsour H, Alweshah M, Hammouri AI, Al Ofeishat H, Mirjalili S (2018) A hybrid grey wolf optimiser algorithm for solving time series classification problems. J Intell Syst 29(1):846–857

    Google Scholar 

  6. Mohamed AW, Hadi AA, Jambi KM (2019) Novel mutation strategy for enhancing SHADE and LSHADE algorithms for global numerical optimization. Swarm and Evolutionary Computation 50:100455

    Google Scholar 

  7. Manoj RJ, Praveena MA, Vijayakumar K (2019) An ACO–ANN based feature selection algorithm for big data. Clust Comput 22(2):3953–3960

    Google Scholar 

  8. Alweshah M, Al-Sendah M, Dorgham OM, Al-Momani A, Tedmori S (2020) Improved water cycle algorithm with probabilistic neural network to solve classification problems. Clust Comput:1–16

  9. M Alweshah, L Rababa, MH Ryalat, A Al Momani, and MF Ababneh (2020). “African Buffalo algorithm: training the probabilistic neural network to solve classification problems,” Journal of King Saud University-Computer and Information Sciences

  10. M Alweshah, A Al-Daradkeh, MA Al-Betar, A Almomani, and S Oqeili (2019). “$$\beta $$ β-hill climbing algorithm with probabilistic neural network for classification problems,” J Ambient Intell Humaniz Comput, pp. 1–12

  11. Hashemi A, Dowlatshahi MB, Nezamabadi-pour H (2020) MGFS: a multi-label graph-based feature selection algorithm via PageRank centrality. Expert Syst Appl 142:113024

    Google Scholar 

  12. H Djellali, NG Zine, and N Azizi (2016). “Two stages feature selection based on filter ranking methods and SVMRFE on medical applications,” In Modelling and Implementation of Complex Systems: Springer, pp. 281–293

  13. Hancer E, Xue B, Zhang M (2018) Differential evolution for filter feature selection based on information theory and feature ranking. Knowl-Based Syst 140:103–119

    Google Scholar 

  14. Apolloni J, Leguizamón G, Alba E (2016) Two hybrid wrapper-filter feature selection algorithms applied to high-dimensional microarray experiments. Appl Soft Comput 38:922–932

    Google Scholar 

  15. Liu H, Sun J, Liu L, Zhang H (2009) Feature selection with dynamic mutual information. Pattern Recogn 42(7):1330–1339

    MATH  Google Scholar 

  16. Karegowda AG, Manjunath A, Jayaram M (2010) Comparative study of attribute selection using gain ratio and correlation based feature selection. International Journal of Information Technology and Knowledge Management 2(2):271–277

    Google Scholar 

  17. Uğuz H (2011) A two-stage feature selection method for text categorization by using information gain, principal component analysis and genetic algorithm. Knowl-Based Syst 24(7):1024–1032

    Google Scholar 

  18. J Tang, S Alelyani, and H Liu (2014). “Feature selection for classification: A review,” Data classification: Algorithms and applications, p. 37

  19. Rodriguez-Galiano V, Luque-Espinar J, Chica-Olmo M, Mendes M (2018) Feature selection approaches for predictive modelling of groundwater nitrate pollution: an evaluation of filters, embedded and wrapper methods. Sci Total Environ 624:661–672

    Google Scholar 

  20. Yusta SC (2009) Different metaheuristic strategies to solve the feature selection problem. Pattern Recogn Lett 30(5):525–534

    Google Scholar 

  21. Wang J, Hedar A-R, Wang S, Ma J (2012) Rough set and scatter search metaheuristic based feature selection for credit scoring. Expert Syst Appl 39(6):6123–6128

    Google Scholar 

  22. Wahono RS, Suryana N, Ahmad S (2014) Metaheuristic optimization based feature selection for software defect prediction. Journal of Software 9(5):1324–1333

    Google Scholar 

  23. Alweshah M, Rashaideh H, Hammouri AI, Tayyeb H, Ababneh M (2017) Solving time series classification problems using support vector machine and neural network. International journal of data analysis techniques and strategies 9(3):237–247

    Google Scholar 

  24. SR Ahmad, AA Bakar, and MR Yaakub (2015). “Metaheuristic algorithms for feature selection in sentiment analysis,” In 2015 Science and Information Conference (SAI), pp. 222–226: IEEE

  25. M Alweshah, E Ramadan, MH Ryalat, M Almi’ani, and AI Hammouri (2020). “Water Evaporation Algorithm With Probabilistic Neural Network For Solving Classification Problems,” Jordanian Journal of Computers and Information Technology (JJCIT), vol. 6, no. 01

  26. Gokalp O, Tasci E, Ugur A (2020) A novel wrapper feature selection algorithm based on iterated greedy metaheuristic for sentiment classification. Expert Syst Appl 146:113176

    Google Scholar 

  27. Alweshah M, Abdullah S (2015) Hybridizing firefly algorithms with a probabilistic neural network for solving classification problems. Appl Soft Comput 35:513–524

    Google Scholar 

  28. Alweshah M, Alzubi OA, Alzubi JA, Alaqeel S (2016) Solving attribute reduction problem using wrapper genetic programming. International Journal of Computer Science and Network Security (IJCSNS) 16(5):77

    Google Scholar 

  29. Alweshah M (2019) Construction biogeography-based optimization algorithm for solving classification problems. Neural Comput & Applic 31(10):5679–5688

    Google Scholar 

  30. X-S Yang (2010). Engineering optimization: an introduction with metaheuristic applications. Wiley

  31. Alweshah M, Hammouri AI, Tedmori S (2017) Biogeography-based optimisation for data classification problems. International Journal of Data Mining, Modelling and Management 9(2):142–162

    Google Scholar 

  32. Sayed GI, Tharwat A, Hassanien AE (2019) Chaotic dragonfly algorithm: an improved metaheuristic algorithm for feature selection. Appl Intell 49(1):188–205

    Google Scholar 

  33. Aghdam MH, Ghasem-Aghaee N, Basiri ME (2009) Text feature selection using ant colony optimization. Expert Syst Appl 36(3):6843–6853

    Google Scholar 

  34. Sakri SB, Rashid NBA, Zain ZM (2018) Particle swarm optimization feature selection for breast cancer recurrence prediction. IEEE Access 6:29637–29647

    Google Scholar 

  35. Shunmugapriya P, Kanmani S (2017) A hybrid algorithm using ant and bee colony optimization for feature selection and classification (AC-ABC hybrid). Swarm and Evolutionary Computation 36:27–36

    Google Scholar 

  36. Mafarja M, Aljarah I, Heidari AA, Hammouri AI, Faris H, al-Zoubi A’M, Mirjalili S (2018) Evolutionary population dynamics and grasshopper optimization approaches for feature selection problems. Knowl-Based Syst 145:25–45

    Google Scholar 

  37. Sayed GI, Hassanien AE, Azar AT (2019) Feature selection via a novel chaotic crow search algorithm. Neural Comput & Applic 31(1):171–188

    Google Scholar 

  38. Taradeh M, Mafarja M, Heidari AA, Faris H, Aljarah I, Mirjalili S, Fujita H (2019) An evolutionary gravitational search-based feature selection. Inf Sci 497:219–239

    Google Scholar 

  39. Lin K-C, Zhang K-Y, Huang Y-H, Hung JC, Yen N (2016) Feature selection based on an improved cat swarm optimization algorithm for big data classification. J Supercomput 72(8):3210–3221

    Google Scholar 

  40. Diao R, Shen Q (2012) Feature selection with harmony search. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 42(6):1509–1523

    Google Scholar 

  41. D Rodrigues et al. (2013). “BCS: A binary cuckoo search algorithm for feature selection,” In 2013 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 465–468: IEEE

  42. Alweshah M, Al-Daradkeh A, Al-Betar MA, Almomani A, Oqeili S (2019) $$\beta $$-hill climbing algorithm with probabilistic neural network for classification problems. J Ambient Intell Humaniz Comput:1–12

  43. Al Nsour H, Alweshah M, Hammouri AI, Al Ofeishat H, Mirjalili S (2019) A hybrid Grey wolf Optimiser algorithm for solving time series classification problems. J Intell Syst

  44. Mohammed Alweshah, Moad Abu Qadoura, Abdelaziz I. Hammouri, Mohd Sanusi Azmi, and S Alkhalaileh (2019). “Flower Pollination Algorithm for solving classification problems,” International Journal of Advances in Soft Computing and Its Applications, vol. In Press, pp. 1–13

  45. Wang G-G, Deb S, Cui Z (2019) Monarch butterfly optimization. Neural Comput & Applic 31(7):1995–2014

    Google Scholar 

  46. Alweshah M, Al Khalaileh S, Gupta BB, Almomani A, Hammouri AI, Al-Betar MA (2020) The monarch butterfly optimization algorithm for solving feature selection problems. Neural Comput & Applic:1–15

  47. Wang G-G, Deb S, Coelho LDS (2018) Earthworm optimisation algorithm: a bio-inspired metaheuristic algorithm for global optimisation problems. International Journal of Bio-Inspired Computation 12(1):1–22

    Google Scholar 

  48. Sayed GI, Khoriba G, Haggag MH (2018) A novel chaotic salp swarm algorithm for global optimization and feature selection. Appl Intell 48(10):3462–3481

    Google Scholar 

  49. Chatterjee B, Bhattacharyya T, Ghosh KK, Singh PK, Geem ZW, Sarkar R (2020) Late acceptance hill climbing based social ski driver algorithm for feature selection. IEEE Access 8:75393–75408

    Google Scholar 

  50. M Ghosh, S Malakar, S Bhowmik, R Sarkar, and M Nasipuri (2019). “Feature Selection for Handwritten Word Recognition Using Memetic Algorithm,” In Advances in Intelligent Computing: Springer, pp. 103–124

  51. HM Zawbaa, E Emary, and B Parv (2015). “Feature selection based on antlion optimization algorithm,” in Complex Systems (WCCS), 2015 Third World Conference on, pp. 1–7: IEEE

  52. C Shivalingegowda and P Jayasree (2020). “Hybrid gravitational search algorithm based model for optimizing coverage and connectivity in wireless sensor networks,” J Ambient Intell Humaniz Comput, pp. 1–14

  53. Abdel-Basset M, El-Shahat D, El-henawy I, de Albuquerque VHC, Mirjalili S (2020) A new fusion of grey wolf optimizer algorithm with a two-phase mutation for feature selection. Expert Syst Appl 139:112824

    Google Scholar 

  54. S Wu (2015). “Comparative analysis of particle swarm optimization algorithms for text feature selection,”

  55. Zawbaa HM, Emary E, Grosan C (2016) Feature selection via chaotic antlion optimization. PloS one 11(3):e0150652

    Google Scholar 

  56. J Wang, B Xue, X Gao, and M Zhang (2016). “A differential evolution approach to feature selection and instance selection,” In Pacific Rim International Conference on Artificial Intelligence, pp. 588–602: Springer

  57. Shahbeig S, SADJAD K, SADEGHI M (2016) Feature selection from Iron direct reduction data based on binary differential evolution optimization. Bulletin de la Société Royale des Sciences de Liège 85:114–122

    Google Scholar 

  58. Malakar S, Ghosh M, Bhowmik S, Sarkar R, Nasipuri M (2019) A GA based hierarchical feature selection approach for handwritten word recognition. Neural Comput & Applic:1–20

  59. R Saidi, W Bouaguel, and N Essoussi (2019). “Hybrid Feature Selection Method Based on the Genetic Algorithm and Pearson Correlation Coefficient,” In Machine Learning Paradigms: Theory and Application: Springer, pp. 3–24

  60. Emary E, Zawbaa HM, Hassanien AE (2016) Binary ant lion approaches for feature selection. Neurocomputing 213:54–65

    Google Scholar 

  61. ME Basiri and S Nemati (2009). “A novel hybrid ACO-GA algorithm for text feature selection,” In Evolutionary Computation, 2009. CEC’09. IEEE Congress on, pp. 2561–2568: IEEE

  62. Jona J, Nagaveni N (2014) Ant-cuckoo colony optimization for feature selection in digital mammogram. Pakistan journal of biological sciences: PJBS 17(2):266–271

    Google Scholar 

  63. Mafarja M, Abdullah S (2013) Investigating memetic algorithm in solving rough set attribute reduction. Int J Comput Appl Technol 48(3):195–202

    Google Scholar 

  64. Zorarpacı E, Özel SA (2016) A hybrid approach of differential evolution and artificial bee colony for feature selection. Expert Syst Appl 62:91–103

    Google Scholar 

  65. Zhang Y, Gong D-w, Cheng J (2015) Multi-objective particle swarm optimization approach for cost-based feature selection in classification. IEEE/ACM transactions on computational biology and bioinformatics 14(1):64–75

    Google Scholar 

  66. Zhang Y, Gong D, Hu Y, Zhang W (2015) Feature selection algorithm based on bare bones particle swarm optimization. Neurocomputing 148:150–157

    Google Scholar 

  67. Mafarja MM, Mirjalili S (2017) Hybrid whale optimization algorithm with simulated annealing for feature selection. Neurocomputing 260:302–312

    Google Scholar 

  68. Mafarja M, Aljarah I, Faris H, Hammouri AI, Ala’M A-Z, Mirjalili S (2019) Binary grasshopper optimisation algorithm approaches for feature selection problems. Expert Syst Appl 117:267–286

    Google Scholar 

  69. Faris H, Mafarja MM, Heidari AA, Aljarah I, al-Zoubi A’M, Mirjalili S, Fujita H (2018) An efficient binary salp swarm algorithm with crossover scheme for feature selection problems. Knowl-Based Syst 154:43–67

    Google Scholar 

  70. Alweshah M, Alkhalaileh S, Albashish D, Mafarja M, Bsoul Q, Dorgham O (2020) A hybrid mine blast algorithm for feature selection problems. Soft Comput:1–18

  71. Feng Y, Yang J, Wu C, Lu M, Zhao X-J (2018) Solving 0–1 knapsack problems by chaotic monarch butterfly optimization algorithm with Gaussian mutation. Memetic Computing 10(2):135–150

    Google Scholar 

  72. S Chakrabarty, AK Pal, N Dey, D Das, and S Acharjee (2014). “Foliage area computation using Monarch butterfly algorithm,” In Non Conventional Energy (ICONCE), 2014 1st International Conference on, pp. 249–253: IEEE

  73. Ghanem WA, Jantan A (2018) Hybridizing artificial bee colony with monarch butterfly optimization for numerical optimization problems. Neural Comput & Applic 30(1):163–181

    Google Scholar 

  74. Devikanniga D, Raj RJS (2018) Classification of osteoporosis by artificial neural network based on monarch butterfly optimisation algorithm. Healthcare technology letters 5(2):70–75

    Google Scholar 

  75. I Strumberger, M Sarac, D Markovic, and N Bacanin (2018). “Hybridized Monarch Butterfly Algorithm for Global Optimization Problems,” International Journal of Computers, vol. 3

  76. Faris H, Aljarah I, Mirjalili S (2018) Improved monarch butterfly optimization for unconstrained global search and neural network training. Appl Intell 48(2):445–464

    Google Scholar 

  77. Yazdani S, Hadavandi E (2018) LMBO-DE: a linearized monarch butterfly optimization algorithm improved with differential evolution. Soft Comput:1–15

  78. I Stromberger, E Tuba, N Bacanin, M Beko, and M Tuba (2018). “Monarch butterfly optimization algorithm for localization in wireless sensor networks,” In Radioelektronika (RADIOELEKTRONIKA), 2018 28th International Conference, pp. 1–6: IEEE

  79. G-G Wang, X Zhao, and S Deb (2015). “A novel monarch butterfly optimization with greedy strategy and self-adaptive,” In Soft Computing and Machine Intelligence (ISCMI), 2015 Second International Conference on, pp. 45–50: IEEE

  80. Feng Y, Wang G-G, Li W, Li N (2018) Multi-strategy monarch butterfly optimization algorithm for discounted {0-1} knapsack problem. Neural Comput & Applic 30(10):3019–3036

    Google Scholar 

  81. G-G Wang, X Zhao, and S Deb (2015). “A novel monarch butterfly optimization with greedy strategy and self-adaptive,” in 2015 Second International Conference on Soft Computing and Machine Intelligence (ISCMI), pp. 45–50: IEEE

  82. Wang G-G, Deb S, Zhao X, Cui Z (2018) A new monarch butterfly optimization with an improved crossover operator. Oper Res 18(3):731–755

    Google Scholar 

  83. Brown CT, Liebovitch LS, Glendon R (2007) Lévy flights in Dobe Ju/‘hoansi foraging patterns. Hum Ecol 35(1):129–138

    Google Scholar 

  84. Pavlyukevich I (2007) Lévy flights, non-local search and simulated annealing. J Comput Phys 226(2):1830–1844

    MathSciNet  MATH  Google Scholar 

  85. Reynolds AM, Frye MA (2007) Free-flight odor tracking in Drosophila is consistent with an optimal intermittent scale-free search. PloS one 2(4):e354

    Google Scholar 

  86. Barthelemy P, Bertolotti J, Wiersma DS (2008) A Lévy flight for light. Nature 453(7194):495

    Google Scholar 

  87. Shlesinger MF (2006) Mathematical physics: Search research. Nature 443(7109):281

    Google Scholar 

  88. P Lévy (1954). Théorie de l’addition des variables aléatoires. Gauthier-Villars

  89. He X, Huang J, Rao Y, Gao L (2016) Chaotic teaching-learning-based optimization with Lévy flight for global numerical optimization. Computational intelligence and neuroscience 2016:43

    Google Scholar 

  90. Gandomi AH, Yang X-S, Alavi AH (2013) Cuckoo search algorithm: a metaheuristic approach to solve structural optimization problems. Eng Comput 29(1):17–35

    Google Scholar 

  91. Jensi R, Jiji GW (2016) An enhanced particle swarm optimization with levy flight for global optimization. Appl Soft Comput 43:248–261

    Google Scholar 

  92. H Liu and H Motoda (2007). Computational methods of feature selection. CRC Press

  93. He J, Yao X (2001) Drift analysis and average time complexity of evolutionary algorithms. Artif Intell 127(1):57–85

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

The work published in this paper was funded by Al-Balqa Applied University, The Deanship of Scientific Research, Al-Salt, Jordan.

Author information

Authors and Affiliations

  1. Prince Abdullah Bin Ghazi Faculty of Information and Communication Technology, Al-Balqa Applied University, Al-Salt, Jordan

    Mohammed Alweshah

Authors
  1. Mohammed Alweshah
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Mohammed Alweshah.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Alweshah, M. Solving feature selection problems by combining mutation and crossover operations with the monarch butterfly optimization algorithm. Appl Intell 51, 4058–4081 (2021). https://doi.org/10.1007/s10489-020-01981-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-020-01981-0

Keywords