Abstract
Cluster analysis is one important field in pattern recognition and machine learning, consisting in an attempt to distribute a set of data patterns into groups, considering only the inner properties of those data. One of the most popular techniques for data clustering is the K-Means algorithm, due to its simplicity and easy implementation. But K-Means is strongly dependent on the initial point of the search, what may lead to suboptima (local optima) solutions. In the past few decades, Evolutionary Algorithms (EAs), like Group Search Optimization (GSO), have been adapted to the context of cluster analysis, given their global search capabilities and flexibility to deal with hard optimization problems. However, given their stochastic nature, EAs may be slower to converge in comparison to traditional clustering models (like K-Means). In this work, three hybrid memetic approaches between K-Means and GSO are presented, named FMKGSO, MKGSO and TMKGSO, in such a way that the global search capabilities of GSO are combined with the fast local search performances of K-Means. The degree of influence of K-Means on the behavior of GSO method is evaluated by a set of experiments considering both real-world problems and synthetic data sets, using five clustering metrics to access how good and robust the proposed hybrid memetic models are.
Similar content being viewed by others
References
Abdel-Kader RF (2010) Genetically improved pso algorithm for efficient data clustering. In: Machine Learning and Computing (ICMLC), 2010 Second International Conference on, pp. 71–75. IEEE
Abualigah L (2020) Group search optimizer: a nature-inspired meta-heuristic optimization algorithm with its results, variants, and applications. Neural Computing and Applications pp. 1–24
Ahmadi A, Karray F, Kamel MS (2010) Flocking based approach for data clustering. Nat Comput 9(3):767–791
Ahmadyfard A, Modares H (2008) Combining pso and k-means to enhance data clustering. In: Telecommunications, 2008. IST 2008. International Symposium on, pp. 688–691. IEEE
Akbari M, Izadkhah H (2019) Gakh: A new evolutionary algorithm for graph clustering problem. In: 2019 4th International Conference on Pattern Recognition and Image Analysis (IPRIA), pp. 159–162. IEEE
Arabie P, Hubert LJ, De Soete G (1996) Clustering and classification. World Scientific, Singapore
Asuncion A, Newman D (2007) Uci machine learning repository
Barnard C, Sibly R (1981) Producers and scroungers: a general model and its application to captive flocks of house sparrows. Anim Behav 29(2):543–550
BEDDAD B, HACHEMI K, POSTAIRE JG, JABLONCIK F, MESSAI O (2019) An improvement of spatial fuzzy c-means clustering method for noisy medical image analysis. In: 2019 6th International Conference on Image and Signal Processing and their Applications (ISPA), pp. 1–5. IEEE
Bhavani R, Sadasivam GS, Kumaran R (2011) A novel parallel hybrid k-means-de-aco clustering approach for genomic clustering using mapreduce. In: Information and Communication Technologies (WICT), 2011 World Congress on, pp. 132–137. IEEE
Bonabeau E, Dorigo M, Theraulaz G (1999) Swarm intelligence: from natural to artificial systems, vol 4. Oxford University Press, New York
Bruse JL, Zuluaga MA, Khushnood A, McLeod K, Ntsinjana HN, Hsia TY, Sermesant M, Pennec X, Taylor AM, Schievano S (2017) Detecting clinically meaningful shape clusters in medical image data: metrics analysis for hierarchical clustering applied to healthy and pathological aortic arches. IEEE Trans Biomed Eng 64(10):2373–2383
Canuto A, Neto AF, Silva HM, Xavier-Júnior JC, Barreto CA (2018) Population-based bio-inspired algorithms for cluster ensembles optimization. Natural Computing pp. 1–18
Chen CY, Ye F (2004) Particle swarm optimization algorithm and its application to clustering analysis. In: Networking, Sensing and Control, 2004 IEEE International Conference on, vol. 2, pp. 789–794. IEEE
Chen G, Luo W, Zhu T (2014) Evolutionary clustering with differential evolution. In: Evolutionary Computation (CEC), 2014 IEEE Congress on, pp. 1382–1389. IEEE
Chen J, Zheng J, Liu Y, Wu Q (2014) Dynamic economic dispatch with wind power penetration using group search optimizer with adaptive strategies. In: IEEE PES Innovative Smart Grid Technologies, Europe, pp. 1–6. IEEE
Cho PPW, Nyunt TTS (2020) Data clustering based on differential evolution with modified mutation strategy. In: 2020 17th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology (ECTI-CON), pp. 222–225. IEEE
Civicioglu P (2013) Backtracking search optimization algorithm for numerical optimization problems. Appl Math Comput 219(15):8121–8144
Couzin ID, Krause J, Franks NR, Levin SA (2005) Effective leadership and decision-making in animal groups on the move. Nature 433(7025):513–516
Cui X, Potok TE, Palathingal P (2005) Document clustering using particle swarm optimization. In: Proceedings 2005 IEEE Swarm Intelligence Symposium, 2005. SIS 2005., pp. 185–191. IEEE
Darwish A, Ezzat D, Hassanien AE (2020) An optimized model based on convolutional neural networks and orthogonal learning particle swarm optimization algorithm for plant diseases diagnosis. Swarm Evol Comput 52:100616
Das S, Abraham A, Konar A (2007) Automatic clustering using an improved differential evolution algorithm. IEEE Trans Syst Man Cybern Part A Syst Hum 38(1):218–237
Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30
Dhiviya S, Sariga A, Sujatha P (2017) Survey on wsn using clustering. In: 2017 Second International Conference on Recent Trends and Challenges in Computational Models (ICRTCCM), pp. 121–125. IEEE
Diderot PKG, Vasudevan N, Sankaran KS (2019) An efficient fuzzy c-means clustering based image dissection algorithm for satellite images. In: 2019 International Conference on Communication and Signal Processing (ICCSP), pp. 0806–0809. IEEE
Dixon A (1959) An experimental study of the searching behaviour of the predatory coccinellid beetle adalia decempunctata (l.). The Journal of Animal Ecology pp. 259–281
Dorigo M, Maniezzo V, Colorni A (1996) Ant system: optimization by a colony of cooperating agents. Syst Man Cybern Part B Cybern IEEE Trans 26(1):29–41
Eiben AE, Smith JE (2015) Introduction to evolutionary computing. Springer, Berlin
Elaziz MA, Nabil N, Ewees AA, Lu S (2019) Automatic data clustering based on hybrid atom search optimization and sine-cosine algorithm. In: 2019 IEEE Congress on Evolutionary Computation (CEC), pp. 2315–2322. IEEE
Esmin AAA, Pereira DL, De Araujo F (2008) Study of different approach to clustering data by using the particle swarm optimization algorithm. In: Evolutionary Computation, 2008. CEC 2008.(IEEE World Congress on Computational Intelligence). IEEE Congress on, pp. 1817–1822. IEEE
Figueiredo E, Macedo M, Siqueira HV, Santana CJ Jr, Gokhale A, Bastos-Filho CJ (2019) Swarm intelligence for clustering-a systematic review with new perspectives on data mining. Eng Appl Artif Intell 82:313–329
Fogel D (2009) Artificial intelligence through simulated evolution. Wiley-IEEE Press, New Jersy
Fogel DB (2006) Evolutionary computation: toward a new philosophy of machine intelligence, vol 1. Wiley, New Jersy
Fogel LJ, Owens AJ, Walsh MJ (1966) Artificial intelligence through simulated evolution. Wiley, New Jersy
Friedman M (1937) The use of ranks to avoid the assumption of normality implicit in the analysis of variance. J Am Stat Assoc 32(200):675–701
Friedman M (1940) A comparison of alternative tests of significance for the problem of m rankings. Ann Math Stat 11(1):86–92
Geem ZW (2010) Recent advances in harmony search algorithm, vol 270. Springer, Berlin
Z.W Geem, Kim JH, Loganathan GV (2001) A new heuristic optimization algorithm: harmony search. Simulation 76(2):60–68
Günen MA, Atasever ÜH, Beşdok E (2017) A novel edge detection approach based on backtracking search optimization algorithm (bsa) clustering. In: 2017 8th International Conference on Information Technology (ICIT), pp. 116–122. IEEE
Hassan BA, Rashid TA (2020) Operational framework for recent advances in backtracking search optimisation algorithm: a systematic review and performance evaluation. Appl Math Comput 370:124919
He H, Tan Y (2012) A two-stage genetic algorithm for automatic clustering. Neurocomputing 81:49–59
He S, Wu Q, Saunders J (2006) A novel group search optimizer inspired by animal behavioural ecology. In: 2006 IEEE Congress on Evolutionary Computation (CEC), pp. 1272–1278. IEEE
He S, Wu QH, Saunders JR (2009) Group search optimizer: an optimization algorithm inspired by animal searching behavior. IEEE Trans Evol Comput 13(5):973–990
Higgins CL, Strauss RE (2004) Discrimination and classification of foraging paths produced by search-tactic models. Behav Ecol 15(2):248–254
Holland JH (1992) Genetic algorithms. Sci Am 267(1):66–72
Hruschka ER, Campello RJ, Freitas AA et al (2009) A survey of evolutionary algorithms for clustering. IEEE Trans Syst Man Cybern Part C Appl Rev 39(2):133–155
Hubert L, Arabie P (1985) Comparing partitions. J Classif 2(1):193–218
Idrissi MAJ, Ramchoun H, Ghanou Y, Ettaouil M (2016) Genetic algorithm for neural network architecture optimization. In: 2016 3rd International Conference on Logistics Operations Management (GOL), pp. 1–4. IEEE
Inkaya T, Kayalıgil S, Özdemirel NE (2016) Swarm intelligence-based clustering algorithms: A survey. In: Unsupervised learning algorithms, pp. 303–341. Springer
Islam MT, Basak PK, Bhowmik P, Khan M (2019) Data clustering using hybrid genetic algorithm with k-means and k-medoids algorithms. In: 2019 23rd International Computer Science and Engineering Conference (ICSEC), pp. 123–128. IEEE
Jain M, Singh V, Rani A (2019) A novel nature-inspired algorithm for optimization: Squirrel search algorithm. Swarm Evol Comput 44:148–175
José-García A, Gómez-Flores W (2016) Automatic clustering using nature-inspired metaheuristics: a survey. Appl Soft Comput 41:192–213
Junaed A, Akhand M, Murase K, et al (2013) Multi-producer group search optimizer for function optimization. In: 2013 International Conference on Informatics, Electronics and Vision (ICIEV), pp. 1–4. IEEE
Karaboga D, Basturk B (2007) A powerful and efficient algorithm for numerical function optimization: artificial bee colony (abc) algorithm. J Global Optim 39(3):459–471
Kennedy J (2006) Swarm intelligence. Handbook of nature-inspired and innovative computing. Springer, Berlin, pp 187–219
Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Neural Networks, 1995. Proceedings., IEEE International Conference on, vol. 4, pp. 1942–1948. IEEE
Kennedy J, Eberhart RC, Shi Y (2001) Swarm intelligence. Kaufmann, San Francisco
Koza JR, Koza JR (1992) Genetic programming: on the programming of computers by means of natural selection, vol 1. MIT press, Cambridge
Krishnaprabha R, Aloor G (2014) Group search optimizer algorithm in wireless sensor network localization. In: 2014 International Conference on Advances in Computing, Communications and Informatics (ICACCI), pp. 1953–1957. IEEE
Latiff NA, Malik NNA, Idoumghar L (2016) Hybrid backtracking search optimization algorithm and k-means for clustering in wireless sensor networks. In: 2016 IEEE 14th Intl Conf on Dependable, Autonomic and Secure Computing, 14th Intl Conf on Pervasive Intelligence and Computing, 2nd Intl Conf on Big Data Intelligence and Computing and Cyber Science and Technology Congress (DASC/PiCom/DataCom/CyberSciTech), pp. 558–564. IEEE
Li L, Liang Y, Li T, Wu C, Zhao G, Han X (2019) Boost particle swarm optimization with fitness estimation. Nat Comput 18(2):229–247
Li L, Xu S, Wang S, Ma X (2016) The diseases clustering for multi-source medical sets. In: 2016 International Conference on Identification, Information and Knowledge in the Internet of Things (IIKI), pp. 294–298. IEEE
Li T, Dong H (2019) Unsupervised feature selection and clustering optimization based on improved differential evolution. IEEE Access 7:140438–140450
Li Xl (2002) An optimizing method based on autonomous animats: fish-swarm algorithm. Syst Eng Theory Pract 22(11):32–38
Li Yz, Zheng Xw, Lu Dj (2015) Virtual network embedding based on multi-objective group search optimizer. In: 2015 10th International Conference on Broadband and Wireless Computing, Communication and Applications (BWCCA), pp. 598–601. IEEE
Li Z, Hu Z, Miao Y, Xiong Z, Xu X, Dai C (2019) Deep-mining backtracking search optimization algorithm guided by collective wisdom. Mathematical Problems in Engineering 2019
Lin CJ, Huang ML (2019) Efficient hybrid group search optimizer for assembling printed circuit boards. AI EDAM 33(3):259–274
Liu F, Xiong L (2011) Survey on text clustering algorithm-research present situation of text clustering algorithm. In: 2011 IEEE 2nd International Conference on Software Engineering and Service Science, pp. 196–199. IEEE
Liu Y, Wu X, Shen Y (2011) Automatic clustering using genetic algorithms. Appl Math Comput 218(4):1267–1279
MacQueen J, et al (1967) Some methods for classification and analysis of multivariate observations. In: Proceedings of the fifth Berkeley symposium on mathematical statistics and probability, vol. 1, pp. 281–297. California, USA
Masoud MZ, Jaradat Y, Zaidan D, Jannoud I (2019) To cluster or not to cluster: A hybrid clustering protocol for wsn. In: 2019 IEEE Jordan International Joint Conference on Electrical Engineering and Information Technology (JEEIT), pp. 678–682. IEEE
Miranda PB, Prudêncio RB (2018) A novel context-free grammar for the generation of pso algorithms. Natural Computing pp. 1–19
Mirjalili S (2016) Sca: a sine cosine algorithm for solving optimization problems. Knowl-Based Syst 96:120–133
Mirjalili S, Lewis A (2016) The whale optimization algorithm. Adv Eng Softw 95:51–67
Mirjalili S, Mirjalili SM, Lewis A (2014) Grey wolf optimizer. Adv Eng Softw 69:46–61
Misra S, Kumar R (2016) A literature survey on various clustering approaches in wireless sensor network. In: 2016 2nd international conference on communication control and intelligent systems (CCIS), pp. 18–22. IEEE
Mortezanezhad A, Daneshifar E (2019) Big-data clustering with genetic algorithm. In: 2019 5th Conference on Knowledge Based Engineering and Innovation (KBEI), pp. 702–706. IEEE
Naldi MC, Campello RJGB (2014) Evolutionary k-means for distributed data sets. Neurocomputing 127:30–42
Nemenyi P (1962) Distribution-free multiple comparisons. Biometrics 18(2):263
Niu B, Duan Q, Liu J, Tan L, Liu Y (2017) A population-based clustering technique using particle swarm optimization and k-means. Nat Comput 16(1):45–59
Oliveira JFL, Pacifico LDS, Ludermir TB (2013) A hybrid group search optimization based on fish swarms. In: 2013 Brazilian Conference on Intelligent Systems, pp. 51–56. IEEE
Pacifico LDS, Ludermir TB (2013) Cooperative group search optimization. In: 2013 IEEE Congress on Evolutionary Computation, pp. 3299–3306. IEEE
Pacifico LDS, Ludermir TB (2014) A group search optimization method for data clustering. In: Intelligent Systems (BRACIS), 2014 Brazilian Conference on, pp. 342–347. IEEE
Pacifico LDS, Ludermir TB (2014) Improved cooperative group search optimization based on divide-and-conquer strategy. In: 2014 Brazilian Conference on Intelligent Systems, pp. 420–425. IEEE
Pacifico LDS, Ludermir TB (2016) Data clustering using group search optimization with alternative fitness functions. In: 2016 5th Brazilian Conference on Intelligent Systems (BRACIS), pp. 301–306. IEEE
Pacifico LDS, Ludermir TB (2018) Hybrid k-means and improved group search optimization methods for data clustering. In: 2018 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE
Pacifico LDS, Ludermir TB (2019) Hybrid k-means and improved self-adaptive particle swarm optimization for data clustering. In: 2019 International Joint Conference on Neural Networks (IJCNN), pp. 1–7. IEEE
Pacifico LDS, Ludermir TB (2019) A partitional cooperative coevolutionary group search optimization approach for data clustering. In: 2019 8th Brazilian Conference on Intelligent Systems (BRACIS), pp. 347–352. IEEE
Pacifico LDS, Ludermir TB, Britto LFS (2018) A hybrid improved group search optimization and otsu method for color image segmentation. In: 2018 7th Brazilian Conference on Intelligent Systems (BRACIS), pp. 296–301. IEEE
Pacifico LDS, Ludermir TB, Oliveira JFL (2018) Evolutionary elms with alternative treatments for the population out-bounded individuals. In: 2018 7th Brazilian Conference on Intelligent Systems (BRACIS), pp. 151–156. IEEE
Parimalam T, Sundaram KM (2017) Efficient clustering techniques for web services clustering. In: 2017 ieee international conference on computational intelligence and computing research (iccic), pp. 1–4. IEEE
Passino KM (2002) Biomimicry of bacterial foraging for distributed optimization and control. IEEE Control Syst Mag 22(3):52–67
Prabha KA, Visalakshi NK (2014) Improved particle swarm optimization based k-means clustering. In: 2014 International Conference on Intelligent Computing Applications, pp. 59–63. IEEE
Premalatha P, Subasree S (2017) Performance analysis of clustering algorithms in medical datasets. In: 2017 Second International Conference on Electrical, Computer and Communication Technologies (ICECCT), pp. 1–6. IEEE
Rahamathunnisa U, Nallakaruppan M, Anith A, Kumar KS (2020) Vegetable disease detection using k-means clustering and svm. In: 2020 6th International Conference on Advanced Computing and Communication Systems (ICACCS), pp. 1308–1311. IEEE
Ramos AC, Vellasco M (2018) Quantum-inspired evolutionary algorithm for feature selection in motor imagery eeg classification. In: 2018 IEEE Congress on Evolutionary Computation (CEC), pp. 1–8. IEEE
Rashedi E, Nezamabadi-Pour H, Saryazdi S (2009) Gsa: a gravitational search algorithm. Inf Sci 179(13):2232–2248
Rechenberg I (1973) Evolution strategy: optimization of technical systems by means of biological evolution. Fromman Holzboog Stuttgart 104:15–16
Ren Z, Zhang A, Wen C, Feng Z (2014) A scatter learning particle swarm optimization algorithm for multimodal problems. Cybern IEEE Trans 44(7):1127–1140
Sapkota N, Alsadoon A, Prasad P, Elchouemi A, Singh AK (2019) Data summarization using clustering and classification: Spectral clustering combined with k-means using nfph. In: 2019 International Conference on Machine Learning, Big Data, Cloud and Parallel Computing (COMITCon), pp. 146–151. IEEE
Saraswathi S, Allirani A (2013) Survey on image segmentation via clustering. In: 2013 International Conference on Information Communication and Embedded Systems (ICICES), pp. 331–335. IEEE
Saremi S, Mirjalili S, Lewis A (2017) Grasshopper optimisation algorithm: theory and application. Adv Eng Softw 105:30–47
Schwefel HPP (1993) Evolution and optimum seeking: the sixth generation. Wiley, New Jersy
Shi H, Xu M (2018) A data classification method using genetic algorithm and k-means algorithm with optimizing initial cluster center. In: 2018 IEEE International Conference on Computer and Communication Engineering Technology (CCET), pp. 224–228. IEEE
Silva DNG, Pacifico LDS, Ludermir TB (2011) An evolutionary extreme learning machine based on group search optimization. In: 2011 IEEE Congress of Evolutionary Computation (CEC), pp. 574–580. IEEE
Simon D (2013) Evolutionary optimization algorithms. Wiley, New Jersy
Souza E, Santos D, Oliveira G, Silva A, Oliveira AL (2018) Swarm optimization clustering methods for opinion mining. Natural computing pp. 1–29
Sreepathi S, Kumar J, Mills RT, Hoffman FM, Sripathi V, Hargrove WW (2017) Parallel multivariate spatio-temporal clustering of large ecological datasets on hybrid supercomputers. In: 2017 IEEE International Conference on Cluster Computing (CLUSTER), pp. 267–277. IEEE
Storn R, Price K (1995) Differential evolution–a simple and efficient adaptive scheme for global optimization over continuous spaces. international computer science institute, berkeley. Tech. rep., CA, 1995, Tech. Rep. TR-95–012
Storn R, Price K (1997) Differential evolution-a simple, efficient heuristic for global optimization over continuous spaces. J Global Optim 11(4):341–359
Taj N, Basu A (2019) Hybridization of genetic and group search optimization algorithm for deadline-constrained task scheduling approach. J Intell Syst 28(1):153–171
Taşci E, Gökalp O, Uğur A (2018) Development of a novel feature weighting method using cma-es optimization. In: 2018 26th Signal Processing and Communications Applications Conference (SIU), pp. 1–4. IEEE
Toman SH, Abed MH, Toman ZH (2020) Cluster-based information retrieval by using (k-means)-hierarchical parallel genetic algorithms approach. arXiv preprint arXiv:2008.00150
Toz G, Yücedağ İ, Erdoğmuş P (2019) A fuzzy image clustering method based on an improved backtracking search optimization algorithm with an inertia weight parameter. J King Saud Univ Comput Inf Sci 31(3):295–303
Wan C, Ye M, Yao C, Wu C (2017) Brain mr image segmentation based on gaussian filtering and improved fcm clustering algorithm. In: 2017 10th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI), pp. 1–5. IEEE
Wang F (2018) A weighted k-means algorithm based on differential evolution. In: 2018 2nd IEEE Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC), pp. 1–2274. IEEE
Wang H, Zuo L, Liu J, Yi W, Niu B (2018) Ensemble particle swarm optimization and differential evolution with alternative mutation method. Natural Computing pp. 1–14
Wei Y, Niu C, Wang H, Liu D (2019) The hyperspectral image clustering based on spatial information and spectral clustering. In: 2019 IEEE 4th International Conference on Signal and Image Processing (ICSIP), pp. 127–131. IEEE
Wong MT, He X, Yeh WC (2011) Image clustering using particle swarm optimization. In: Evolutionary Computation (CEC), 2011 IEEE Congress on, pp. 262–268. IEEE
Xu D, Tian Y (2015) A comprehensive survey of clustering algorithms. Ann Data Sci 2(2):165–193
Xu H, Xue B, Zhang M (2020) A duplication analysis based evolutionary algorithm for bi-objective feature selection. IEEE Transactions on Evolutionary Computation
Xu Y, Shu Y (2006) Evolutionary extreme learning machine-based on particle swarm optimization. International Symposium on Neural Networks. Springer, Berlin, pp 644–652
Yang XS (2009) Firefly algorithms for multimodal optimization. In: International symposium on stochastic algorithms, pp. 169–178. Springer
Yang XS (2010) Firefly algorithm, levy flights and global optimization. In: Research and development in intelligent systems XXVI, pp. 209–218. Springer
Zhang M, Cao J (2020) An elitist-based differential evolution algorithm for multiobjective clustering. In: 2020 3rd International Conference on Artificial Intelligence and Big Data (ICAIBD), pp. 161–166. IEEE
Zhu L, Ma Y, Bai Y (2020) A self-adaptive multi-population differential evolution algorithm. Nat Comput 19(1):211–235
Acknowledgements
The authors would like to thank FACEPE, CNPq and CAPES (Brazilian Research Agencies) for their financial support.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Pacifico, L.D.S., Ludermir, T.B. An evaluation of k-means as a local search operator in hybrid memetic group search optimization for data clustering. Nat Comput 20, 611–636 (2021). https://doi.org/10.1007/s11047-020-09809-z
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11047-020-09809-z