Abstract
Real-parameter optimisation is a prolific research line with hundreds of publications per year. There exists an impressive number of alternatives in both algorithm families and enhancements over their respective original proposals. In this work, we analyse if this growth in the number of publications is correlated with a real progress in the field. We have selected five approaches from one of the most significant journals in the field and compared them with the winner of the competition celebrated within the IEEE Congress on Evolutionary Computation 2005. We observe that not only these methods are unable to get the good results of the winner of the competition, published several years before, but that they often avoid this type of comparison. Instead, they usually compare with other approaches from the same family. We conclude that the comparison with the state-of-the-art of the field should be mandatory to promote a real progress and to prevent that the area becomes obfuscated for outsiders.
Similar content being viewed by others
References
Abedinia O, Amjady N, Ghasemi A (2014) A new metaheuristic algorithm based on shark smell optimization. Complexity. doi:10.1002/cplx.21634
Addis B, Locatelli M (2007) A new class of test functions for global optimization. J Global Optim 38(3):479–501
Auger A, Hansen N (2005a) A restart CMA evolution strategy with increasing population size. In: IEEE Congress on Evolutionary Computation (CEC’05), vol 2, pp 1769–1776
Auger A, Hansen N (2005b) Performance evaluation of an advanced local search evolutionary algorithm. In: IEEE Congress on Evolutionary Computation (CEC’05), pp 1777–1784
Auger A, Hansen N, Schoenauer M (2012) Benchmarking of continuous black box optimization algorithms. Evol Comput 20(4):481–481
Awad NH, Ali MZ, Suganthan, PN, Reynolds RG (2016) An ensemble sinusoidal parameter adaptation incorporated with L-SHADE for solving CEC 2014 problems. In: IEEE Congress on Evolutionary Computation (CEC’16), pp 2958–2965
Bäck T, Schwefel HP (1993) An overview of evolutionary algorithms for parameter optimization. Evol Comput 1(1):1–23
Bersini H, Dorigo M, Langerman S, Seront G, Gambardella L (1996) Results of the first international contest on evolutionary optimisation (1st ICEO), In: IEEE Congress on Evolutionary Computation (CEC’96), pp 611–615
Box G (1957) Evolutionary operation: a method for increasing industrial productivity. Appl Stat 6:639–641
Bremermann H (1962) Optimization through evolutiona dn recombination. Spartan Books, Washington, pp 93–106
Brest J, Greiner S, Boskovic B, Mernik M, Zumer V (2006) Self-adapting control parameters in differential evolution. A comparative study on numerical benchmark problems. IEEE Trans Evol Comput 10(6):646–657
Clerc M, Kennedy J (2002) The particle swarm-explosion, stability, and convergence in a multidimensional complex space. IEEE Trans Evol Comput 6(1):58–73
Coello CAC (2002) Theoretical and numerical constraint-handling techniques used with evolutionary algorithms: a survey of the state of the art. Comput Methods Appl Mech Eng 191:1245–1287
Das S, Abraham A, Chakraborty U, Konar A (2009) Differential evolution using a neighbourhood-based mutation operator. IEEE Trans Evol Comput 13(3):526–553
de Oca MM, Stützle T, Birattari M, Dorigo M (2009) Frankenstein’s PSO: a composite particle swarm optimization algorithm. IEEE Trans Evol Comput 13(5):1120–1132
Deb K, Anand A, Joshi D (2001) A computationally efficient evolutionary algorithm for real-parameter optimization. Evol Comput 9(2):159–195
Demsar J (2006) Statistical comparisons of classifers over multiple data sets. J Mach Learn Res 7:1–30
Dorigo M, Stützle T (2004) Ant colony optimization. MIT Press, Cambridge
Dorigo M, Maniezzo V, Colorni A (1996) The ant system: optimization by a colony of cooperating agents. IEEE Trans Syst Man Cybern B Cybern 26(1):29–41
Eberhart R, Shi Y (2001) Tracking and optimizing dynamic systems with particle swarms. In: IEEE Congress on Evolutionary Computation (CEC’01), pp 94–100
Esbensen H, Mazumder P (1994) SAGA: a unification of the genetic algorithm with simulated annealing and its application to macro-cell placement. In: IEEE Int. Conf. VLSI Des., pp 211–214
Eshelman L, Schaffer J (1993) Real-coded genetic algorithms and interval schemata. In: Foundation of Genetic Algorithm-2. Morgan Kaufmann
Fogel L (1962) Autonomous automata. Ind Res 4:14–19
Fogel DB (2000) Evolutionary computation. Toward a new philosophy of machine intelligence. IEEE Press, Piscataway
Fogel L, Owens A, Walsh M (1966) Artificial intelligence through simulated evolution. Wiley, New York
Friedberg R (1958) A learning machine: Part I. IBM J 2:2–13
Garcia S, Molina D, Lozano M, Herrera F (2009) A study on the use of non-parametric tests for analyzing the evolutionary algorithms’ behaviour: a case study on the CEC’2005 special session on real parameter optimization. J Heuristics 15(6):617–644
García-Martínez C, Rodriguez FJ, Lozano M (2012) Arbitrary function optimisation with metaheuristics. No free lunch and real-world problems. Soft Comput 16(12):2115–2133
Garden RW, Engelbrecht AP (2014) Analysis and classification of optimisation benchmark functions and benchmark suites. In: IEEE Congress on Evolutionary Computation (CEC’2014), pp 1664–1669
Geem ZW, Kim JH, Loganathan G (2001) A new heuristic optimization algorithm: harmony search. Simulation 76:60–68
Glover F (1977) Heuristics for integer programming using surrogate constraints. Decis Sci 8:156–166
Goldberg D (1989) Genetic algorithms in search, optimization, and machine learning. Addison-Wesley, Reading
Guo SM, Yang CC (2015) Enhancing differential evolution utilizing eigenvector-based crossover operator. IEEE Trans Evol Comput 19(1):31–49
Guo SM, Tsai JSH, Yang CC, Hsu PH (2015) A self-optimization approach for L-SHADE incorporated with eigenvector-based crossover and successful-parent-selecting framework on CEC 2015 benchmark set. In: IEEE Congress on Evolutionary Computation (CEC’2015), pp 1003–1010
Hansen N (2005) Compilation of results on the CEC benchmark function set. Tech. rep., Institute of Computational Science, ETH Zurich, Switzerland
Hansen N (2009) Benchmarking a BI-Population CMA-ES on the BBOB-2009 Function Testbed. In: Genetic and Evolutionary Computation Conference (GECCO’09), pp 2389–2396
Hansen N, Auger A, Mersmann O, Tuv̀ar T, Brockhoff D (2016) COCO: a platform for comparing continuous optimizers in a black-box setting. In: ArXiv e-prints, arXiv:1603.08785
Herrera F, Lozano M, Sánchez A (2003) A taxonomy for the crossover operator for real-coded genetic algorithms. An experimental study. Int J Intell Syst 18(3):309–338
Ho SY, Lin HS, Liauh WH, Ho SJ (2008) OPSO: orthogonal particle swarm optimization and its application to task assignment problems. IEEE Trans Syst Man CybernPart A 38(2):288–298
Holland J (1962) Outline for a logical theory of adaptive systems. J Assoc Comput Mach 3:297–314
Holland J (1975) Adaptation in natural and artificial systems. University of Michigan Press, Ann Arbor
Jamil M, Yang XS (2013) A literature survey of benchmark functions for global optimisation problems. Int J Math Model Numer Optim 4(2):150–194
Jamil M, Yang X-S, Zepernick H-JD (2013) Test functions for global optimization: a comprehensive survey. In: Swarm Intelligence and Bio-Inspired Computation, pp 193–222
Janson S, Middendorf M (2005) A hierarchical particle swarm optimizer and its adaptive variant. IEEE Trans Syst Man Cybern Part B Cybern 35(6):1272–1282
Karaboga D, Basturk B (2007) A powerful and efficient algorithm for numerical function optimization: artificial bee colony (ABC) algorithm. J Global Optim 39:459–471
Keenedy J (1999) Small worlds and mega-minds: effects of neighborhood topology on particle swarm performance. In: IEEE Congress on Evolutionary Computation (CEC’99), vol 3, pp 1931–1938
Kennedy J, Eberhart R (1995) Particle swarm optimization. IEEE Conf Neural Netw 4:1942–1947
Kennedy J, Mendes R (2002) Population structure and particle swarm performance. In: IEEE Congress on Evolutionary Computation (CEC’02), pp 1671–1676
KrishnaKumar K, Narayanaswamy S, Garg S (1995) Solving large parameter optimization problems using a genetic algorithm with stochastic coding. In: Genetic Algorithms in Engineering and Computer Science, pp 287–303. Wiley
Lee C, Yao X (2004) Evolutionary programming using mutations based on the Lévy probability distribution. IEEE Trans Evol Comput 8(1):1–13
Leung YW, Wang Y (2001) An orthogonal genetic algorithm with quantization for global numerical optimization. IEEE Trans Evol Comput 5(1):41–53
Li Z (2015) Genetic algorithm that considers scattering for THz quantitative analysis. IEEE Trans Terahertz Sci Technol 5(6):1062–1067
Liang J, Suganthan P (2005) Dynamic multi-swarm particle swarm optimizer. In: Swarm Intell. Symposium, pp 124–129
Liang JJ, Suganthan PN, Deb K (2005) Novel composition test functions for numerical global optimization. In: IEEE Swarm Intelligence Symposium, pp 68–75
Liang J, Qin A, Suganthan P, Baskar S (2006) Comprehensive learning particle swarm optimizer for global optimization of multimodal functions. IEEE Trans Evol Comput 10(3):281–295
Liao T, Molina D, de Oca M, Stützle T (2014) A note on bound constraints handling for the IEEE CEC’05 benchmark function suite. Evol Comput 22(2):351–359
Liao T, Molina D, Sttzle T (2015) Performance evaluation of automatically tuned continuous optimizers on different benchmark sets. Soft Comput J 27:490–503
Liu J, Lampinen (2005) A fuzzy adaptive differential evolution algorithm. Soft Comput 9(6):448–462
Mendes R, Kennedy J, Neves J (2004) The fully informed particle swarm: simpler, maybe better. IEEE Trans Evol Comput 8(3):204–210
Omidvar MN, Li X, Tang K (2015) Designing benchmark problems for large-scale continuous optimization. Inf Sci 2015:419–436
Omran M, Salman A, Engelbrecht A (2005) Self-adaptive differential evolution. In: Computational Intelligence and Security (LNCS 3801), pp 192–199. Springer
Ong YS, Keane A (2004) Meta-lamarckian learning in memetic algorithms. IEEE Trans Evol Comput 8(2):99–110
Parsopoulos K, Vrahatis M (2004) UPSO A unified particle swarm optimization scheme. In: Lecture Series on Computational Sciences, pp 868–873
Particle Swarm Central (2007) http://www.particleswarm.info/Programs.html#Standard_PSO_2007
Passino K (2002) Biomimicry of bacterial foraging for distributed optimization and control. IEEE Control Syst Mag 22:52–67
Peram T, Veeramachaneni K, Mohan C (2003) Fitness-distance-ration based particle swarm optimization. In: Swarm Intelligence Symposium, pp 174–181
Piotrowski AP (2015) Regarding the rankings of optimization heuristics based on artificially-constructed benchmark functions. Inf Sci 297:191–201
Pošic P, Kubalík J (2012) Experimental comparison of six population-based algorithms for continuous black box optimization. Evol Comput 20(4):483–508
Pošic P, Huyer W, Pál L (2012) A comparison of global search algorithms for continuous black box optimization. Evol Comput 20(4):509–541
Qin A, Huang V, Suganthan P (2009) Differential evolution algorithm with strategy adaptation for global numerical optimization. IEEE Trans Evol Comput 13(2):398–417
Qu BY, Liang JJ, Wang ZY, Chen Q, Suganthan PN (2016) Novel benchmark functions for continuous multimodal optimization with comparative results. Swarm Evol Comput 26:23–34
Ratnaweera A, Halgamuge S, Watson H (2004) Self-organizing hierarchical particle swarm optimizer with time-varying acceleration coefficients. IEEE Trans Evol Comput 8(3):240–255
Rechenberg I (1965) Cybernetic solution path of an experimental problem. Royal Aircraft Establishment Translation, 1122
Rönkkönen J, Li X, Kyrki V, Lampinen J (2011) A framework for generating tunable test functions for multimodal optimization. Soft Comput 15(9):1689–1706
Schwefel HP (1968) Experimemelle Optimierung einer Zweiphasend. Tech. Rep. 35, Project MHD_Staustrahirohr. 11.034/68
Schwefel HP (1975) Evolutionsstrategie und numerische Optimierung. Ph.D. thesis, Technische Universität Berlin
Schwefel H-P (1981) Numerical optimization of computer models. Wiley, Chichester
Shi Y, Eberhart R (1998a) A modified particle swarm optimizer. In: IEEE Congress on Evolutionary Computation (CEC’98), pp 69–73
Shi Y, Eberhart R (1998b) Parameter selection in particle swarm optimization. In: International Conference on Evolutionary Programming (LNCS 1447), pp 591–600
Shi Y, Eberhart R (1999) Empirical study of particle swarm optimization. In: IEEE Congress on Evolutionary Computation (CEC’99), pp 1945–1950
Snyman J (1982) A new and dynamic method for unconstrained minimization. Appl Math Model 6:449–462
Sörensen K (2015) Metaheuristics—the metaphor exposed. Int Trans Oper Res 22:3–18
Srinivas M, Patnaik L (1994) Adaptive probabilities of crossover and mutation in genetic algorithms. IEEE Trans Syst Man Cybern 24(4):656–667
Storn R, Price K (1997) Differential Evolution. A simple and efficient heuristic for global optimization over continuous spaces. J Global Optim 11:341–359
Suganthan PN, Hansen N, Liang JJ, Deb K, Chen YP, Auger A, Tiwari S (2005) Problem definitions and evaluation criteria for the CEC 2005 special session on real parameter optimization. Tech. report, Nanyang Technological University
Tanabe R, Fukunaga A (2013) Success-history based parameter adaptation for differential evolution. In: IEEE Congress on Evolutionary Computation (CEC’13), pp 71–78
Tanabe R, Fukunaga A (2014) Improving the search performance of SHADE using linear population size reduction. In: IEEE Congress on Evolutionary Computation (CEC’14), pp 1658–1665
Trelea I (2003) The particle swarm optimization algorithm: convergence analysis and parameter selection. Inf Process Lett 85(6):317–325
van den Bergh F, Engelbrecht A (2004) A cooperative approach to particle swarm optimization. IEEE Trans Evol Comput 8(3):225–239
Weyland D (2010) A rigorous analysis of the harmony search algorithm: How the research community can be misled by a novel methodology. Int J Appl Metaheuristic Comput 1(2):50–60
Wilcoxon F (1945) Individual comparisons by ranking methods. Biometrics 1:80–83
Wolpert D, Macready W (1997) No free lunch theorems for optimization. IEEE Trans Evol Comput 1(1):67–82
Xiong N, Molina D, Ortiz ML, Herrera F (2015) A walk into metaheuristics for engineering optimization: principles, methods and recent trends. Int J Comput Intell Syst 8(4):606–636
Yang Z, He J, Yao X (2007) Making a difference to differential evolution. In: Advances Metaheuristics for Hard Optimization, pp 397–414. Springer
Yao X, Liu Y, Lin G (1999) Evolutionary programming made faster. IEEE Trans Evol Comput 3(2):82–102
Zaharie D (2003) Control of population diversity and adaptation in differential evolution algorithms. In: Mendel 9th Int. Conf. Soft Computing, pp 41–46
Zhan ZH, Zhang J, Li Y, Shi YH (2011) Orthogonal learning particle swarm optimization. IEEE Trans Evol Comput 15(6):832–847
Zhang J, Sanderson A (2009) JADE: adaptive differential evolution with optional external archive. IEEE Trans Evol Comput 13(5):945–958
Zhang Q, Sun J, Tsang E, Ford J (2004) Hybrid estimation of distribution algorithm for global optimization. Eng Comput 21(1):91–107
Zheng YL, Ma LH, Zhang LY, Qian JX (2003a) Empirical study of particle swarm optimizer with an increasing inertia weight. In: IEEE Congress on Evolutionary Computation (CEC’03), pp 221–226
Zheng YL, Ma LH, Zhang LY, Qian JX (2003b) On the convergence analysis and parameter selection in particle swarm optimization. In: IEEE International Conference on Machine Learning and Cybernetics, pp 1802–1807
Acknowledgements
This work was supported by the Research Projects TIN2012-37930-C02-01, TIN2013-47210-P and P12-TIC-2958. P.D. Gutiérrez holds an FPI scholarship from the Spanish Ministry of Economy and Competitiveness (BES-2012-060450).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Communicated by C. M. Vide and A. H. Dediu.
Rights and permissions
About this article
Cite this article
García-Martínez, C., Gutiérrez, P.D., Molina, D. et al. Since CEC 2005 competition on real-parameter optimisation: a decade of research, progress and comparative analysis’s weakness. Soft Comput 21, 5573–5583 (2017). https://doi.org/10.1007/s00500-016-2471-9
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-016-2471-9