Skip to main content
SpringerLink
Log in

Arbitrary function optimisation with metaheuristics

No free lunch and real-world problems

  • Original Paper
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

No free lunch theorems for optimisation suggest that empirical studies on benchmarking problems are pointless, or even cast negative doubts, when algorithms are being applied to other problems not clearly related to the previous ones. Roughly speaking, reported empirical results are not just the result of algorithms’ performances, but the benchmark used therein as well; and consequently, recommending one algorithm over another for solving a new problem might be always disputable. In this work, we propose an empirical framework, arbitrary function optimisation framework, that allows researchers to formulate conclusions independent of the benchmark problems that were actually addressed, as long as the context of the problem class is mentioned. Experiments on sufficiently general scenarios are reported with the aim of assessing this independence. Additionally, this article presents, to the best of our knowledge, the first thorough empirical study on the no free lunch theorems, which is possible thanks to the application of the proposed methodology, and whose main result is that no free lunch theorems unlikely hold on the set of binary real-world problems. In particular, it is shown that exploiting reasonable heuristics becomes more beneficial than random search when dealing with binary real-world applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  • Auger A, Teytaud O (2007) Continuous lunches are free. In: Proceedings of the genetic and evolutionary computation conference, ACM Press, New York, pp 916–922

  • Auger A, Teytaud O (2008) Continuous lunches are free plus the design of optimal optimization algorithms. Algorithmica 57(1):121–146. doi:10.1007/s00453-008-9244-5

    Google Scholar 

  • Barr RS, Golden BL, Kelly JP, Resende MG, Stewart WRJ (1995) Designing and reporting on computational experiments with heuristic methods. J Heuristics 1:9–32

    Google Scholar 

  • Beasley J (1998) Heuristic algorithms for the unconstrained binary quadratic programming problem. Techical report, The Management School, Imperial College

  • Blancke S, Boudry M, Braeckman J (2010) Simulation of biological evolution under attack, but not really: a response to Meester. Biol Philos 26(1):113–118. doi:10.1007/s10539-009-9192-8

    Google Scholar 

  • Blum C, Puchinger J, Raidl GR, Roli A (2011) Hybrid metaheuristics in combinatorial optimization: a survey. Appl Soft Comput 11(6):4135–4151

    Article  Google Scholar 

  • Chen DS, Batson R, Dang Y (2010) Applied integer programming: modeling and solution. Wiley, Chichester

  • Corne DW, Knowles JD (2003) No free lunch and free leftovers theorems for multiobjective optimisation problems. Evol Multi Crit Optim LNCS 2632:327–341. doi:10.1007/3-540-36970-8_23

  • Dembski WA, Marks II RJ (2009) Conservation of information in aearch: measuring the cost of auccess. IEEE Trans Syst Man Cybernet Part A 39(5):1051–1061. doi:10.1109/TSMCA.2009.2025027

    Google Scholar 

  • Dembski WA, Marks II RJ (2010) The Search for a aearch: measuring the information cost of higher level search. J Advan Comput Intell Intell Informatics 14(5):475–486

    Google Scholar 

  • Derrac J, García S, Molina D, Herrera F (2011) A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm Evol Comput 1(1):3–18

    Article  Google Scholar 

  • Droste S, Jansen T, Wegener I (1999) Perhaps not a free lunch but at least a free appetizer. In: Proceedings of the genetic and evolutionary computation conference (GECCO’99), Morgan Kaufmann, pp 833–839

  • Droste S, Jansen T, Wegener I (2002) Optimization with randomized search heuristics—the (A)NFL theorem, realistic scenarios, and difficult functions. Theor Comput Sci 287(1):131–144

    Article  MathSciNet  MATH  Google Scholar 

  • Eshelman L, Schaffer J (1991) Preventing premature convergence in genetic algorithms by preventing incest. In: Belew R, Booker L (eds) International conference on genetic algorithms. Morgan Kaufmann, San Mateo, pp 115–122

  • Forrest S, Mitchell M (1993) Relative building block fitness and the building block hypothesis. In: Whitley L (ed) Foundations of genetic algorithms 2. Morgan Kaufmann, San Mateo, pp 109–126

  • Friedman M (1940) A comparison of alternative tests of significance for the problem of m rankings. Ann Math Stat 11(1):86–92

    Article  MATH  Google Scholar 

  • Garcia S, Fernández A, Luengo J, Herrera F (2009a) A study of statistical techniques and performance measures for genetics-based machine learning: accuracy and interpretability. Soft Comput 13(10):959–977

    Article  Google Scholar 

  • Garcia S, Molina D, Lozano M, Herrera F (2009b) A Study on the use of non-parametric tests for analyzing the evolutionary algorithms’ behaviour: a case study on the CEC’2005 special session on real parameter optimization. J Heuristics 15(6):617–644

    Article  MATH  Google Scholar 

  • García-Martínez C, Lozano M (2010) Evaluating a local genetic algorithm as context-independent local search operator for metaheuristics. Soft Comput 14(10):1117–1139

    Article  Google Scholar 

  • García-Martínez C, Lozano M, Rodriguez FJ (2011a) Arbitrary function optimization. No free lunch and real-world problems. http://www.uco.es/grupos/kdis/kdiswiki/index.php/AFO-NFL

  • García-Martínez C, Rodríguez-Díaz FJ, Lozano M (2011b) Role differentiation and malleable mating for differential evolution: an analysis on large-scale optimisation. Soft Comput 15(11):2109–2126. doi:10.1007/ s00500-010-0641-8

    Google Scholar 

  • García-Martínez C, Lozano M, Rodríguez-Dìaz FJ (2012) A simulated annealing method based on a specialised evolutionary algorithm. Appl Soft Comput 12(2):573–588

    Article  Google Scholar 

  • Glover F, Laguna M (1997) Tabu search. Kluwer Academic Publishers, Norwell

  • Goldberg D, Korb B, Deb K (1989) Messy genetic algorithms: motivation, analysis, and first results. Complex Syst 3:493–530

    MathSciNet  MATH  Google Scholar 

  • Gortázar F, Duarte A, Laguna M, Martí R (2010) Black box scatter search for general classes of binary optimization problems. Comput Operat Res 37(11):1977–1986. doi:10.1016/j.cor.2010.01.013

    Google Scholar 

  • Hansen N (2005) Compilation of results on the CEC benchmark function set. Technical report, Institute of Computational Science, ETH Zurich, Switzerland

  • Herrera F, Lozano M, Verdegay J (1998) Tackling realcoded genetic algorithms: operators and tools for behavioral analysis. Artif Intell Rev 12(4):265–319

    Article  MATH  Google Scholar 

  • Holm S (1979) A simple sequentially rejective multiple test procedure. Scand J Stat 6:65–70

    MathSciNet  MATH  Google Scholar 

  • Hooker JN (1995) Testing heuristics: We have it all wrong. J Heuristics 1(1):33–42. doi:10.1007/ BF02430364

    Google Scholar 

  • Igel C, Toussaint M (2003) On classes of functions for which no free lunch results hold. Inform Process Lett 86(6):317–321

    Article  MathSciNet  MATH  Google Scholar 

  • Igel C, Toussaint M (2004) A no-free-lunch theorem for non-uniform distributions of target functions. J Math Modell Algorithms 3(4):313–322

    Article  MathSciNet  MATH  Google Scholar 

  • Iman R, Davenport J (1980) Approximations of the critical region of the Friedman statistic. Commun Stat 9:571–595

    Google Scholar 

  • Jiang P, Chen Y (2010) Free lunches on the discrete Lipschitz class. Theor Comput Sci 412(17):1614–1628. doi:10.1016/j.tcs.2010.12.028

    Google Scholar 

  • Jünger M, Liebling T, Naddef D, Nemhauser G, Pulleyblank W et al (eds) (2009) 50 Years of integer programming 1958–2008: from the early years to the state-of-the-art. Springer, Berlin

  • Karp R (1972) Reducibility among combinatorial problems. In: Miller R, Thatcher J (eds) Complexity of computer computations. Plenum Press, New York, pp 85–103

  • Kauffman S (1989) Adaptation on rugged fitness landscapes. Lect Sci Complex 1:527–618

    Google Scholar 

  • Kirkpatrick S, Gelatt Jr C, Vecchi M (1983) Optimization by simulated annealing. Science 220(4598):671–680

    Article  MathSciNet  MATH  Google Scholar 

  • Koehler GJ (2007) Conditions that obviate the no-free- lunch theorems for optimization. INFORMS J Comput 19(2):273–279. doi:10.1287/ijoc.1060.0194

  • Laplace PS (1814) Essai philosophique sur les probabilités. Technical report, Paris, Courcier

  • Lozano M, García-Martínez C (2010) Hybrid metaheuristics with evolutionary algorithms specializing in intensification and diversification: overview and progress report. Comput Operat Res 37:481–497

    Article  MATH  Google Scholar 

  • Lozano M, Herrera F, Molina D (eds) (2011) Scalability of evolutionary algorithms and other metaheuristics for large scale continuous optimization problems, vol 15. Soft Computing

  • Marshall JAR, Hinton TG (2010) Beyond no free lunch: realistic algorithms for arbitrary problem classes. In: IEEE Congr Evol Comput 1:18–23

    Google Scholar 

  • Pelikan M, Goldberg D, Cantú-Paz E (2000) Linkage problem, distribution estimation, and bayesian networks. Evol Comput 8(3):311–340

    Article  Google Scholar 

  • Rodriguez FJ, García-Martínez C, Lozano M (2012) Hybrid metaheuristics based on evolutionary algorithms and simulated annealing: taxonomy, comparison, and synergy test. IEEE Trans Evol Comput. doi:10.1109/TEVC.2012.2182773

  • Schaffer J, Eshelman L (1991) On crossover as an evolutionary viable strategy. In: Belew R, Booker L (eds) Proceedings of the international conference on genetic algorithms. Morgan Kaufmann, San Mateo, pp 61–68

  • Schumacher C, Vose MD, Whitley LD (2001) The No Free Lunch and Problem Description Length. In: Proc. of the Genetic and Evolutionary Computation Conference, pp 565–570

  • Service TC (2010) A no free lunch theorem for multiobjective optimization. Inform Process Lett 110(21):917–923. doi:10.1016/j.ipl.2010.07.026

    Google Scholar 

  • Smith K, Hoos H, Stützle T (2003) Iterated robust tabu search for MAX-SAT. In: Carbonell J, Siekmann J (eds) Proceedings of the Canadian society for computational studies of intelligence Conference, vol LNCS 2671. Springer, Berlin, pp 129–144

  • Talbi E (2002) A taxonomy of hybrid metaheuristics. J Heuristics 8(5):541–564

    Article  Google Scholar 

  • Thierens D (2002) Adaptive mutation rate control schemes in genetic algorithms. In: Proceedings of the congress on evolutionary computation, pp 980–985

  • Thierens D (2004) Population-based iterated local search: restricting neighborhood search by crossover. In: Proceedings of the genetic and evolutionary computation conference, vol LNCS 3103. Springer, Berlin, pp 234–245

  • Watson R, Pollack J (1999) Hierarchically consistent test problems for genetic algorithms. Proc Congr Evol Comput 2:1406–1413

    Google Scholar 

  • Whitley D, Rowe J (2008) Focused no free lunch theorems. In: Proceedings of the genetic and evolutionary computation conference. ACM Press, New York, pp 811–818. doi:10.1145/1389095.1389254

  • Whitley D, Watson JP (2005) Complexity theory and the no free lunch theorem. Search Methodologies, Springer, Berlin, pp 317–339. doi:10.1007/0-387-28356-0_11

  • Whitley D, Rana S, Dzubera J, Mathias E (1996) Evaluating evolutionary algorithms. Artif Intell 85:245–276

    Article  Google Scholar 

  • Wolpert D, Macready W (1997) No free lunch theorems for optimization. IEEE Trans Evol Comput 1(1):67–82

    Article  Google Scholar 

  • Zar J (1999) Biostatistical analysis. Prentice Hall, Upper Saddle River

Download references

Acknowledgments

Beliefs usually need to be critically analysed before becoming real knowledge. Being loyal to this idea, the authors would like to express that this study would not have been initiated without the fact that, their journal submissions proposing new approaches, and analysed on many different kinds of problems, were sometimes rejected on the claim that '’according to the NFL, if your proposal wins, then it loses on the rest of problems that have not been analysed”. Therefore and being honest with ourselves, this study, we are really glad of having developed, is in part thanks to the corresponding reviewers and deciding editors’ comments that put us on the way.

Author information

Authors and Affiliations

  1. Department of Computing and Numerical Analysis, University of Córdoba, Córdoba, 14071, Spain

    Carlos García-Martínez

  2. Department of Computer Sciences and Artificial Intelligence CITIC-UGR, University of Granada, Granada, 18071, Spain

    Francisco J. Rodriguez & Manuel Lozano

Authors
  1. Carlos García-Martínez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Francisco J. Rodriguez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Manuel Lozano
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Carlos García-Martínez.

Additional information

Research Projects TIN2011-24124 and P08-TIC-4173.

Rights and permissions

Reprints and permissions

About this article

Cite this article

García-Martínez, C., Rodriguez, F.J. & Lozano, M. Arbitrary function optimisation with metaheuristics. Soft Comput 16, 2115–2133 (2012). https://doi.org/10.1007/s00500-012-0881-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-012-0881-x

Keywords

Search

Navigation