Benjamin Doerr
Franziska Ebel
ABSTRACT
The recently active research area of black-box complexity revealed that for many optimization problems the best possible black-box optimization algorithm is significantly faster than all known evolutionary approaches. While it is not to be expected that a general-purpose heuristic competes with a problem-tailored algorithm, it still makes sense to look for the reasons for this discrepancy.
In this work, we exhibit one possible reason---most optimal black-box algorithms profit also from solutions that are inferior to the previous-best one, whereas evolutionary approaches guided by the "survival of the fittest" paradigm often ignore such solutions. Trying to overcome this shortcoming, we design a simple genetic algorithm that first creates λ offspring from a single parent by mutation with a mutation probability that is k times larger than the usual one. From the best of these offspring (which often is worse than the parent) and the parent itself, we generate further offspring via a uniform crossover operator that takes bits from the winner offspring with probability 1/k only.
A rigorous runtime analysis proves that our new algorithm for suitable parameter choices on the OneMax test function class is asymptotically faster (in terms of the number of fitness evaluations) than what has been shown for μ +, λ EAs. This is the first time that crossover is shown to give an advantage for the OneMax class that is larger than a constant factor. Using a fitness-dependent choice of k and λ, the optimization time can be reduced further to linear in n.
Our experimental analysis on several test function classes shows advantages already for small problem sizes and broad parameter ranges. Also, a simple self-adaptive choice of these parameters gives surprisingly good results.
- P. Afshani, M. Agrawal, B. Doerr, C. Winzen, K. G. Larsen, and K. Mehlhorn. The deterministic and randomized query complexity of a simple guessing game. ECCC, TR12-087, 2012.Google Scholar
- G. Anil and R. P. Wiegand. Black-box search by elimination of fitness functions. In Proc. of FOGA'09, pages 67--78. ACM, 2009. Google ScholarDigital Library
- A. Auger. Benchmarking the (1 + 1) evolution strategy with one-fifth success rule on the BBOB-2009 function testbed. In Proc. of GECCO'09 (Companion), pages 2447--2452. ACM, 2009. Google ScholarDigital Library
- B. Doerr. Analyzing randomized search heuristics: Tools from probability theory. In A. Auger and B. Doerr, editors, Theory of Randomized Search Heuristics, pages 1--20. World Scientific Publishing, 2011.Google ScholarCross Ref
- B. Doerr, D. Johannsen, T. Kötzing, P. K. Lehre, M. Wagner, and C. Winzen. Faster black-box algorithms through higher arity operators. In Proc. of FOGA'11, pages 163--172. ACM, 2011. Google ScholarDigital Library
- B. Doerr, T. Kötzing, J. Lengler, and C. Winzen. Black-box complexities of combinatorial problems. In Proc. of GECCO'11, pages 981--988. ACM, 2011. Google ScholarDigital Library
- B. Doerr and C. Winzen. Towards a complexity theory of randomized search heuristics: Ranking-based black-box complexity. In Proc. of CSR'11, pages 15--28. Springer, 2011. Google ScholarDigital Library
- B. Doerr and C. Winzen. Playing Mastermind with constant-size memory. In Proc. of STACS'12, pages 441--452. Schloss Dagstuhl - Leibniz-Zentrum fuer Informatik, 2012.Google Scholar
- S. Droste, T. Jansen, K. Tinnefeld, and I. Wegener. A new framework for the valuation of algorithms for black-box optimization. In Proc. of FOGA'03, pages 253--270. Morgan Kaufmann, 2003.Google Scholar
- S. Droste, T. Jansen, and I. Wegener. On the analysis of the (1 + 1) evolutionary algorithm. Theor. Comput. Sci., 276:51--81, 2002 Google ScholarDigital Library
- S. Droste, T. Jansen, and I. Wegener. Upper and lower bounds for randomized search heuristics in black-box optimization. Theory Comput. Syst., 39:525--544, 2006. Google ScholarDigital Library
- P. Erd\Hos and A. Rényi. On two problems of information theory. Magyar Tudományos Akadémia Matematikai Kutató Intézet Közleményei, 8:229--243, 1963.Google Scholar
- E. Happ, D. Johannsen, C. Klein, and F. Neumann. Rigorous analyses of fitness-proportional selection for optimizing linear functions. In Proc. of GECCO'08, pages 953--960. ACM, 2008. Google ScholarDigital Library
- J. Jägersküpper and T. Storch. When the plus strategy outperforms the comma strategy and when not. In Proc. of FOCI'07, pages 25--32. IEEE, 2007.Google ScholarCross Ref
- T. Jansen, K. A. D. Jong, and I. Wegener. On the choice of the offspring population size in evolutionary algorithms. Evolutionary Computation, 13:413--440, 2005. Google ScholarDigital Library
- T. Kötzing, D. Sudholt, and M. Theile. How crossover helps in pseudo-Boolean optimization. In Proc. of GECCO'11, pages 989--996. ACM, 2011. Google ScholarDigital Library
- P. K. Lehre and C. Witt. Black-box search by unbiased variation. In Proc. of GECCO'10, pages 1441--1448. ACM, 2010. Google ScholarDigital Library
- P. K. Lehre and C. Witt. Black-box search by unbiased variation. Algorithmica, 64:623--642, 2012.Google ScholarDigital Library
- M. Mitchell, S. Forrest, and J. H. Holland. The royal road for genetic algorithms: Fitness landscapes and GA performance. In Proc. of the First European Conference on Artificial Life, pages 245--254. MIT Press, 1992.Google Scholar
- P. S. Oliveto and C. Witt. On the analysis of the simple genetic algorithm. In Proc. of GECCO'12, pages 1341--1348. ACM, 2012. Google ScholarDigital Library
- J. E. Rowe and D. Sudholt. The choice of the offspring population size in the (1,λ) EA. In Proc. of GECCO'12, pages 1349--1356. ACM, 2012. Google ScholarDigital Library
- T. Storch. On the choice of the parent population size. Evolutionary Computation, 16:557--578, 2008. Google ScholarDigital Library
- D. Sudholt. Crossover speeds up building-block assembly. In Proc. of GECCO'12, pages 689--702. ACM, 2012. Google ScholarDigital Library
- C. Witt. Runtime analysis of the (μ + 1) EA on simple pseudo-Boolean functions. Evolutionary Computation, 14:65--86, 2006. Google ScholarDigital Library
Index Terms
- Lessons from the black-box: fast crossover-based genetic algorithms
Recommendations
OneMax in Black-Box Models with Several Restrictions
GECCO '15: Proceedings of the 2015 Annual Conference on Genetic and Evolutionary ComputationAs in classical runtime analysis the OneMax problem is the most prominent test problem also in black-box complexity theory. It is known that the unrestricted, the memory-restricted, and the ranking-based black-box complexities of this problem are all of ...
Black-box Complexity of Parallel Search with Distributed Populations
FOGA '15: Proceedings of the 2015 ACM Conference on Foundations of Genetic Algorithms XIIIMany metaheuristics such as island models and cellular evolutionary algorithms use a network of distributed populations that communicate search points along a spatial communication topology. The idea is to slow down the spread of information, reducing ...
From black-box complexity to designing new genetic algorithms
Black-box complexity theory recently produced several surprisingly fast black-box optimization algorithms. In this work, we exhibit one possible reason: These black-box algorithms often profit from solutions inferior to the previous-best. In contrast, ...
Comments