Skip to main content
SpringerLink
Log in

Black-Box Search by Unbiased Variation

  • Published:
Algorithmica Aims and scope Submit manuscript

Abstract

The complexity theory for black-box algorithms, introduced by Droste, Jansen, and Wegener (Theory Comput. Syst. 39:525–544, 2006), describes common limits on the efficiency of a broad class of randomised search heuristics. There is an obvious trade-off between the generality of the black-box model and the strength of the bounds that can be proven in such a model. In particular, the original black-box model provides for well-known benchmark problems relatively small lower bounds, which seem unrealistic in certain cases and are typically not met by popular search heuristics.

In this paper, we introduce a more restricted black-box model for optimisation of pseudo-Boolean functions which we claim captures the working principles of many randomised search heuristics including simulated annealing, evolutionary algorithms, randomised local search, and others. The key concept worked out is an unbiased variation operator. Considering this class of algorithms, significantly better lower bounds on the black-box complexity are proved, amongst them an Ω(nlogn) bound for functions with unique optimum. Moreover, a simple unimodal function and plateau functions are considered. We show that a simple (1+1) EA is able to match the runtime bounds in several cases.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Algorithm 1

Similar content being viewed by others

References

  1. Anil, G., Wiegand, R.P.: Black-box search by elimination of fitness functions. In: Proceedings of the 10th International Workshop on Foundations of Genetic Algorithms (FOGA’09), pp. 67–78. ACM Press, New York (2009)

    Google Scholar 

  2. Böttcher, S., Doerr, B., Neumann, F.: Optimal fixed and adaptive mutation rates for the leadingones problem. In: Proceedings of the 11th international conference on Parallel Problem Solving from Nature (PPSN’10), pp. 1–10. Springer, Berlin (2010)

    Chapter  Google Scholar 

  3. Cathabard, S., Lehre, P.K., Yao, X.: Non-uniform mutation rates for problems with unknown solution lengths. In: Proceedings of the 11th International Workshop on Foundations of Genetic Algorithms (FOGA’11), pp. 173–180. ACM, New York (2011)

    Google Scholar 

  4. Chvátal, V.: The tail of the hypergeometric distribution. Discrete Math. 25(3), 285–287 (1979)

    Article  MathSciNet  MATH  Google Scholar 

  5. Doerr, B., Fouz, M., Witt, C.: Quasirandom evolutionary algorithms. In: Proceedings of the 12th Annual Conference on Genetic and Evolutionary Computation (GECCO’10), pp. 1457–1464. ACM, New York (2010)

    Chapter  Google Scholar 

  6. Dorigo, M., Stützle, T.: Ant Colony Optimization. MIT Press, Cambridge (2004)

    Book  MATH  Google Scholar 

  7. Droste, S., Jansen, T., Wegener, I.: On the analysis of the (1+1) Evolutionary Algorithm. Theor. Comput. Sci. 276, 51–81 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  8. Droste, S., Jansen, T., Wegener, I.: Upper and lower bounds for randomized search heuristics in black-box optimization. Theory Comput. Syst. 39(4), 525–544 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  9. Droste, S., Wiesmann, D.: Metric based evolutionary algorithms. In: Proceedings of Genetic Programming, European Conference. LNCS, vol. 1802, pp. 29–43. Springer, Berlin (2000)

    Google Scholar 

  10. Fournier, H., Teytaud, O.: Lower bounds for comparison based evolution strategies using VC-dimension and sign patterns. Algorithmica 59, 387–408 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  11. Giel, O., Wegener, I.: Evolutionary algorithms and the maximum matching problem. In: Proceedings of the 20th Annual Symposium on Theoretical Aspects of Computer Science (STACS’03), pp. 415–426 (2003)

    Google Scholar 

  12. Hajek, B.: Hitting-time and occupation-time bounds implied by drift analysis with applications. Adv. Appl. Probab. 13(3), 502–525 (1982)

    Article  Google Scholar 

  13. He, J., Yao, X.: A study of drift analysis for estimating computation time of evolutionary algorithms. Natural Computing, 3(1) (2004)

  14. Jägersküpper, J.: Algorithmic analysis of a basic evolutionary algorithm for continuous optimization. Theor. Comput. Sci. 39(3), 329–347 (2007)

    Article  Google Scholar 

  15. Jansen, T., De Jong, K.A., Wegener, I.: On the choice of the offspring population size in evolutionary algorithms. Evol. Comput. 13(4), 413–440 (2005)

    Article  Google Scholar 

  16. Jansen, T., Sudholt, D.: Analysis of an asymmetric mutation operator. Evol. Comput. 18, 1–26 (2010)

    Article  Google Scholar 

  17. Johnson, D.S., Papadimitriou, C.H., Yannakakis, M.: How easy is local search? J. Comput. Syst. Sci. 37(1), 79–100 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  18. Kennedy, J., Eberhart, R.C.: Swarm Intelligence. Morgan Kaufmann, San Mateo (2001)

    Google Scholar 

  19. Kirkpatrick, S., Gelatt, C.D., Vecchi, M.P.: Optimization by simulated annealing. Science 220(4598), 671–680 (1983)

    Article  MathSciNet  MATH  Google Scholar 

  20. Larrañaga, P., Lozano, J.A.: Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation. Kluwer Academic, Dordrecht (2002)

    MATH  Google Scholar 

  21. Lehre, P.K., Witt, C.: Black-box search by unbiased variation. In: Proceedings of the 12th Annual Conference on Genetic and Evolutionary Computation (GECCO’10), pp. 1441–1448. ACM, New York (2010)

    Chapter  Google Scholar 

  22. Motwani, R., Raghavan, P.: Randomized Algorithms. Cambridge University Press, Cambridge (1995)

    MATH  Google Scholar 

  23. Neumann, F., Wegener, I.: Randomized local search, evolutionary algorithms, and the minimum spanning tree problem. Theor. Comput. Sci. 378(1), 32–40 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  24. Neumann, F., Witt, C.: Ant colony optimization and the minimum spanning tree problem. In: Proceedings of Learning and Intelligent Optimization (LION’08), pp. 153–166 (2008)

    Chapter  Google Scholar 

  25. Neumann, F., Witt, C.: Bioinspired Computation in Combinatorial Optimization: Algorithms and Their Computational Complexity, 1st edn. Springer, New York (2010)

    Book  MATH  Google Scholar 

  26. Oliveto, P.S., Lehre, P.K., Neumann, F.: Theoretical analysis of rank-based mutation—combining exploration and exploitation. In: Proceedings of the 10th IEEE Congress on Evolutionary Computation (CEC’09), pp. 1455–1462. IEEE, New York (2009)

    Chapter  Google Scholar 

  27. Rowe, J.E., Vose, M.D.: Unbiased black box search algorithms. In: Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation (GECCO’11), pp. 2035–2042. ACM, New York (2011)

    Google Scholar 

  28. Rowe, J.E., Vose, M.D., Wright, A.H.: Neighborhood graphs and symmetric genetic operators. In: Proceedings of the 9th International Workshop on Foundations of Genetic Algorithms (FOGA’07). LNCS, vol. 4436, pp. 110–122 (2007)

    Google Scholar 

  29. Sudholt, D.: General lower bounds for the running time of evolutionary algorithms. In: Proceedings of the 11th International Conference on Parallel Problem Solving from Nature (PPSN’10), pp. 124–133. Springer, Berlin (2010)

    Chapter  Google Scholar 

  30. Teytaud, O., Gelly, S.: General lower bounds for evolutionary algorithms. In: Proceedings of the 9th International Conference on Parallel Problem Solving from Nature (PPSN’06). LNCS, vol. 4193, pp. 21–31. Springer, Berlin (2006)

    Chapter  Google Scholar 

  31. Teytaud, O., Gelly, S., Mary, J.: On the ultimate convergence rates for isotropic algorithms and the best choices among various forms of isotropy. In: Proceedings of the 9th International Conference on Parallel Problem Solving from Nature (PPSN’06). LNCS, vol. 4193, pp. 32–41. Springer, Berlin (2006)

    Chapter  Google Scholar 

  32. Wegener, I.: Methods for the analysis of evolutionary algorithms on pseudo-Boolean functions. In: Sarker, R., Mohammadian, M., Yao, X. (eds.) Evolutionary Optimization, pp. 349–369. Kluwer Academic, Dordrecht (2002)

    Google Scholar 

  33. Wegener, I., Witt, C.: On the optimization of monotone polynomials by simple randomized search heuristics. Comb. Probab. Comput. 14(1), 225–247 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  34. Witt, C.: Runtime analysis of the (μ+1) EA on simple pseudo-Boolean functions. Evol. Comput. 14(1), 65–86 (2006)

    MathSciNet  Google Scholar 

  35. Zarges, C.: Theoretical foundations of artificial immune systems. PhD thesis, Technische Universität Dortmund (2011)

Download references

Author information

Authors and Affiliations

  1. School of Computer Science, University of Nottingham, Jubilee Campus, Wollaton Road, Nottingham, NG8 1BB, UK

    Per Kristian Lehre

  2. Technical University of Denmark, Kgs. Lyngby, Denmark

    Carsten Witt

Authors
  1. Per Kristian Lehre
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carsten Witt
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Per Kristian Lehre.

Additional information

This work is based on earlier work in [21].

P.K. Lehre was supported by EPSRC under grant no. EP/D052785/1, and Deutsche Forschungsgemeinschaft (DFG) under grant no. WI 3552/1-1.

C. Witt was supported by Deutsche Forschungsgemeinschaft (DFG) under grant no. WI 3552/1-1.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lehre, P.K., Witt, C. Black-Box Search by Unbiased Variation. Algorithmica 64, 623–642 (2012). https://doi.org/10.1007/s00453-012-9616-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00453-012-9616-8

Keywords

Search

Navigation