Abstract
Evolution strategies belong to the best performing modern natural computing methods for continuous optimization. This paper takes a new look at the covariance matrix adaptation, a mechanism which is central to the algorithm. The adaptation focusses strongly on the sample covariance. However, as known from modern statistics, this estimate may be of poor quality if certain conditions are not fulfilled. Unfortunately, this is often the case in practice. This paper compares the established methods for the covariance correction in evolution strategies with the approaches in modern statistics. Furthermore, it introduces and evaluates new covariance correction schemes.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Audet, C.: A survey on direct search methods for blackbox optimization and their applications. In: Pardalos, P.M., Rassias, T.M. (eds.) Mathematics without Boundaries: Surveys in Interdisciplinary Research. Springer, New York (2013). Also Les Cahiers du GERAD G-2012-53 (2012)
Bai, Z.D., Silverstein, J.W.: No eigenvalues outside the support of the limiting spectral distribution of large-dimensional sample covariance matrices. Ann. Probab. 26(1), 316–345 (1998)
Beyer, H.G., Meyer-Nieberg, S.: Self-adaptation of evolution strategies under noisy fitness evaluations. Genet. Program. Evolvable Mach. 7(4), 295–328 (2006)
Beyer, H.G., Schwefel, H.P.: Evolution strategies: a comprehensive introduction. Nat. Comput. 1(1), 3–52 (2002)
Beyer, H.-G., Sendhoff, B.: Covariance matrix adaptation revisited – the CMSA evolution strategy –. In: Rudolph, G., Jansen, T., Lucas, S., Poloni, C., Beume, N. (eds.) PPSN 2008. LNCS, vol. 5199, pp. 123–132. Springer, Heidelberg (2008)
Chen, X., Wang, Z., McKeown, M.: Shrinkage-to-tapering estimation of large covariance matrices. IEEE Trans. Signal Process 60(11), 5640–5656 (2012)
Chen, Y., Wiesel, A., Eldar, Y.C., Hero, A.O.: Shrinkage algorithms for MMSE covariance estimation. IEEE Trans. Signal Process. 58(10), 5016–5029 (2010)
Sarkar, M., Theuwissen, A.: Introduction. In: Sarkar, M., Theuwissen, A. (eds.) A Biologically Inspired CMOS Image Sensor. SCI, vol. 461, pp. 1–14. Springer, Heidelberg (2013)
Dong, W., Yao, X.: Covariance matrix repairing in gaussian based EDAs. In: IEEE Congress on, Evolutionary Computation, CEC 2007, pp. 415–422 (2007)
Eiben, A.E., Smith, J.E.: Introduction to Evolutionary Computing. Natural Computing Series. Springer, Berlin (2003)
Finck, S., Hansen, N., Ros, R., Auger, A.: Real-parameter black-box optimization benchmarking 2010: presentation of the noiseless functions. Technical report, Institute National de Recherche en Informatique et Automatique (2010) 2009/22
Hansen, N.: The CMA evolution strategy: a comparing review. In: Lozano J., et al. (eds.) Towards a new evolutionary computation. Advances in estimation of distribution algorithms, pp. 75–102. Springer (2006)
Hansen, N., Auger, A., Finck, S., Ros, R.: Real-parameter black-box optimization benchmarking 2012: experimental setup. Technical report, INRIA (2012). http://coco.gforge.inria.fr/bbob2012-downloads
Hansen, N., Ostermeier, A.: Completely derandomized self-adaptation in evolution strategies. Evolutionary Computation 9(2), 159–195 (2001)
Hansen, N., Auger, A., Ros, R., Finck, S., Pošík, P.: Comparing results of 31 algorithms from the black-box optimization benchmarking BBOB-2009. In: Proceedings of the 12th Annual Conference Companion on Genetic and Evolutionary Computation, GECCO 2010, pp. 1689–1696. ACM, New York (2010). http://doi.acm.org/10.1145/1830761.1830790
Huber, P.J.: Robust Statistics. Wiley, New York (1981)
Ledoit, O., Wolf, M.: A well-conditioned estimator for large dimensional covariance matrices. J. Multivar. Anal. Arch. 88(2), 265–411 (2004)
Ledoit, O., Wolf, M.: Non-linear shrinkage estimation of large dimensional covariance matrices. Ann. Stat. 40(2), 1024–1060 (2012)
Meyer-Nieberg, S., Beyer, H.G.: Self-adaptation in evolutionary algorithms. In: Lobo, F., Lima, C., Michalewicz, Z. (eds.) Parameter Setting in Evolutionary Algorithms, pp. 47–76. Springer Verlag, Heidelberg (2007)
Rechenberg, I.: Evolutionsstrategie: Optimierung technischer Systeme nach Prinzipien der biologischen Evolution. Frommann-Holzboog Verlag, Stuttgart (1973)
Schäffer, J., Strimmer, K.: A shrinkage approach to large-scale covariance matrix estimation and implications for functional genomics. Stat. Appl. Genet. Mol. Biol. 4(1), Article 32 (2005)
Schwefel, H.P.: Numerical Optimization of Computer Models. Wiley, Chichester (1981)
Stein, C.: Inadmissibility of the usual estimator for the mean of a multivariate distribution. In: Proceedings 3rd Berkeley Symposium Mathematical Statistics and Probability, vol. 1, pp. 197–206. Berkeley, CA (1956)
Stein, C.: Estimation of a covariance matrix. In: Rietz Lecture, 39th Annual Meeting. IMS, Atlanta, GA (1975)
Thomaz, C.E., Gillies, D., Feitosa, R.: A new covariance estimate for bayesian classifiers in biometric recognition. IEEE Trans. Circuits Syst. Video Technol. 14(2), 214–223 (2004)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Meyer-Nieberg, S., Kropat, E. (2015). A New Look at the Covariance Matrix Estimation in Evolution Strategies. In: Pinson, E., Valente, F., Vitoriano, B. (eds) Operations Research and Enterprise Systems. ICORES 2014. Communications in Computer and Information Science, vol 509. Springer, Cham. https://doi.org/10.1007/978-3-319-17509-6_11
Download citation
DOI: https://doi.org/10.1007/978-3-319-17509-6_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-17508-9
Online ISBN: 978-3-319-17509-6
eBook Packages: Computer ScienceComputer Science (R0)