Skip to main content
Log in

A multi-recombinative active matrix adaptation evolution strategy for constrained optimization

  • Foundations
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

This paper presents the multi-recombinative constraint active matrix adaptation evolution strategy (Constraint Active-MA-ES). It extends the MA-ES recently introduced by Beyer and Sendhoff in order to handle constrained black-box optimization problems. The active covariance matrix adaptation approach for constraint handling similar to the method proposed by Arnold and Hansen for the \((1+1)\) covariance matrix adaptation evolution strategy is used. As a first step toward constraint handling, active covariance matrix adaptation is incorporated into the MA-ES and evaluated on unconstrained problems (Active-MA-ES). As the second step, constraint handling based on active covariance matrix adaptation for the MA-ES is proposed (Constraint Active-MA-ES). The algorithm has been tested on different test functions and it has been compared to other methods. The experiments show that for cases where directional sampling of the offspring mutations is beneficial, the Active-MA-ES can reach the target faster than the MA-ES. In particular, the Active-MA-ES reaches the final target precision on average by a factor of 1.4 generations (Ellipsoid), a factor of 1.6 generations (Different Powers), and a factor of 2.0 generations (Tablet) faster than the MA-ES. The experiments for the Constraint Active-MA-ES reveal that it achieves 80% of the considered targets with about \(N \times 10^5\) function and constraint evaluations. With this result, it is the best method among the compared approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. We have also empirically investigated the effect of using negative weights for the next generation’s parental candidate solution update (Line 16 in Algorithm 1) as well as for the \(\mathbf {s}\)-update (Line 17 in Algorithm 1). To this end, we have first used an approximation for the optimal weights computed by Arnold (2006) for the infinite-dimensional sphere model and second the weights proposed by Hansen (2016). It turned out that for the dimensions considered, negative weights in the centroid computation and the \(\mathbf {s}\)-update are not beneficial.

  2. Note that by distributing M over the terms in the brackets, the matrix–matrix multiplication is not necessary. It can thus be implemented in a running time of \(O(\lambda N^2)\).

  3. https://www.lri.fr/~hansen/purecmaes.m (with last change on April, 29, 2014). The pure CMA-ES implementation was used instead of the production CMA-ES implementation because the latter code contains ad hoc solutions for improving the performance of the CMA-ES.

  4. Technically, Matlab’s implementation in the function ranksum has been used.

  5. We provide additional experimental results in “Appendix” (Sect. 6.2). There, the results are presented in more detail. In particular, the results are shown for every function separately.

  6. In the BBOB COCO framework, one call to the constraint evaluation function yields the values of all the constraints at the given query point.

  7. \(\varepsilon \)DEag: http://www.ints.info.hiroshima-cu.ac.jp/%7etakahama/download/eDEa-2010.0430.tar.gz, conSaDE: http://web.mysites.ntu.edu.sg/epnsugan/PublicSite/Shared%20Documents/Codes/2006-CEC-Const-SaDE.rar.

  8. Because the bbob-constrained suite has still been under development during creation of this manuscript, we provide a fork in https://github.com/patsp/coco. This makes our results reproducible. Even though it is not the final BBOB COCO constrained framework, it gives a good indication of the algorithm’s performance in comparison with other methods. The changes we made are in our fork in the new branch development-sppa-2. This branch is based on the development branch of https://github.com/numbbo/coco with changes up to and including Dec 10, 2017.

  9. V. Klee and G. J. Minty, “How good is the simplex algorithm?” in Inequalities, O. Shisha, Ed. New York: Academic Press, 1972, vol. III, pp. 159–175.

References

  • Arnold DV (2006) Weighted multirecombination evolution strategies. Theor Comput Sci 361(1):18–37

    Article  MathSciNet  MATH  Google Scholar 

  • Arnold DV (2016) An active-set evolution strategy for optimization with known constraints. In: International conference on parallel problem solving from nature. Springer, pp 192–202

  • Arnold DV (2017) Reconsidering constraint release for active-set evolution strategies. In: Proceedings of the genetic and evolutionary computation conference, GECCO ’17. ACM, New York, NY, USA, pp 665–672. https://doi.org/10.1145/3071178.3071294

  • Arnold DV, Hansen N (2012) A (1 + 1)-CMA-ES for constrained optimisation. In: Proceedings of the 14th annual conference on genetic and evolutionary computation. ACM, pp 297–304

  • Arnold DV, Porter J (2015) Towards an augmented Lagrangian constraint handling approach for the (1 + 1)-ES. In: Proceedings of the 2015 annual conference on genetic and evolutionary computation, GECCO ’15. ACM, New York, NY, USA, pp 249–256. https://doi.org/10.1145/2739480.2754813

  • Atamna A, Auger A, Hansen N (2016) Augmented Lagrangian constraint handling for CMA-ES—case of a single linear constraint. In: International conference on parallel problem solving from nature. Springer, pp 181–191

  • Atamna A, Auger A, Hansen N (2017) Linearly convergent evolution strategies via augmented Lagrangian constraint handling. In: Proceedings of the 14th ACM/SIGEVO conference on foundations of genetic algorithms. ACM, pp 149–161

  • Banks A, Vincent J, Anyakoha C (2007) A review of particle swarm optimization. Part I: background and development. Nat Comput 6(4):467–484

    Article  MathSciNet  MATH  Google Scholar 

  • Banks A, Vincent J, Anyakoha C (2008) A review of particle swarm optimization. Part II: hybridisation, combinatorial, multicriteria and constrained optimization, and indicative applications. Nat Comput 7(1):109–124

    Article  MathSciNet  MATH  Google Scholar 

  • Beyer HG (1989) Ein Evolutionsverfahren zur mathematischen Modellierung stationärer Zustände in dynamischen Systemen. Hochschule für Architektur und Bauwesen, Weimar

    MATH  Google Scholar 

  • Beyer HG, Finck S (2012) On the design of constraint covariance matrix self-adaptation evolution strategies including a cardinality constraint. IEEE Trans Evol Comput 16(4):578–596

    Article  Google Scholar 

  • Beyer HG, Sendhoff B (2017) Simplify your covariance matrix adaptation evolution strategy. IEEE Trans Evol Comput 21(5):746–759

    Article  Google Scholar 

  • Brockhoff D, Hansen N, Mersmann O, Ros R, Tusar D, Tusar T (2017) COCO documentation repository. http://github.com/numbbo/coco-doc

  • Chocat R, Brevault L, Balesdent M, Defoort S (2015) Modified covariance matrix adaptation—evolution strategy algorithm for constrained optimization under uncertainty, application to rocket design. Int J Simul Multidiscip Des Optim 6:A1

    Article  Google Scholar 

  • Conn AR, Scheinberg K, Vicente LN (2009) Introduction to derivative-free optimization. SIAM, Philadelphia

  • Eaton JW, Bateman D, Hauberg S, Wehbring R (2016) GNU Octave version 4.2.0 manual: a high-level interactive language for numerical computations. http://www.gnu.org/software/octave/doc/interpreter

  • Hansen N (2016) The CMA evolution strategy: a tutorial. CoRR arXiv:1604.00772

  • Hansen N, Auger A, Brockhoff D, Tusar D, Tusar T (2016) COCO: performance assessment. ArXiv e-prints arXiv:1605.03560

  • Huang VL, Qin AK, Suganthan PN (2006) Self-adaptive differential evolution algorithm for constrained real-parameter optimization. In: IEEE congress on evolutionary computation (CEC). IEEE, pp 17–24

  • Jamal MB, Ming F, Zhengang J (2018) Solving constrained optimization problems by using covariance matrix adaptation evolutionary strategy with constraint handling methods. In: Proceedings of the 2nd international conference on innovation in artificial intelligence, ICIAI ’18. ACM, New York, NY, USA, pp 6–15. https://doi.org/10.1145/3194206.3194215

  • Jastrebski GA, Arnold DV (2006) Improving evolution strategies through active covariance matrix adaptation. In: 2006 IEEE international conference on evolutionary computation, pp 2814–2821

  • Kolda TG, Lewis RM, Torczon V (2003) Optimization by direct search: new perspectives on some classical and modern methods. SIAM Rev 45(3):385–482

    Article  MathSciNet  MATH  Google Scholar 

  • Kramer O (2010) A review of constraint-handling techniques for evolution strategies. Appl Comput Intell Soft Comput

  • Kusakci AO, Can M (2013) An adaptive penalty based covariance matrix adaptation-evolution strategy. Comput Oper Res 40(10):2398–2417. https://doi.org/10.1016/j.cor.2013.03.013. http://www.sciencedirect.com/science/article/pii/S03050548HrB13000890HrB

  • Loshchilov I, Glasmachers T, Beyer HG (2018) Large scale black-box optimization by limited-memory matrix adaptation. IEEE Trans Evol Comput 1. https://doi.org/10.1109/TEVC.2018.2855049

  • Mallipeddi R, Suganthan PN (2010) Differential evolution with ensemble of constraint handling techniques for solving CEC 2010 benchmark problems. In: IEEE congress on evolutionary computation (CEC). IEEE, pp 1–8

  • Mezura-Montes E, Coello CAC (2005) A simple multimembered evolution strategy to solve constrained optimization problems. IEEE Trans Evol Comput 9(1):1–17. https://doi.org/10.1109/TEVC.2004.836819

    Article  MATH  Google Scholar 

  • Mezura-Montes E, Coello CAC (2011) Constraint-handling in nature-inspired numerical optimization: past, present and future. Swarm Evol Comput 1(4):173–194

    Article  Google Scholar 

  • Oyman AI, Deb K, Beyer HG (1999) An alternative constraint handling method for evolution strategies. In: Angeline P (ed) Proceedings of the CEC’99 conference. IEEE, Piscataway, NJ, pp 612–619

  • Poláková R (2017) L-SHADE with competing strategies applied to constrained optimization. In: 2017 IEEE congress on evolutionary computation (CEC), pp 1683–1689. https://doi.org/10.1109/CEC.2017.7969504

  • Rechenberg I (1973) Evolutionsstrategie Optimierung technischer Systeme nach Prinzipien der biologishen Evolution. Frommann-Holzboog, Stuttgart

    Google Scholar 

  • Runarsson TP, Yao X (2000) Stochastic ranking for constrained evolutionary optimization. IEEE Trans Evol Comput 4(3):284–294. https://doi.org/10.1109/4235.873238

    Article  Google Scholar 

  • Sakamoto N, Akimoto Y (2017) Modified box constraint handling for the covariance matrix adaptation evolution strategy. In: Proceedings of the genetic and evolutionary computation conference companion, GECCO ’17. ACM, New York, NY, USA, pp 183–184. https://doi.org/10.1145/3067695.3075986

  • Schwefel HP (1981) Numerical optimization of computer models. Wiley, Hoboken

    MATH  Google Scholar 

  • Takahama T, Sakai S (2010) Constrained optimization by the \(\varepsilon \) constrained differential evolution with an archive and gradient-based mutation. In: IEEE congress on evolutionary computation (CEC). IEEE, pp 1–9

  • Trosset MW (1997) I know it when I see it: toward a definition of direct search methods. SIAG/OPT Views News Forum SIAM Act Group Optim 9:7–10

    Google Scholar 

  • Wright MH (1996) Direct search methods: once scorned, now respectable. In: Griffiths DF, Watson GA (eds) Numerical analysis 1995 (proceedings of the 1995 Dundee Biennial conference in numerical analysis), Pitman research notes in mathematics, vol 344. CRC Press, Boca Raton, pp 191–208

    Google Scholar 

Download references

Acknowledgements

This work was supported by the Austrian Science Fund FWF under Grant P29651-N32.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Patrick Spettel.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Communicated by A. Di Nola.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

6 Appendix

6 Appendix

This appendix contains further material accompanying the main paper. It is organized as follows: Section 6.1 shows the derivation of the \(\mathbf {M}\)-update for the infeasible mutations. Section 6.2 presents additional experimentation results, and Sect. 6.3 summarizes the changes we performed in the BBOB COCO framework.

1.1 6.1 Derivation of \(\mathbf {M} \leftarrow \mathbf {M}\left( \mathbf {I} - \beta \mathbf {v}_j\mathbf {v}_j^\mathrm{T}\right) \)

The covariance matrix \(\mathbf {C}\) is factored into

$$\begin{aligned} \mathbf {C} = \mathbf {M}\mathbf {M}^\mathrm{T}. \end{aligned}$$
(8)

With this, Lines 12 and 13 of Algorithm 2 correspond to sampling the mutation from \(\mathcal {N}(\mathbf {0}, \mathbf {C})\). Hence, for taking into account the mutations of infeasible offspring in the active covariance adaptation, we aim for

$$\begin{aligned} \mathbf {C} \leftarrow \mathbf {C} - \sum _{j}\mathbb {1}_{\{\text {constraint }j\text { violated}\}} 2\beta \mathbf {w}_j\mathbf {w}_j^\mathrm{T} \end{aligned}$$
(9)

where

$$\begin{aligned} \mathbf {w}_j=\mathbf {M}\mathbf {v}_j. \end{aligned}$$
(10)

That is, the vectors \(\mathbf {w}_j\) are the fading records of the violation vectors corresponding to the mutations that are denoted \(\tilde{\mathbf {d}}\) (cf. Line 13 of Algorithm 2). Notice that the sum in Eq. (9) can also be written as a for loop

figure c

Now, to obtain Eq. (5), the derivations are very similar to those in Beyer and Sendhoff (2017), Eqs. (25)–(30). In the following, let j be the index of a violated constraint. The starting point of the derivation is

$$\begin{aligned} \mathbf {C}^{(g+1)} = \mathbf {C}^{(g)} - 2\beta \mathbf {w}^{(g)}_j\left( {\mathbf {w}^{(g)}_j}\right) ^\mathrm{T}. \end{aligned}$$
(11)

Taking into account Eqs. (8) and (10),

$$\begin{aligned}&\mathbf {M}^{(g+1)}\left( \mathbf {M}^{(g+1)}\right) ^\mathrm{T} =\mathbf {M}^{(g)}\left( \mathbf {M}^{(g)}\right) ^\mathrm{T} - 2\beta \mathbf {M}^{(g)}\mathbf {v}^{(g)}_j \left( \mathbf {M}^{(g)}\mathbf {v}^{(g)}_j\right) ^\mathrm{T} \end{aligned}$$
(12)
$$\begin{aligned}&=\mathbf {M}^{(g)}\left( \mathbf {M}^{(g)}\right) ^\mathrm{T} - 2\beta \mathbf {M}^{(g)}\mathbf {v}^{(g)}_j \left( \mathbf {v}^{(g)}_j\right) ^\mathrm{T}\left( {\mathbf {M}^{(g)}}\right) ^\mathrm{T} \end{aligned}$$
(13)

follows. Pulling out \(\mathbf {M}^{(g)}\) and \(\left( {\mathbf {M}^{(g)}}\right) ^\mathrm{T}\) in Eq. (13) results in

$$\begin{aligned} \mathbf {M}^{(g+1)}\left( \mathbf {M}^{(g+1)}\right) ^\mathrm{T} =\mathbf {M}^{(g)}\left[ \mathbf {I} + \left( -2\beta \mathbf {v}^{(g)}_j \left( \mathbf {v}^{(g)}_j\right) ^\mathrm{T}\right) \right] \left( {\mathbf {M}^{(g)}}\right) ^\mathrm{T}. \end{aligned}$$
(14)

Expansion of Eq. (14) into a matrix power series and cutoff after the linear term analogously to Beyer and Sendhoff (2017), Eqs. (27)–(30) yields

$$\begin{aligned} \mathbf {M}^{(g+1)} =\mathbf {M}^{(g)}\left[ \mathbf {I} + \left( -\beta \mathbf {v}^{(g)}_j \left( \mathbf {v}^{(g)}_j\right) ^\mathrm{T}\right) + \cdots \right] , \end{aligned}$$
(15)

which corresponds to Eq. (5) with neglecting the higher-order terms.

1.2 6.2 Constraint active-\((\mu /\mu _w,\lambda )\)-MA-ES experimental evaluation: presentation of additional results

Figures 5 and 6 show the ECDF plots of the Constraint Active-\((\mu /\mu _w, \lambda )\)-MA-ES for all the functions and dimensions in the BBOB COCO constrained suite. Figures 7 and 8 show the aRT plots.

Tables 4 show the performance in tabular form of the Constraint Active-\((\mu /\mu _w, \lambda )\)-MA-ES in comparison with the conSaDE, the \(\varepsilon \)DEag, and the \((1+1)\)-CMA-ES for constrained optimization for dimension 40.

Fig. 5
figure 5

Bootstrapped empirical cumulative distribution of the number of objective function and constraint evaluations divided by dimension for 51 targets with target precision in \(10^{[-8..2]}\) for all functions and subgroups for the dimensions 2, 3, 5, 10, 20, and 40 for the Constraint Active-\((\mu /\mu _w, \lambda )\)-MA-ES. (Part 1) The horizontal axis shows the \(\log _{10}\) of the sum of objective function and constraint evaluations. The vertical axis shows the proportion of target objective function values reached with the given number of objective function and constraint evaluations (horizontal axis). The crosses indicate the medians of the sum of objective function and constraint evaluations of instances that did not reach the most difficult target

Fig. 6
figure 6

Bootstrapped empirical cumulative distribution of the number of objective function and constraint evaluations divided by dimension for 51 targets with target precision in \(10^{[-8..2]}\) for all functions and subgroups for the dimensions 2, 3, 5, 10, 20, and 40 for the Constraint Active-\((\mu /\mu _w, \lambda )\)-MA-ES. (Part 2) The horizontal axis shows the \(\log _{10}\) of the sum of objective function and constraint evaluations. The vertical axis shows the proportion of target objective function values reached with the given number of objective function and constraint evaluations (horizontal axis). The crosses indicate the medians of the sum of objective function and constraint evaluations of instances that did not reach the most difficult target

Fig. 7
figure 7

Scaling of runtime with dimension of the Constraint Active-\((\mu /\mu _w, \lambda )\)-MA-ES to reach certain target values \(\varDelta f\). (Part 1) Lines: average runtime (\(\mathrm {aRT}\)); Cross \((+)\): median runtime of successful runs to reach the most difficult target that was reached at least once (but not always); Cross (\(\times \)): maximum number of \((f+g)\)-evaluations in any trial. Notched boxes: interquartile range with median of simulated runs; all values are divided by dimension and plotted as \(\log _{10}\) values versus dimension. Numbers above \(\mathrm {aRT}\)-symbols (if appearing) indicate the number of trials reaching the respective target. Horizontal lines mean linear scaling, and slanted grid lines depict quadratic scaling

Fig. 8
figure 8

Scaling of runtime with dimension of the Constraint Active-\((\mu /\mu _w, \lambda )\)-MA-ES to reach certain target values \(\varDelta f\). (Part 2) Lines: average runtime (\(\mathrm {aRT}\)); Cross \((+)\): median runtime of successful runs to reach the most difficult target that was reached at least once (but not always); Cross (\(\times \)): maximum number of \((f+g)\)-evaluations in any trial. Notched boxes: interquartile range with median of simulated runs; all values are divided by dimension and plotted as \(\log _{10}\) values versus dimension. Numbers above \(\mathrm {aRT}\)-symbols (if appearing) indicate the number of trials reaching the respective target. Horizontal lines mean linear scaling, and slanted grid lines depict quadratic scaling

1.3 6.3 BBOB COCO adaptations

Work has already been done for a constrained test suite in the BBOB COCO framework. But some changes of the BBOB COCO framework were necessary to perform the experiments. The GitHub repository of the original BBOB COCO framework is here: https://github.com/numbbo/coco. A fork was created for this work in https://github.com/patsp/coco. The adaptations are in a new branch called development-sppa-2. The main changes are summarized here. For all the details, it is referred to the GitHub repository https://github.com/patsp/coco. The commit logs show all the changes in detail.Footnote 8

  • Log of the relative instead of the absolute objective function distance to the optimum (in code-experiments/src/logger_bbob.c): this is more informative because of the easier comparison of large objective function values with small ones. In the original code, the value \(f - f_{\mathrm{opt}}\) is logged. This change is done to log the value \(|f - f_{\mathrm{opt}}| / |f_{\mathrm{opt}}|\). The division is only done if \(|f_{\mathrm{opt}}|\) is not too small; currently, \(|f_{\mathrm{opt}}| > 10^{-5}\) must hold for the division to be done. Absolute values are used to avoid negative values.

  • Possibility to disable the asymmetric and oscillation nonlinear transformations for the bbob-constrained test suite (in code-experiments/src/transform_vars_oscillate.c and code-experiments/src/transform_vars_asymmetric.c). Currently, this is done with compile-time definitions ENABLE_NON_LINEAR_TRANSFORMATIONS_ON_CONSTRAINTS and ENABLE_NON_LINEAR_TRANSFORMATIONS_ON_OBJECTIVEFUNC that enable the oscillation and asymmetry transformations for the constraint functions and objective function, respectively. This means that by default the transformations are disabled in the current changed version.

  • Addition of a new test suite called custom. It contains at the time of writing this document only the Klee-Minty problemFootnote 9 (new file code-experiments/src/suite_custom.c and changes in code-experiments/src/coco_suite.c).

  • Python post-processing module: Support for the new custom test suite.

  • Python post-processing module: Negative values on the x-axis of the ECDF-plots are clipped to 0. It is possible that negative values occur in the x-axis of the ECDF-plots. This is due to the division by the dimension. It occurs if the value of function and constraint evaluations is less than the dimension because then the value after division is less than 1. The log is taken, and thus, we have a negative value.

  • Python post-processing module: Smaller font size in the plots.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Spettel, P., Beyer, HG. A multi-recombinative active matrix adaptation evolution strategy for constrained optimization. Soft Comput 23, 6847–6869 (2019). https://doi.org/10.1007/s00500-018-03736-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-018-03736-z

Keywords

Navigation