A modified Covariance Matrix Adaptation Evolution Strategy with adaptive penalty function and restart for constrained optimization

Highlights

• We introduce a modified Covariance Matrix Adaptation Evolution Strategy.

• Our CMA-ES is specifically designed for solving constrained optimization problems.

• The proposed method makes use of a restart mechanism and adaptive penalty function.

• This novel CMA-ES presents competitive results on a set of benchmark functions.

Abstract

In the last decades, a number of novel meta-heuristics and hybrid algorithms have been proposed to solve a great variety of optimization problems. Among these, constrained optimization problems are considered of particular interest in applications from many different domains. The presence of multiple constraints can make optimization problems particularly hard to solve, thus imposing the use of specific techniques to handle fitness landscapes which generally show complex properties. In this paper, we introduce a modified Covariance Matrix Adaptation Evolution Strategy (CMA-ES) specifically designed for solving constrained optimization problems. The proposed method makes use of the restart mechanism typical of most modern variants of CMA-ES, and handles constraints by means of an adaptive penalty function. This novel CMA-ES scheme presents competitive results on a broad set of benchmark functions and engineering problems, outperforming most state-of-the-art algorithms as for both efficiency and constraint handling.

Introduction

In real-world, several problems are modeled as nonlinear optimization problems, where objective functions have to be optimized under some given constraints. In this kind of problem, one wants to find $\overrightarrow{x}$ that optimizes $f(\overrightarrow{x})$ which is

$$\text{subject to}:\begin{array}{cc}{h}_i(\overrightarrow{x})=0\text{,}\hfill & i=1\text{,}2\text{,}\dots \text{,}m\hfill \\ g_i(\overrightarrow{x})\le 0\text{,}\hfill & i=1\text{,}2\text{,}\dots \text{,}p\hfill \end{array}$$

where $f(\overrightarrow{x})$ is the objective (or fitness) function to be optimized, and $\overrightarrow{x}\in {\mathbb{R}}^n$ is an n-dimensional vector of design variables, $\overrightarrow{x}=[x_1\text{,}{x}_2\text{,}\dots \text{,}{x}_n]$. This vector can contain mixed variables such as integer, discrete and continuous ones. Each $x_k\text{,}k=1\text{,}\dots \text{,}n$ can be bounded by lower and upper limits $L_i⩽x_k⩽U_k$ (box constraints); $h_i(\overrightarrow{x})$ and $g_i(\overrightarrow{x})$ are called equality and inequality constraints, being their number m and p, respectively. Both kinds of constraint can be linear or nonlinear.

Over the years, several meta-heuristics have been proposed to solve constrained problems and find global optimum solutions. Among these, Evolutionary Algorithms, such as Genetic Algorithms (GA), see (Coello Coello, 2000, Coello Coello and Mezura-Montes, 2002, Michalewicz, 1996), and Evolution Strategies (ES), see (Mezura-Montes & Coello Coello, 2005), have attracted most of the research interest. Swarm Intelligence has also been investigated as tool for solving constrained optimization problems, see for instance the studies on Particle Swarm Optimization (PSO) (Aguirre et al., 2007, Cagnina et al., 2008, Coelho, 2010, He and Wang, 2007, Pant et al., 2009, Hu et al., 2003), Cuckoo Search (Yang & Deb, 2010), or Artificial Bee Colonies (Brajevic, Tuba, & Subotic, 2011). Another method which is becoming increasingly popular in this field is Differential Evolution (DE): in the last decade, several variants of DE specifically designed for constrained optimization have been proposed, see the papers (Mezura-Montes et al., 2004, Mezura-Montes et al., Mezura-Montes and Palomeque-Ortiz, 2009a, Mezura-Montes and Palomeque-Ortiz, 2009b), or the most recent works (de Melo and Carosio, 2012, Mohamed and Sabry, 2012, Wang and Cai, 2011).

In the excellent comparative study proposed in Mezura-Montes and Lopez-Ramirez (2007), DE outperformed PSO, GA, ES on a broad set of numerical and engineering constrained optimization problems, although it was also observed that PSO tends to reach the feasible region faster, while ES is very effective in improving upon feasible solutions previously found. Motivated by this result, recent studies focused on hybrid techniques, in the attempt of overcoming the specific limitations of each meta-heuristic: for example, Liu, Cai, and Wang (2010) embedded three mutation rules typical of DE (rand/1, current-to-best/1, and rand/2) into a PSO framework to force it jump out of stagnation; in a similar way, He and Wang (2007) hybridized PSO with Simulated Annealing, which was used as local-search to improve upon the best solution of the swarm. Similar ideas were investigated in Sun and Garibaldi, 2010, Ullah et al., 2007, where different kinds of memetic algorithms were introduced, combining a global search scheme with one or more local-search methods. More recently, Pescador Rojas and Coello Coello (2012) presented instead an agent based memetic algorithm including a co-evolution approach.

One of the most popular—and perhaps intuitive—constraint-handling technique known from the literature is the use of a penalty function which worsens the fitness of unfeasible functions obtained during the search (Michalewicz, 1996). According to this strategy, feasible solutions (i.e., solutions belonging to the feasible set, in the following denoted as F) are simply evaluated based on their function value (i.e. the original objective function), while a penalty function is applied to unfeasible solutions to artificially generate a poor fitness. The above scheme can be formulated as:

$$f(\overrightarrow{x})=\left\{\begin{array}{cc}\mathit{objfun}(\overrightarrow{x})\hfill & \mathit{if}\overrightarrow{x}\in F\hfill \\ \mathit{objfun}(\overrightarrow{x})+\text{<mi mathsize="small" is="true">PENALTY</mi>}(\overrightarrow{x})\hfill & \mathit{otherwise}\hfill \end{array}\right.$$

where $\mathit{objfun}(\overrightarrow{x})$ and $\mathit{penalty}(\overrightarrow{x})$ obviously indicate the objective and penalty function, respectively. Despite its simplicity, the main drawback of this method lies in the determination of suitable penalty functions, which in turn affects the efficiency of the search. Several specific mechanisms have been designed to solve this issue, see for example (Coelho, 2010, Coello Coello and Mezura-Montes, 2002, He and Wang, 2007, Pant et al., 2009). Other studies focused on self-adapting schemes, see (Coello Coello, 2000, Mezura-Montes et al., Michalewicz, 1996, Noman and Iba, 2008), where the penalty function is adapted during the optimization process. An alternative approach, called filter-based EA (Clevenger, Ferguson, & Hart, 2005), aggregates the constraint violations and tries to solve an equivalent bi-objective problem, minimizing both the original objective function and the aggregated constraint violation function.

It can be easily seen that all these works have in common the underlying idea that population-based meta-heuristics, due to their parallel nature, intrinsically form an efficient tool for solving constrained optimization problems. On the other hand, in order to apply population-based algorithms to this kind of problems, it is imperative to endow them of proper ways to handle constraints, so to avoid unwanted situations such as stagnation and premature convergence, and guide the optimization process towards (and within) the bounds of the feasible region (s) of the search-space.

In this paper, we investigate a well-known meta-heuristic which has proven extremely successful on unconstrained problems, being yet barely studied in constrained optimization, namely the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) (Hansen, Müller, & Koumoutsakos, 2003). The choice of looking at CMA-ES as tool for constrained optimization is mostly due to the fact that this algorithm represents the current state-of-the-art in optimization, although very few studies focus on its application to constrained problems. Therefore we believe that it is of great interest trying to exploit the powerful features of robustness of CMA-ES also in constrained contexts. To overcome the limitations of CMA-ES on constrained problems, (1) we modify the basic CMA-ES framework by applying some simple changes in its stop criteria and sampling mechanism and (2) we introduce an adaptive penalty function within the algorithm, which efficiently handles constraints and dynamically adjusts the penalty during the optimization. In our view, this adaptive penalty scheme represents the main contribution of this paper. Interestingly, this penalty scheme could be applied also to meta-heuristics other than CMA-ES. However, since our purpose is to extend the range of applications of CMA-ES to constrained optimization, here we prove the applicability of this penalty function only on CMA-ES, and we assess the advantages due to its use. Extensive numerical experiments and comparisons with state-of-the-art methods complete this paper, where we evaluate the performance of the proposed modified CMA-ES with adaptive penalty function on a broad set of benchmark problems and engineering applications take from specialized literature.

The paper is organized as follows: in Section 2, we briefly introduce the basic CMA-ES algorithm. Section 3 describes the proposed algorithm based on modified CMA-ES, with particular focus on the adaptive penalty function. Section 4 includes the description of the test problems, the details of the experiments, the results obtained, and the related discussion. Finally, in Section 5 concludes this work.