A Benchmark-Suite of real-World constrained multi-objective optimization problems and some baseline results

Abstract

Generally, Synthetic Benchmark Problems (SBPs) are utilized to assess the performance of metaheuristics. However, these SBPs may include various unrealistic properties. As a consequence, performance assessment may lead to underestimation or overestimation. To address this issue, few benchmark suites containing real-world problems have been proposed for all kinds of metaheuristics except for Constrained Multi-objective Metaheuristics (CMOMs). To fill this gap, we develop a benchmark suite of Real-world Constrained Multi-objective Optimization Problems (RWCMOPs) for performance assessment of CMOMs. This benchmark suite includes 50 problems collected from various streams of research. We also present the baseline results of this benchmark suite by using state-of-the-art algorithms. Besides, for comparative analysis, a ranking scheme is also proposed.

Introduction

During the past decades, Constrained Multi-objective Optimization Problems (CMOPs) has gained a lot of attention since the majority of optimization problems of real-world applications contain constraints. Generally, a CMOP has multiple conflicting objectives with one or more constraints that demand to optimize these objectives while satisfying the constraints simultaneously. In CMOPs, Evolutionary Algorithms (EAs) and other metaheuristics have to provide proper tradeoffs among the conflicting objectives while satisfying all constraints, which is a great challenge to them [1], [2].

Without losing generality, a CMOP can be defined mathematically:

$$\text{Minimize}{f}_1\left(\overline{x}\right),f_2\left(\overline{x}\right),\dots ,f_M\left(\overline{x}\right),$$

$$\text{Subject}\text{to}{g}_i\left(\overline{x}\right)\le 0,i\in \{1,2,\dots ,ng\}$$

$$h_j\left(x\right)=0,j\in \{ng+1,ng+2,\dots ,ng+nh\}$$

$$L_k\le x_k\le U_k,k\in \{1,\dots .,D\}$$

where $f_i$ represents the $i$-th objective function, $M$ is the total number of the conflicting objective functions, $\overline{x}=(x_1,x_2$ $,\dots .x_D){}^T$ is a solution vector of length $D$, $L_k$ and $U_k$ are the lower and upper bound of the search-space at $k$-th dimension, $ng$ and $nh$ are the total number of the inequality and equality constraints, respectively. Here, solution $\overline{x}$ can be of two types: feasible and infeasible solution. The feasible solutions satisfy all $(ng+nh)$ constraints of the given problem and blackthe set of all possible feasible solutions within the bound of the search-space creates a subspace in the search-space, called a feasible region. blackHowever, blackthe solution that does not lie in the feasible region is called an infeasible solution. Similarly, a set of all possible infeasible solutions formed an infeasible subspace in the search-space.

The constraint violation of blackthe solution ${\overline{x}}_i$ over a $j$-th constraint can be calculated by the following equation:

$${\nu }_j=\{\begin{array}{cc}max(0,g_j\left({\overline{x}}_i\right)),\hfill & j\le ng\hfill \\ max\left(0,|h_j\left({\overline{x}}_i\right)|-ϵ\right),\hfill & ng<j\le (ng+nh)\hfill \end{array},$$

where ${\nu }_j$ is the value of constraint violation for ${\overline{x}}_i$ on $j$-th constraint and $ϵ$ is a very small value (${10}^{-4}$) for relaxing the equality constraints. On the basis of this definition, a solution can be called as a feasible solution if that solution has zero constraint violation at each constraint or the sum of total constraint violations of that solution is zero, i.e.

$$CV\left({\overline{x}}_i\right)=\sum _{i=1}^{ng+nh}{\nu }_i=0,$$

where $CV\left({\overline{x}}_i\right)$ is the total constraint violation at solution ${\overline{x}}_i$. In the case of a nonzero total constraint violation, the solution is termed as an infeasible solution.

Given two solutions $\overline{a}$ and $\overline{b}$ in Constrained Multi-objective Optimization (CMOO), $\overline{a}$ constrained Pareto dominates $\overline{b}$ (can be denoted as $\overline{a}{\prec }_c\overline{b}$), if and only if

• $f_i\left(\overline{b}\right)\ge f_i\left(\overline{a}\right)$ $\forall$ $i\in \{1,2,\dots ,M\}$,

• $f_j\left(\overline{b}\right)>f_j\left(\overline{a}\right)$ $\exists$ $i\in \{1,2,\dots ,M\}$, and

• $CV\left(\overline{b}\right)\ge CV\left(\overline{a}\right)$.

Here, a feasible solution ${\overline{x}}^{*}$ can be said constrained Pareto optimal solution if all possible feasible solutions do not Pareto dominates ${\overline{x}}^{*}$. The set of all possible constrained Pareto solutions is termed as Pareto set, and the image formed by this Pareto set on objective space is called Pareto front.

In the majority of CMOPs, some solutions of bound-constrained Pareto front become infeasible and loses its optimility due to some constraints. Therefore, CMOPs cannot be solved by using Multi-objective Metaheuristics (MOMs). We need to incorporate a Constraint Handling Technique (CHT) in the framework of the MOMs to handle the constraints. Several CHTs have been utilized with MOMs in the literature, such as constrained dominance principle [3], self-adaptive penalty function [4], and stochastic ranking [5].

As compared to bound-constrained Pareto front, CMOPs can be divided into four types [6].

• Type I: In this case, the constrained Pareto front is the same as the bound-constrained Pareto front, i.e., both Pareto fronts have the same Pareto set.

• Type II: In this case, the constrained Pareto set is the subset of the bound-constrained Pareto set.

• Type III: In this case, some portions of the constrained Pareto front are the same as the bound-constrained Pareto front, i.e., the intersection of both Pareto sets is not a null set.

• Type IV: In this case, the intersection of both Pareto set is a null set, i.e., there is no common region in both Pareto fronts.

While solving the CMOPs, there is a need for the proper balance between minimizing the objective functions and minimizing the constraint violations [7]. Consequently, we can characterize the above-mentioned types of CMOPs according to their required level of balance between minimizing objective functions and minimizing constraint violation [8]. From Type I to Type IV, the required level is gradually increased. Therefore, in the case of Type I CMOPs, there is no need of minimizing constraint violation to calculate the constrained Pareto front. While in case of Type IV CMOPs, more focus is required on minimizing the constraint violations as compared to objective functions.

Generally, theoretical evaluation of the performance of algorithms is difficult due to their stochastic behavior [9]. This is the major reason behind the use of benchmark problems to assess the performance of algorithms empirically. SBPs have been usually used in the performance assessment of the algorithms [10]. The main reasons are that performance evaluation on a real-world application requires domain knowledge of that real-world application and assessment on one problem cannot effectively demonstrate the generality of an algorithm [11].

To cope with this issue, several test-suites having artificial test problems have been designed for CMOPs, see, for example, MFs [6], CFs [12], C-DTLZs [13], SRN [14], TNK [15], OSY [16], and CTPs [17]. There are several advantages to these artificial test suites. They can be easily represented by simple mathematical equations and calculations of objective functions and constraints are computationally cheap and usually fast. Pareto front of these problems is known. Thus, different indicators can be used to represent the experimental results. Most of these problems are scalable to a different number of objectives, the number of decision variables, and the number of constraints. Despite all these advantages, these test problems suffer from serious drawbacks. Usually, they have synthetic properties that may never appear in real-world applications [18], [19]. Consequently, the performance of CMOMs can become overrated on some problems and underrated on other problems. For example, most of the problems of these test-suites are Type-I or Type-II having a regular Pareto front, which can be easily calculated by some of decomposition-based algorithms [18] (MOEA-D [20] and NSGAIII [13]). Since artificial test problems may contain undesirable characteristics, there is a requirement for a test suite of problems of real-world applications to assess the performance of newly developed algorithms more reliably and effectively. In literature, several benchmark suites have been proposed for assessing the performance of the different class of optimization algorithms, see, for example, [21], [22], [23], [24], [25], [26]. However, a benchmark suite of RWCMOPs do not exist, where problems have advantages similar to artificial test problems such as easy to implement, computationally cheap, etc.

To overcome the above-mentioned issues, an easy-to-use test-suite having RWCMOPs is proposed for assessing the performance of CMOPs in this paper. This test suite contains 50 RWCMOPs collected from several areas from mechanical design problems to power system problems. The proposed test-suite provides a diverse set of computationally cheap problems where all problems are implemented by simple mathematical equations. In contrast, the difficulty level of these problems has been maintained at different levels from moderate to high levels. Additionally, these problems do not have unrealistic features as compared to SBPs. However, we do not claim that the proposed test problems will always have better properties than existing synthetic or artificial problems in terms of the performance assessment of CMOMs. We develop this test suite to provide a better tool for conducting the performance assessment of CMOMs over problems of real-world applications in a more realistic way.

The main contributions of this work can be summarized as follows:

• A test suite of 50 RWCMOPs is proposed where problems are collected from different scientific and engineering fields.

• In this paper, we have described all RWCMOPs mathematically. Therefore, there is no need to refer to each original article to implement these problems as this paper is self-contained.

• Moreover, we have implemented this test suite on MATLAB and uploaded it on the official GITHUB page (https://github.com/P-N-Suganthan/2021-RW-MOP). Researchers can easily download this test-suite for examining their CMOMs on RWCMOPs with minimum assistance.

• The performance of seven state-of-the-art algorithms is assessed on these problems and some baseline results are included in this study.

• A ranking scheme is also proposed to compare the performance of CMOMs on this test suite.

The remaining parts of this paper are organized as follows. In Section 2, we describe the 50 RWCMOPs mathematically. In Section 3, experimental settings and a ranking scheme are presented for conducting the experiments for the performance assessment of CMOMs on the proposed test suite. In Section 4, the baseline results of this test-suite calculated by seven state-of-the-art algorithms are reported. Finally, Section 5 concludes the works of this paper.