Regular PaperNovel performance metrics for robust multi-objective optimization algorithms
Introduction
Recently, evolutionary algorithms (EAs) have become very popular. The general framework of EAs is to create a population of random solutions for a particular problem and effectively evolve it over a pre-defined number of steps. In addition to many theoretical studies, EA techniques have been applied to many real problems in different fields of study such as mechanical engineering, chemistry, civil engineering, etc. [1], [2]. The reason of this increasing popularity is the unique advantages of EAs.
EAs have many advantages compared to conventional optimization techniques, two of which are their derivative-free mechanism and flexibility. Both of these advantages originate from the stochastic nature of EAs, which assists them in considering an optimization problem as a black box and optimizing it without gradient information about its search space. Since extracting gradient information of real problems is not trivial or is occasionally impossible [3], derivative-free optimization techniques are often more applicable. Moreover, random operators prevent EAs from easily stagnating in local optima in contrast to the conventional gradient-based optimization methods [4]. These two advantages are the main reasons for the wide applicability of EAs in practice.
The large number of theoretical and practical works in the literature may also be due to the NFL theorem [5]. This theorem logically demonstrates that there is no optimization technique best suited for solving all optimization problems [6]. In other words, a very good algorithm for a particular set of problems might provide very poor performance on others. This is the main foundation of many theoretical works, in which current EAs have been improved or new EAs have been proposed.
Two essential components when improving or developing an EA are test problems [7], [8], [9] and performance metrics [10], [11]. Test problems provide challenging test beds in order to benchmark different capabilities of EAs, whereas performance metrics measure the performance of EAs from different perspectives. In addition, performance metrics allow designers to conduct quantitative comparative studies. The literature shows that there are a considerable number of performance metrics in the field of evolutionary multi-objective optimization (EMOO) [12]. Due to the complexity of the optimization process and multi-objectivity in the EMOO field, there should be several performance metrics when comparing algorithms in order to provide a fair and objective comparison [13], [14].
The literature shows that different branches of EMOO also need specific or adapted metrics for effectively quantifying the performance of algorithms. For instance, dynamic multi-objective optimization (DMOO) [15] and interactive multi-objective optimization (IMOO) [16] need their own modified metrics as discussed in [17], [18], [19], respectively. To the best of our knowledge, however, there is no performance metric in the field of robust multi-objective optimization (RMOO) despite its significant importance. This motivates us to propose three novel performance metrics for robust multi-objective algorithms. The remainder of the paper is organized as follows.
Section 2 provides the required definitions and preliminaries of EMOO and RMOO. The current performance metrics in EMOO are discussed in Section 3. Section 4 proposes three novel performance metrics for RMOO. In Section 5 the proposed performance metrics are employed to compare the performance of the Multi-objective Particle Swarm Optimization (MOPSO) algorithm and the Non-dominated Sorting Genetic Algorithm (NSGA-II) on seven test function. Eventually, Section 6 concludes the work and advises several guidelines for future studies.
Section snippets
Definitions and preliminaries of multi-objective and robust optimization
This section provides basics, preliminaries, and concepts of multi-objective and robust multi-objective optimization.
Current performance metrics in EMOO
This section covers the current performance metrics in the literature of EMOO. The main purpose of a performance indicator in EMOO is to quantify the performance from a specific point of view. Generally speaking, the ultimate goal in EMOO is to find a very accurate approximation and large number of the true Pareto optimal solutions with uniform distribution among all objectives [44]. Therefore, the current performance measures can be classified into three main categories: convergence, coverage,
Proposed metrics
Similar to the performance metrics in EMOO, there should be more than one unary metric in order to efficiently evaluate and compare the performance of an algorithm. The three main performance characteristics for an algorithm when finding an approximation of the robust front are: convergence, distribution, and the number of robust Pareto optimal solutions obtained. The first characteristic refers to the convergence of an algorithm towards the true robust Pareto optimal solutions. In this case,
Results and discussion
In this section the proposed metrics are employed to compared Robust Multi-Objective Particle Swarm Optimization (RMOPSO) [59] and Robust Non-dominated Sorting Genetic Algorithm (RNSGA-II) [32], [54] on 7 selected benchmark problems. We used 100 search agents over 1000 iterations to approximate the robust Pareto optimal fronts of the test functions. It should be noted that other initial parameters of both algorithms are identical to those of their original papers in [59], [32], [54]. For
Conclusion
This paper proposed three novel performance metrics in the field of robust multi-objective optimization (RMOO) for measuring convergence, coverage, and the success ratio of EMOO algorithms in finding robust Pareto optimal solutions. The proposed performance metrics were used to quantitatively evaluate and compare two popular EMOO algorithms (RMOPSO vs. RNSGA-II) using a set of 7 benchmark multi-objective optimization problems.
The experimental results showed that the proposed performance metrics
References (62)
- et al.
Training feedforward neural networks using hybrid particle swarm optimization and gravitational search algorithm
Appl. Math. Comput.
(2012) - et al.
Three new metrics to measure the convergence of metaheuristics towards the Pareto frontier and the aesthetic of a set of solutions in biobjective optimization
Comput. Oper. Res.
(2005) - et al.
Performance measures for dynamic multi-objective optimisation algorithms
Inf. Sci.
(2013) - et al.
Multiobjective evolutionary algorithms: a survey of the state of the art
Swarm Evol. Comput.
(2011) - et al.
Robust optimization—a comprehensive survey
Comput. Methods Appl. Mech. Eng.
(2007) - et al.
An investigation on noise-induced features in robust evolutionary multi-objective optimization
Expert Syst. Appl.
(2010) - et al.
Multiobjective Evolutionary: Algorithms and Applications
(2005) - et al.
Differential evolution: a survey of the state-of-the-art
IEEE Trans. Evol. Comput.
(2011) - O. Kramer, D.E. Ciaurri, S. Koziel, Derivative-free optimization, In: S. Koziel, X.-S. Yang (Eds.) Computational...
Free lunch or no free lunch: that is not just a question?
Int. J. Artif. Intell. Tools
(2012)
Simple explanation of the no-free-lunch theorem and its implications
J. Optim. Theory Appl.
Performance assessment of multiobjective optimizers: an analysis and review
IEEE Trans. Evol. Comput.
Dynamic multi-objective optimization and decision-making using modified NSGA-II: a case study on hydro-thermal power scheduling
Evol. Multi-Criterion Optim.
An interactive evolutionary multiobjective optimization method based on progressively approximated value functions
IEEE Trans. Evol. Comput.
Evolutionary Algorithms for Solving Multi-objective Problems
Mathematical Physics
Cours d׳economie politique
Evolutionary multi-objective optimization: some current research trends and topics that remain to be explored
Front. Comput. Sci. China
Evolutionary optimization in uncertain environments—a survey
IEEE Trans. Evol. Comput.
Towards practical evolutionary robust multi-objective optimization
Creating robust solutions by means of evolutionary algorithms
Cited by (69)
A bi-objective optimization for finance-based and resource-constrained robust project scheduling
2023, Expert Systems with ApplicationsAn allocation-routing optimization model for integrated solid waste management
2023, Expert Systems with ApplicationsA bi-stage multi-objectives optimization of smart multi energy system with active consumers
2022, Sustainable Production and ConsumptionMulti-objective optimization of a reluctance actuator for precision motion applications
2022, Journal of Magnetism and Magnetic Materials