tutorial

Difficulties in fair performance comparison of multiobjective evolutionary algorithms

Authors:

Hisao Ishibuchi,

Lie Meng Pang,

Ke ShangAuthors Info & Claims

GECCO '22: Proceedings of the Genetic and Evolutionary Computation Conference Companion

Pages 937 - 957

https://doi.org/10.1145/3520304.3533634

Published: 19 July 2022 Publication History

Get Access

References

[1]

A. Kumar et al., "A benchmark-suite of real-world constrained multi-objective optimization problems and some baseline results," Swarm and Evolutionary Computation, vol. 67, Article no. 100961, 2021.

Crossref

Google Scholar

[2]

C. He, Y. Tian, H. Wang, Y. Jin, "A repository of real-world datasets for data-driven evolutionary multiobjective optimization," Complex & Intelligent Systems, vol. 6, pp. 189--197, 2020.

Crossref

Google Scholar

[3]

H. Ishibuchi, Y. Setoguchi, H. Masuda, and Y. Nojima, "How to compare many-objective algorithms under different settings of population and archive sizes," in Proceedings of IEEE CEC 2016, pp. 1149--1156, 2016.

Google Scholar

[4]

H. Ishibuchi, Y. Setoguchi, H. Masuda, and Y. Nojima, "Performance of decomposition-based many-objective algorithms strongly depends on Pareto front shapes," IEEE Transactions on Evolutionary Computation, vol. 21, no. 2, pp. 169--190, 2017.

Digital Library

Google Scholar

[5]

H. Ishibuchi, R. Imada, Y. Setoguchi, and Y. Nojima, "How to specify a reference point in hypervolume calculation for fair performance comparison," Evolutionary Computation, vol. 26, no. 3, pp. 411--440, 2018.

Digital Library

Google Scholar

[6]

H. Ishibuchi, R. Imada, Y. Setoguchi, and Y. Nojima, "Reference point specification in inverted generational distance for triangular linear Pareto front," IEEE Transactions on Evolutionary Computation, vol. 22, no. 6, pp. 961--975, 2018.

Digital Library

Google Scholar

[7]

L. While, L. Bradstreet, and L. Barone, "A fast way of calculating exact hypervolumes," IEEE Transactions on Evolutionary Computation, vol. 16, no. 1, pp. 86--95, 2012.

Digital Library

Google Scholar

[8]

K. Shang, H. Ishibuchi, M.-L. Zhang, and Y. Liu, "A new R2 indicator for better hypervolume approximation," in Proceedings of the Genetic and Evolutionary Computation Conference, 2018, pp. 745--752.

Digital Library

Google Scholar

[9]

K. Shang, W. Chen, W. Liao, H. Ishibuchi, "HV-Net: Hypervolume approximation based on DeepSets," arXiv preprint arXiv:2203.02185 (2022).

Google Scholar

[10]

H. Ishibuchi, L. He, and K. Shang, "Regular Pareto front shape is not realistic," in Proceedings of IEEE CEC 2019, 2019.

Google Scholar

[11]

H. Ishibuchi, T. Matsumoto, N. Masuyama, and Y. Nojima, "Many-objective problems are not always difficult for Pareto dominance-based evolutionary algorithms," in Proceedings of 24th European Conference on Artificial Intelligence (ECAI), 2020.

Google Scholar

[12]

H. Ishibuchi, L. M. Pang, and K. Shang, "A new framework of evolutionary multi-objective algorithms with an unbounded external archive," in Proceedings of 24th European Conference on Artificial Intelligence (ECAI), 2020.

Google Scholar

[13]

H. Ishibuchi, L. M. Pang, K. Shang, "Difficulties in fair performance comparison of multi-objective evolutionary algorithms," IEEE Computational Intelligence Magazine, vol.17, no.1, pp. 86--101, 2022.

Digital Library

Google Scholar

[14]

H. K. Singh, K. S. Bhattacharjee, and T. Ray, "Distance-based subset selection for benchmarking in evolutionary multi/many-objective optimization," IEEE Transactions on Evolutionary Computation, vol. 23, no. 5, pp. 904--912, 2019.

Crossref

Google Scholar

[15]

J. E. Fieldsend, R. M. Everson, and S. Singh, "Using unconstrained elite archive for multiobjective optimization," IEEE Transactions on Evolutionary Computation, vol. 7, no. 3, pp. 305--323, 2003.

Digital Library

Google Scholar

[16]

J. Knowles, "ParEGO: A hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems", IEEE Transactions on Evolutionary Computation, vol. 10, no. 1, pp. 50--66, 2006.

Digital Library

Google Scholar

[17]

K. Bringmann, T. Friedrich, and P. Klitzke, "Generic postprocessing via subset selection for hypervolume and epsilon-indicator," in Proceedings of PPSN 2014, pp. 518--527, 2014. 82

Google Scholar

[18]

K. Deb and H. Jain, "An evolutionary many-objective optimization algorithm using reference-point-based nondominated sorting approach, Part I: Solving problems with box constraints," IEEE Transactions on Evolutionary Computation, vol. 18, no. 4, pp. 577--601, 2014.

Crossref

Google Scholar

[19]

R. Tanabe, H. Ishibuchi, and A. Oyama, "Benchmarking multi- and many-objective evolutionary algorithms under two optimization scenarios," IEEE Access, vol. 5, pp. 19597--19619, 2017.

Crossref

Google Scholar

[20]

K. Shang et al, "Hypervolume-Optimal μ-Distributions on Line/Plane-Based Pareto Fronts in Three Dimensions," IEEE Transactions on Evolutionary Computation, vol. 26, no. 2, pp. 349--363, 2022.

Digital Library

Google Scholar

[21]

R. Tanabe and H. Ishibuchi, "An analysis of quality indicators using approximated optimal distributions in a three-dimensional objective space," IEEE Transactions on Evolutionary Computation, vol. 24, no. 5, pp. 853--867, 2020.

Crossref

Google Scholar

[22]

R. Tanabe and H. Ishibuchi, "An easy-to-use real-world multi-objective optimization problem suite," Applied Soft Computing, vol. 89, Article no. 106078, 2020.

Digital Library

Google Scholar

[23]

S. Mostaghim and H. Schmeck, "Distance based ranking in many-objective particle swarm optimization," in Proceedings of PPSN 2008, pp. 753--762, 2008.

Google Scholar

[24]

T. Wagner, N. Beume, and B. Naujoks, "Pareto-, aggregation-, and indicator-based methods in many-objective optimization," in Proceedings of EMO 2007, pp. 742--756, 2007.

Google Scholar

[25]

W. Chen, H. Ishibuchi, and K. Shang, "Fast greedy subset selection from large candidate solution sets in evolutionary multi-objective optimization," IEEE Transactions on Evolutionary Computation (Early Access).

Google Scholar

[26]

X. Zhang, Y. Tian, R. Cheng, and Y. Jin, "A decision variable clustering-based evolutionary algorithm for large-scale many-objective optimization," IEEE Transactions on Evolutionary Computation, vol. 22, no. 1, pp. 97--112, 2018.

Crossref

Google Scholar

Cited By

View all

Zatarain Salazar JHadka DReed PSeada HDeb K(2024)Diagnostic benchmarking of many-objective evolutionary algorithms for real-world problemsEngineering Optimization10.1080/0305215X.2024.238181857:1(287-308)Online publication date: 22-Aug-2024
https://doi.org/10.1080/0305215X.2024.2381818
Ming FGong WZhen HWang LGao L(2024)Constrained multi-objective optimization evolutionary algorithm for real-world continuous mechanical design problemsEngineering Applications of Artificial Intelligence10.1016/j.engappai.2024.108673135(108673)Online publication date: Sep-2024
https://doi.org/10.1016/j.engappai.2024.108673

Index Terms

Difficulties in fair performance comparison of multiobjective evolutionary algorithms

Index terms have been assigned to the content through auto-classification.

Recommendations

Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach

Evolutionary algorithms (EAs) are often well-suited for optimization problems involving several, often conflicting objectives. Since 1985, various evolutionary approaches to multiobjective optimization have been developed that are capable of searching ...
Behavior of Multiobjective Evolutionary Algorithms on Many-Objective Knapsack Problems
We examine the behavior of three classes of evolutionary multiobjective optimization (EMO) algorithms on many-objective knapsack problems. They are Pareto dominance-based, scalarizing function-based, and hypervolume-based algorithms. NSGA-II, MOEA/D, SMS-...
Comparison of Multiobjective Evolutionary Algorithms: Empirical Results

In this paper, we provide a systematic comparison of various evolutionary approaches to multiobjective optimization using six carefully chosen test functions. Each test function involves a particular feature that is known to cause difficulty in the ...

Comments

Information & Contributors

Information

Published In

GECCO '22: Proceedings of the Genetic and Evolutionary Computation Conference Companion

July 2022

2395 pages

ISBN:9781450392686

DOI:10.1145/3520304

Editor:
Jonathan E. Fieldsend
University of Exeter
,
General Chair:
Markus Wagner
The University of Adelaide

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 July 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Tutorial

Funding Sources

the Program for Guangdong Introducing Innovative and Enterpreneurial Teams
National Natural Science Foundation of China
Shenzhen Science and Technology Program
Guangdong Provincial Key Laboratory
The Stable Support Plan Program of Shenzhen Natural Science Fund

Conference

GECCO '22

Sponsor:

SIGEVO

GECCO '22: Genetic and Evolutionary Computation Conference

July 9 - 13, 2022

Massachusetts, Boston

Acceptance Rates

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
176
Total Downloads

Downloads (Last 12 months)56
Downloads (Last 6 weeks)2

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Zatarain Salazar JHadka DReed PSeada HDeb K(2024)Diagnostic benchmarking of many-objective evolutionary algorithms for real-world problemsEngineering Optimization10.1080/0305215X.2024.238181857:1(287-308)Online publication date: 22-Aug-2024
https://doi.org/10.1080/0305215X.2024.2381818
Ming FGong WZhen HWang LGao L(2024)Constrained multi-objective optimization evolutionary algorithm for real-world continuous mechanical design problemsEngineering Applications of Artificial Intelligence10.1016/j.engappai.2024.108673135(108673)Online publication date: Sep-2024
https://doi.org/10.1016/j.engappai.2024.108673

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

References

Cited By

Index Terms

Recommendations

Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach

Behavior of Multiobjective Evolutionary Algorithms on Many-Objective Knapsack Problems

Comparison of Multiobjective Evolutionary Algorithms: Empirical Results

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Qualifiers

Funding Sources

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations