research-article

A Performance Comparison Indicator for Pareto Front Approximations in Many-Objective Optimization

Authors:

Shengxiang Yang,

Xiaohui LiuAuthors Info & Claims

GECCO '15: Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation

Pages 703 - 710

https://doi.org/10.1145/2739480.2754687

Published: 11 July 2015 Publication History

GECCO '15: Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation

A Performance Comparison Indicator for Pareto Front Approximations in Many-Objective Optimization

Pages 703 - 710

Abstract
References

Abstract

Increasing interest in simultaneously optimizing many objectives (typically more than three objectives) of problems leads to the emergence of various many-objective algorithms in the evolutionary multi-objective optimization field. However, in contrast to the development of algorithm design, how to assess many-objective algorithms has received scant concern. Many performance indicators are designed in principle for any number of objectives, but in practice are invalid or infeasible to be used in many-objective optimization. In this paper, we explain the difficulties that popular performance indicators face and propose a performance comparison indicator (PCI) to assess Pareto front approximations obtained by many-objective algorithms. PCI evaluates the quality of approximation sets with the aid of a reference set constructed by themselves. The points in the reference set are divided into many clusters, and the proposed indicator estimates the minimum moves of solutions in the approximation sets to weakly dominate these clusters. PCI has been verified both by an analytic comparison with several well-known indicators and by an empirical test on four groups of Pareto front approximations with different numbers of objectives and problem characteristics.

References

[1]

J. Bader and E. Zitzler. HypE: An algorithm for fast hypervolume-based many-objective optimization. Evol. Comput., 19(1):45--76, 2011.

Digital Library

[2]

P. A. N. Bosman and D. Thierens. The balance between proximity and diversity in multiobjective evolutionary algorithms. IEEE Trans. Evol. Comput., 7(2):174--188, 2003.

Digital Library

[3]

K. Bringmann, T. Friedrich, C. Igel, and T. Voß. Speeding up many-objective optimization by Monte Carlo approximations. Artif. Intell., 204:22--29, 2013.

Digital Library

[4]

D. W. Corne and J. D. Knowles. Techniques for highly multiobjective optimisation: some nondominated points are better than others. In Proc. 9th Genetic Evol. Comput. Conf. (GECCO), pages 773--780, 2007.

Digital Library

[5]

K. Deb and H. Jain. An evolutionary many-objective optimization algorithm using reference-point-based nondominated sorting approach, part I: Solving problems with box constraints. IEEE Trans. Evol. Comput., 18(4):577--601, 2014.

[6]

K. Deb and S. Jain. Running performance metrics for evolutionary multi-objective optimization. Technical Report 2002004, KanGAL, Indian Institute of Technology, 2002.

[7]

K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evol. Comput., 6(2):182--197, 2002.

Digital Library

[8]

K. Deb, L. Thiele, M. Laumanns, and E. Zitzler. Scalable test problems for evolutionary multiobjective optimization. In A. Abraham, L. Jain, and R. Goldberg, editors, Evolutionary Multiobjective Optimization. Theoretical Advances and Applications, pages 105--145. Berlin, Germany: Springer, 2005.

[9]

P. Fleming, R. Purshouse, and R. Lygoe. Many-objective optimization: An engineering design perspective. In Proc. 3rd Int. Conf. Evol. Multi-Criterion Optimiz. (EMO), pages 14--32. 2005.

Digital Library

[10]

D. Hadka and P. Reed. Diagnostic assessment of search controls and failure modes in many-objective evolutionary optimization. Evol. Comput., 20(3):423--452, 2012.

Digital Library

[11]

Z. He, G. G. Yen, and J. Zhang. Fuzzy-based Pareto optimality for many-objective evolutionary algorithms. IEEE Trans. Evol. Comput., 18(2):269--285, 2014.

Digital Library

[12]

E. J. Hughes. MSOPS-II: A general-purpose many-objective optimiser. In Proc. 2007 IEEE Congr. Evol. Comput. (CEC), pages 3944--3951, 2007.

[13]

K. Ikeda, H. Kita, and S. Kobayashi. Failure of Pareto-based MOEAs: does non-dominated really mean near to optimal? In Proce. 2001 IEEE Congr. Evol. Comput. (CEC), volume 2, pages 957--962, 2001.

[14]

H. Ishibuchi, N. Akedo, and Y. Nojima. Behavior of multi-objective evolutionary algorithms on many-objective knapsack problems. IEEE Trans. Evol. Comput., 19(2):264--283, 2015.

Digital Library

[15]

H. Ishibuchi, H. Masuda, Y. Tanigaki, and Y. Nojima. Difficulties in specifying reference points to calculate the inverted generational distance for many-objective optimization problems. In Proc. 2014 IEEE Symp. Comput. Intell. in Multi-Criteria Decision-Making, pages 170--177, 2014.

[16]

H. Ishibuchi, H. Masuda, Y. Tanigaki, and Y. Nojima. Modified distance calculation in generational distance and inverted generational distance. In Proc. 8th Int. Conf. Evol. Multi-Criterion Optimiz. (EMO), pages 110--125, 2015.

[17]

H. Ishibuchi, N. Tsukamoto, and Y. Nojima. Evolutionary many-objective optimization: A short review. In Proc. 2008 IEEE Congr. Evol. Comput. (CEC), pages 2419--2426, 2008.

[18]

A. L. Jaimes and C. A. Coello Coello. Study of preference relations in many-objective optimization. In Proc. 11th Genetic Evol. Comput. Conf. (GECCO), pages 611--618, 2009.

Digital Library

[19]

J. Knowles and D. Corne. Properties of an adaptive archiving algorithm for storing nondominated vectors. IEEE Trans. Evol. Comput., 7(2):100--116, 2003.

Digital Library

[20]

M. Laumanns, L. Thiele, K. Deb, and E. Zitzler. Combining convergence and diversity in evolutionary multiobjective optimization. Evol. Comput., 10(3):263--282, 2002.

Digital Library

[21]

M. Li, S. Yang, and X. Liu. Diversity comparison of Pareto front approximations in many-objective optimization. IEEE Trans. Cybern., 44(12):2568--2584, 2014.

[22]

M. Li, S. Yang, and X. Liu. Shift-based density estimation for Pareto-based algorithms in many-objective optimization. IEEE Trans. Evol. Comput., 18(3):348--365, 2014.

[23]

M. Li, S. Yang, and X. Liu. A test problem for visual investigation of high-dimensional multi-objective search. In Proc. 2014 IEEE Congr. Evol. Comput. (CEC), pages 2140--2147, 2014.

[24]

M. Li, S. Yang, X. Liu, and R. Shen. A comparative study on evolutionary algorithms for many-objective optimization. In Proc. 7th Int. Conf. Evol. Multi-Criterion Optimiz. (EMO), pages 261--275, 2013.

[25]

M. Li, J. Zheng, K. Li, Q. Yuan, and R. Shen. Enhancing diversity for average ranking method in evolutionarymany-objective optimization. In Proc. 11th Int. Conf. Parallel Problem Solving From Nature (PPSN), pages 647--656, 2010.

Digital Library

[26]

G. Lizárraga-Lizárraga, A. Hernández-Aguirre, and S. Botello-Rionda. G-Metric: an M-ary quality indicator for the evaluation of non-dominated sets. In Proc. 10th Genetic Evol. Comput. Conf., pages 665--672, 2008.

Digital Library

[27]

R. C. Purshouse and P. J. Fleming. On the evolutionary optimization of many conflicting objectives. IEEE Trans. Evol. Comput., 11(6):770--784, 2007.

Digital Library

[28]

O. Schütze, X. Esquivel, A. Lara, and C. A. C. Coello. Using the averaged Hausdorff distance as a performance measure in evolutionary multi-objective optimization. IEEE Trans. Evol. Comput., 16(4):504--522, 2012.

Digital Library

[29]

D. A. Van Veldhuizen. Multiobjective evolutionary algorithms: Classifications, analyses, and new innovations. PhD thesis, Department of Electrical and Computer Engineering, Graduate School of Engineering, Air Force Institute of Technology, Wright-Patterson AFB, Ohio, 1999.

Digital Library

[30]

T. Wagner, N. Beume, and B. Naujoks. Pareto-, aggregation-, and indicator-based methods in many-objective optimization. In Proc. 4th Int. Conf. Evol. Multi-Criterion Optimiz. (EMO), pages 742--756. 2007.

Digital Library

[31]

R. Wang, R. C. Purshouse, and P. J. Fleming. Preference-inspired co-evolutionary algorithms for many-objective optimisation. IEEE Trans. Evol. Comput., 17(4):474--494, 2013.

Digital Library

[32]

S. Yang, M. Li, X. Liu, and J. Zheng. A grid-based evolutionary algorithm for many-objective optimization. IEEE Trans. Evol. Comput., 17(5):721--736, 2013.

Digital Library

[33]

Q. Zhang and H. Li. MOEA/D: A multiobjective evolutionary algorithm based on decomposition. IEEE Trans. Evol. Comput., 11(6):712--731, 2007.

Digital Library

[34]

X. Zhang, Y Tian, and Y. Jin. A knee point driven evolutionary algorithm for many-objective optimization. IEEE Trans. Evol. Comput., 2014, in press.

[35]

E. Zitzler and S. Künzli. Indicator-based selection in multiobjective search. In Proc. 8th Int. Conf. Parallel Problem Solving from Nature (PPSN), pages 832--842. 2004.

[36]

E. Zitzler and L. Thiele. Multiobjective evolutionary algorithms: A comparative case study and the strength Pareto approach. IEEE Trans. Evol. Comput., 3(4):257--271, 1999.

Digital Library

[37]

E. Zitzler, L. Thiele, M. Laumanns, C. M. Fonseca, and V. G. da Fonseca. Performance assessment of multiobjective optimizers: An analysis and review. IEEE Trans. Evol. Comput., 7(2):117--132, 2003.

Digital Library

Cited By

Li MLópez-Ibáñez MYao X(2024)Multi-Objective ArchivingIEEE Transactions on Evolutionary Computation10.1109/TEVC.2023.331415228:3(696-717)Online publication date: Jun-2024
https://doi.org/10.1109/TEVC.2023.3314152
Wu XSardahi Y(2023)Many-Objective Optimal Control for Quadcopters2023 American Control Conference (ACC)10.23919/ACC55779.2023.10156094(2105-2110)Online publication date: 31-May-2023
https://doi.org/10.23919/ACC55779.2023.10156094
Singer DSypniewski M(2023)A Framework for Evaluating Inherent Biases within Ship Design ToolsSSRN Electronic Journal10.2139/ssrn.4353611Online publication date: 2023
https://doi.org/10.2139/ssrn.4353611
Show More Cited By

Index Terms

A Performance Comparison Indicator for Pareto Front Approximations in Many-Objective Optimization

Recommendations

Characterizing Pareto Front Approximations in Many-objective Optimization
GECCO '15: Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation

A Pareto Optimal Front (POF) provides the set of optimal trade-off solutions formulti-objective optimization problems. The characteristics of the POF, e.g. continuity, convexity, spread and uniformity are important in the context of decision making and ...
Approximate non-dominated sorting for evolutionary many-objective optimization

Non-dominated sorting has widely been adopted and shown to be very effective in dominance based evolutionary multi-objective optimization where the number of objectives is two or three. In dealing with many-objective optimization problems, where the ...
An objective reduction algorithm using representative Pareto solution search for many-objective optimization problems

In recent years, many-objective optimization problems (i.e. more than three objectives) have attracted the interests of many researchers. The main difficulties of many-objective optimization problems lie in high computational cost, stagnation in search ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

GECCO '15: Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation

July 2015

1496 pages

ISBN:9781450334723

DOI:10.1145/2739480

Editor:
Sara Silva
Universidade de Lisboa, Portugal
,
General Chair:
Anna I. Esparcia-Alcázar
Universitat Politècnica de València, Spain

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGEVO: ACM Special Interest Group on Genetic and Evolutionary Computation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 July 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

GECCO '15

Sponsor:

SIGEVO

GECCO '15: Genetic and Evolutionary Computation Conference

July 11 - 15, 2015

Madrid, Spain

Acceptance Rates

GECCO '15 Paper Acceptance Rate 182 of 505 submissions, 36%;

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

33
Total Citations
View Citations
347
Total Downloads

Downloads (Last 12 months)21
Downloads (Last 6 weeks)4

Reflects downloads up to 27 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Li MLópez-Ibáñez MYao X(2024)Multi-Objective ArchivingIEEE Transactions on Evolutionary Computation10.1109/TEVC.2023.331415228:3(696-717)Online publication date: Jun-2024
https://doi.org/10.1109/TEVC.2023.3314152
Wu XSardahi Y(2023)Many-Objective Optimal Control for Quadcopters2023 American Control Conference (ACC)10.23919/ACC55779.2023.10156094(2105-2110)Online publication date: 31-May-2023
https://doi.org/10.23919/ACC55779.2023.10156094
Singer DSypniewski M(2023)A Framework for Evaluating Inherent Biases within Ship Design ToolsSSRN Electronic Journal10.2139/ssrn.4353611Online publication date: 2023
https://doi.org/10.2139/ssrn.4353611
Dubey RHickinbotham SPrice MTyrrell A(2023)Local Fitness Landscape Exploration Based Genetic AlgorithmsIEEE Access10.1109/ACCESS.2023.323477511(3324-3337)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3234775
Sypniewski MSinger D(2023)A framework for evaluating inherent biases within ship design toolsOcean Engineering10.1016/j.oceaneng.2023.115019284(115019)Online publication date: Sep-2023
https://doi.org/10.1016/j.oceaneng.2023.115019
Wüthrich DZeverino MBourhis JBochud FMoeckli R(2023)Influence of optimisation parameters on directly deliverable Pareto fronts explored for prostate cancerPhysica Medica10.1016/j.ejmp.2023.103139114(103139)Online publication date: Oct-2023
https://doi.org/10.1016/j.ejmp.2023.103139
Guerreiro AKlamroth KFonseca C(2023)Theoretical Aspects of Subset Selection in Multi-Objective OptimisationMany-Criteria Optimization and Decision Analysis10.1007/978-3-031-25263-1_8(213-239)Online publication date: 29-Jul-2023
https://doi.org/10.1007/978-3-031-25263-1_8
Afsar BFieldsend JGuerreiro AMiettinen KRojas Gonzalez SSato H(2023)Many-Objective Quality MeasuresMany-Criteria Optimization and Decision Analysis10.1007/978-3-031-25263-1_5(113-148)Online publication date: 29-Jul-2023
https://doi.org/10.1007/978-3-031-25263-1_5
Li MChen TYao X(2022)How to Evaluate Solutions in Pareto-Based Search-Based Software Engineering: A Critical Review and Methodological GuidanceIEEE Transactions on Software Engineering10.1109/TSE.2020.303610848:5(1771-1799)Online publication date: 1-May-2022
https://doi.org/10.1109/TSE.2020.3036108
Cai XXiao YLi ZSun QXu HLi MIshibuchi H(2022)A Kernel-Based Indicator for Multi/Many-Objective OptimizationIEEE Transactions on Evolutionary Computation10.1109/TEVC.2021.310556526:4(602-615)Online publication date: Aug-2022
https://doi.org/10.1109/TEVC.2021.3105565
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten