research-article

A Study on Performance Evaluation Ability of a Modified Inverted Generational Distance Indicator

Authors:

Hisao Ishibuchi,

Hiroyuki Masuda,

Yusuke NojimaAuthors Info & Claims

GECCO '15: Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation

Pages 695 - 702

https://doi.org/10.1145/2739480.2754792

Published: 11 July 2015 Publication History

Abstract

The inverted generational distance (IGD) has been frequently used as a performance indicator for many-objective problems where the use of the hypervolume is difficult. However, since IGD is not Pareto compliant, it is possible that misleading Pareto incompliant results are obtained. Recently, a simple modification of IGD was proposed by taking into account the Pareto dominance relation between a solution and a reference point when their distance is calculated. It was also shown that the modified indicator called IGD+ is weakly Pareto compliant. However, actual effects of the modification on performance comparison have not been examined. Moreover, IGD+ has not been compared with other distance-based weakly Pareto compliant indicators such as the additive epsilon indicator and the D1 indicator (i.e., IGD with the weighted achievement scalarizing function). In this paper, we examine the effect of the modification by comparing IGD+ with IGD for multiobjective and many-objective problems. In computational experiments, we generate a large number of ordered pairs of non-dominated solution sets where one is better than the other. Two solution sets in each pair are compared by the above-mentioned performance indicators. We examine whether each indicator can correctly say which solution set is better between them.

References

[1]

Bader, J. and Zitzler, E. 2011. HypE: An algorithm for fast hypervolume-based many-objective optimization. Evolutionary Computation 19, 1 (Spring 2011) 45--76.

Digital Library

[2]

Beume, N., Naujoks, B., and Emmerich, M. 2007. SMS-EMOA: multiobjective selection based on dominated hypervolume. European Journal of Operational Research 180, 3 (September 2007) 1653--1669.

[3]

Bosman, P.A.N. and Thierens, D. 2003. The balance between proximity and diversity in multiobjective evolutionary algorithms. IEEE Trans. on Evolutionary Computation 7, 2 (April 2003) 174--188.

Digital Library

[4]

Bradstreet, L., While, L., and Barone, L. 2008. A fast incremental hypervolume algorithm. IEEE Trans. on Evolutionary Computation 12, 6 (December 2008) 714--723.

Digital Library

[5]

Coello, C.A.C. and Sierra, M.R. 2004. A study of the parallelization of a coevolutionary multi-objective evolutionary algorithm. Lecture Notes in Computer Science 2972: MICAI 2004 (April 2004) 688--697.

[6]

Czyzak, P. and Jaszkiewicz, A. 1998. Pareto simulated annealing - A metaheuristic technique for multiple-objective combinatorial optimization. Journal of Multi-Criteria Decision Analysis 7, 1 (January 1998) 34--47.

[7]

Deb, K., Paratop, A., Agarwal, S., and Meyarivan, T. 2002. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. on Evolutionary Computation 6, 2 (April 2002) 182--197.

Digital Library

[8]

Deb, K., Thiele, L., Laumanns, M., and Zitzler, E. 2002. Scalable multi-objective optimization test problems. Proc. of 2002 Congress on Evolutionary Computation (Honolulu, USA, May 12--17, 2002) 825--830.

[9]

Hansen, M.P. and Jaszkiewicz, A. 1998. Evaluating the quality of approximations of the non-dominated set. Technical Report IMM-REP-1998-7, Technical University of Denmark.

[10]

Huband, S., Hingston, P., Barone, L., and While, L. 2006. A review of multiobjective test problems and a scalable test problem toolkit. IEEE Trans. on Evolutionary Computation 10, 5 (October 2006) 477--506.

Digital Library

[11]

Ishibuchi, H., Masuda, H., Tanigaki, Y., and Nojima, Y. 2014. Difficulties in specifying reference points to calculate the inverted generational distance for many-objective optimization problems. Proc. of 2014 IEEE Symposium on Computational Intelligence in Multi-Criteria Decision-Making (Orlando, USA, December 9--12, 2014) 170--177.

[12]

Ishibuchi, H., Masuda, H., Tanigaki, Y., and Nojima, Y. 2015. Modified distance calculation in generational distance and inverted generational distance. Lecture Notes in Computer Science 9019: EMO 2015, Part II (Springer, Berlin, March-April 2015) 110--125.

[13]

Ishibuchi, H., Tsukamoto, N., and Nojima Y. 2008. Evolutionary many-objective optimization: A short review. Proc. of 2008 IEEE Congress on Evolutionary Computation (Hong Kong, China, June 1--6, 2008) 2424--2431.

[14]

Ishibuchi, H., Yoshida, T., and Murata, T. 2003. Balance between genetic search and local search in memetic algorithms for multiobjective permutation flowshop scheduling. IEEE Trans. on Evolutionary Computation 7, 2 (April 2003) 204--223.

Digital Library

[15]

Jiang, S., Ong, Y., Zhang, J., and Feng, L. 2014. Consistencies and contradictions of performance metrics in multiobjective optimization. IEEE Trans. on Cybernetics 44, 12 (December 2014) 2391--2404.

[16]

Knowles, J.D. and Corne, D.W. 2002. On metrics for comparing non-dominated sets. Proc. of 2002 Congress on Evolutionary Computation (Honolulu, May 12--17, 2002) 711--716.

[17]

Li, M., Yang, S., and Liu, X. Shift-based density estimation for Pareto-based algorithms in many-objective optimization. IEEE Trans. on Evolutionary Computation 18, 3 (June 2014) 348--365.

[18]

Russo, L.M.S. and Francisco, A.P. 2014. Quick hypervolume. IEEE Trans. on Evolutionary Computation 18, 4 (August 2014) 481--502.

[19]

Schuetze, O., Esquivel, X., Lara, A., and Coello, C.A.C. 2012. Using the averaged Hausdorff distance as a performance measure in evolutionary multiobjective optimization. IEEE Trans. on Evolutionary Computation 16, 4 (August 2012) 504--522.

Digital Library

[20]

Sierra, M.R. and Coello, C.A.C. 2004. A new multi-objective particle swarm optimizer with improved selection and diversity mechanisms. Technical Report of CINVESTAV-IPN (November 23, 2004).

[21]

Van Veldhuizen D.A. 1999. Multiobjective evolutionary algorithms: Classifications, analyses, and new innovations. Ph.D. Thesis, Graduate School of Engineering, Air Force Institute of Technology, Wright-Patterson AFB, USA (1999)

Digital Library

[22]

von Lücken, C., Barán, B., and Brizuela, C. 2014. A survey on multi-objective evolutionary algorithms for many-objective problems. Computational Optimization and Applications 53, 3 (July 2014) 707--756.

Digital Library

[23]

Wagner, T., Beume, N., and Naujoks, B. 2007. Pareto-, aggregation-, and indicator-based methods in many-objective optimization. Lecture Notes in Computer Science 4403: Evolutionary Multi-Criterion Optimization - EMO 2007, (Springer, Berlin, March 2007), 742--756.

Digital Library

[24]

While, L., Bradstreet, L., and Barone, L. 2012. A fast way of calculating exact hypervolumes. IEEE Trans. on Evolutionary Computation 16, 1 (February 2012) 86--95.

Digital Library

[25]

Yang, S., Li, M., Liu, X., and Zheng, J. 2013. A grid-based evolutionary algorithm for many-objective optimization. IEEE Trans. on Evolutionary Computation 17, 5 (October 2013) 721--736.

Digital Library

[26]

Zhang, Q. and Li, H. 2007. MOEA/D: A multiobjective evolutionary algorithm based on decomposition. IEEE Trans. on Evolutionary Computation 11, 6 (December 2007) 712--731.

Digital Library

[27]

Zitzler, E., Brockhoff, D., and Thiele, L. 2007. The hypervolume indicator revisited: On the design of Pareto-compliant indicators via weighted integration. Lecture Notes in Computer Science 4403: EMO 2007 (Springer, Berlin, March 2007) 862--876.

Digital Library

[28]

Zitzler, E., Laumanns, M., and Thiele, L. 2001. SPEA2: Improving the strength Pareto evolutionary algorithm. TIK-Report, No. 103, ETH Zurich.

[29]

Zitzler, E. and Thiele, L. 1998. Multiobjective optimization using evolutionary algorithms - A comparative case study. Lecture Notes in Computer Science 1498: Parallel Problem Solving from Nature - PPSN V (September 1998) 292--301.

Digital Library

[30]

Zitzler, E. and Thiele, L. 1999. Multiobjective evolutionary algorithms: A comparative case study and the strength Pareto approach. IEEE Trans. on Evolutionary Computation 3, 4 (November 1999) 257--271.

Digital Library

[31]

Zitzler, E., Thiele, L., Laumanns, M., Fonseca, C.M., and da Fonseca, V.G. 2003. Performance assessment of multiobjective optimizers: An analysis and review. IEEE Trans. on Evolutionary Computation 7, 2 (April 2003) 117--132.

Digital Library

Cited By

Li FYang YLiu YLiu YQian M(2025)Bi-Level Model Management Strategy for Solving Expensive Multi-Objective Optimization ProblemsIEEE Transactions on Emerging Topics in Computational Intelligence10.1109/TETCI.2024.34040209:1(332-346)Online publication date: Feb-2025
https://doi.org/10.1109/TETCI.2024.3404020
Wang LWang LJiang QWang ZZhu WWang Z(2025)An adaptive transfer strategy guided by reference vectors for many-objective optimization problemsThe Journal of Supercomputing10.1007/s11227-024-06547-281:1Online publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1007/s11227-024-06547-2
Helbig MEngelbrecht A(2025)Solving many-objective optimisation problems using partial dominanceNeural Computing and Applications10.1007/s00521-023-09145-037:2(653-694)Online publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1007/s00521-023-09145-0
Show More Cited By

Index Terms

A Study on Performance Evaluation Ability of a Modified Inverted Generational Distance Indicator
1. Computing methodologies
  1. Artificial intelligence
    1. Control methods
    2. Search methodologies
2. Theory of computation
  1. Design and analysis of algorithms
    1. Algorithm design techniques
      1. Dynamic programming

Recommendations

An Empirical Assessment of the Properties of Inverted Generational Distance on Multi- and Many-Objective Optimization
EMO 2017: 9th International Conference on Evolutionary Multi-Criterion Optimization - Volume 10173

The inverted generational distanceï źIGD is a metric for assessing the quality of approximations to the Pareto front obtained by multi-objective optimization algorithms. The IGD has become the most commonly used metric in the context of many-objective ...
Effectiveness of scalability improvement attempts on the performance of NSGA-II for many-objective problems
GECCO '08: Proceedings of the 10th annual conference on Genetic and evolutionary computation

Recently a number of approaches have been proposed to improve the scalability of evolutionary multiobjective optimization (EMO) algorithms to many-objective problems. In this paper, we examine the effectiveness of those approaches through computational ...
Combining modified inverted generational distance indicator with reference-vector-guided selection for many-objective optimization
Abstract
The modified inverted generational distance (IGD⁺) indicator has been widely used to handle optimization problems with two or three objectives due to its ability to obtain weak Pareto optimal solutions. However, only using the IGD⁺ indicator ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

GECCO '15: Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation

July 2015

1496 pages

ISBN:9781450334723

DOI:10.1145/2739480

Editor:
Sara Silva
Universidade de Lisboa, Portugal
,
General Chair:
Anna I. Esparcia-Alcázar
Universitat Politècnica de València, Spain

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGEVO: ACM Special Interest Group on Genetic and Evolutionary Computation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 July 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

GECCO '15

Sponsor:

SIGEVO

GECCO '15: Genetic and Evolutionary Computation Conference

July 11 - 15, 2015

Madrid, Spain

Acceptance Rates

GECCO '15 Paper Acceptance Rate 182 of 505 submissions, 36%;

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

70
Total Citations
View Citations
496
Total Downloads

Downloads (Last 12 months)23
Downloads (Last 6 weeks)0

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Li FYang YLiu YLiu YQian M(2025)Bi-Level Model Management Strategy for Solving Expensive Multi-Objective Optimization ProblemsIEEE Transactions on Emerging Topics in Computational Intelligence10.1109/TETCI.2024.34040209:1(332-346)Online publication date: Feb-2025
https://doi.org/10.1109/TETCI.2024.3404020
Wang LWang LJiang QWang ZZhu WWang Z(2025)An adaptive transfer strategy guided by reference vectors for many-objective optimization problemsThe Journal of Supercomputing10.1007/s11227-024-06547-281:1Online publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1007/s11227-024-06547-2
Helbig MEngelbrecht A(2025)Solving many-objective optimisation problems using partial dominanceNeural Computing and Applications10.1007/s00521-023-09145-037:2(653-694)Online publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1007/s00521-023-09145-0
Castán Rocha JSantiago AGarcía-Ruiz ATerán-Villanueva JMartínez STreviño Berrones M(2024)Pareto Approximation Empirical Results of Energy-Aware Optimization for Precedence-Constrained Task Scheduling Considering Switching Off Completely Idle MachinesMathematics10.3390/math1223373312:23(3733)Online publication date: 27-Nov-2024
https://doi.org/10.3390/math12233733
Silva-Rubio SSalgueiro YMora-Meliá DGutiérrez-Bahamondes J(2024)Improving Water and Energy Resource Management: A Comparative Study of Solution Representations for the Pump Scheduling Optimization ProblemMathematics10.3390/math1213199412:13(1994)Online publication date: 27-Jun-2024
https://doi.org/10.3390/math12131994
Binte Azhar NGunawan ACheng SLeonardi E(2024)Comparison of Evolutionary Algorithms: A Case Study on the Multi-Objective Carbon-Aware Mine Planning*2024 IEEE 20th International Conference on Automation Science and Engineering (CASE)10.1109/CASE59546.2024.10711825(962-969)Online publication date: 28-Aug-2024
https://doi.org/10.1109/CASE59546.2024.10711825
Xiong JLiu GGao ZZhou CHu PBao Q(2024)A many-objective evolutionary algorithm based on learning assessment and mapping guidance of historical superior informationJournal of Computational Design and Engineering10.1093/jcde/qwae02211:2(194-229)Online publication date: 11-Mar-2024
https://doi.org/10.1093/jcde/qwae022
Zhang QYang NWu YTang Z(2024)SEA: Many-objective evolutionary algorithm with selection evolution strategyExpert Systems with Applications10.1016/j.eswa.2024.124226252(124226)Online publication date: Oct-2024
https://doi.org/10.1016/j.eswa.2024.124226
Yuste JPardo EDuarte AHao J(2024)Multi-objective general variable neighborhood search for software maintainability optimizationEngineering Applications of Artificial Intelligence10.1016/j.engappai.2024.108593133(108593)Online publication date: Jul-2024
https://doi.org/10.1016/j.engappai.2024.108593
Segura-Ortiz AGarcía-Nieto JAldana-Montes JNavas-Delgado I(2024)Multi-objective context-guided consensus of a massive array of techniques for the inference of Gene Regulatory NetworksComputers in Biology and Medicine10.1016/j.compbiomed.2024.108850179(108850)Online publication date: Sep-2024
https://doi.org/10.1016/j.compbiomed.2024.108850
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten