Skip to main content
SpringerLink
Log in

Performance Evaluation of Multi-objective Evolutionary Algorithms Using Artificial and Real-world Problems

  • Conference paper
  • First Online:
Evolutionary Multi-Criterion Optimization (EMO 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13970))

Included in the following conference series:

Abstract

Performance of evolutionary multi-objective optimization (EMO) algorithms is usually evaluated using artificial test problems such as DTLZ and WFG. Every year, new EMO algorithms with high performance on those test problems are proposed. One question is whether they also work well on real-world problems. In this paper, we try to find an answer to this question by examining the performance of ten EMO algorithms including both well-known representative algorithms and recently-proposed new algorithms. First, those algorithms are applied to five artificial test suites (DTLZ, WFG, Minus-DTLZ, Minus-WFG and MaF) and three real-world problem suites. The performance of each algorithm is evaluated by the hypervolume indicator. Next, the ranking of the ten EMO algorithms is created for each problem suite. That is, eight different rankings are obtained (each ranking is for each problem suite). Then, the eight different rankings are visually compared to answer our research question. The distance between two rankings is also calculated to support visual comparison results. Our experimental results show that similar rankings of the ten EMO algorithms are obtained for the three real-world problem suites and Minus-WFG. It is also shown that the ranking for each of the three real-world problem suites is clearly different from their ranking for DTLZ.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Deb, K., Thiele, L., Laumanns, M., Zitzler, E.: Scalable multi-objective optimization test problems. Proc. of IEEE CEC 2002, 825–830 (2002)

    MATH  Google Scholar 

  2. Huband, S., Hingston, P., Barone, L., While, L.: A review of multiobjective test problems and a scalable test problem toolkit. IEEE Trans. on Evolutionary Computation 10(5), 477–506 (2006)

    Article  MATH  Google Scholar 

  3. Ishibuchi, H., Setoguchi, Y., Masuda, H., Nojima, Y.: Performance of decomposition-based many-objective algorithms strongly depends on Pareto front shapes. IEEE Trans. on Evolutionary Computation 21(2), 169–190 (2017)

    Article  Google Scholar 

  4. Cheng, R., et al.: A benchmark test suite for evolutionary many-objective optimization. Complex & Intelligent Syst. 3(1), 67–81 (2017)

    Article  Google Scholar 

  5. Ishibuchi, H., He, L., Shang, K.: Regular Pareto front shape is not realistic. Proc. of IEEE CEC 2019, 2035–2042 (2019)

    Google Scholar 

  6. Zhu, Q., et al.: An elite gene guided reproduction operator for many-objective optimization. IEEE Trans. on Cybernetics 51(2), 765–778 (2021)

    Article  MathSciNet  Google Scholar 

  7. Liang, Z., Hu, K., Ma, X., Zhu, Z.: A many-objective evolutionary algorithm based on a two-round selection strategy. IEEE Trans. on Cybernetics 51(3), 1417–1429 (2021)

    Article  Google Scholar 

  8. Li, L., Chang, L., Gu, T., Sheng, W., Wang, W.: On the norm of dominant difference for many-objective particle swarm optimization. IEEE Trans. on Cybernetics 51(4), 2055–2067 (2021)

    Article  Google Scholar 

  9. Liang, Z., Luo, T., Hu, K., Ma, X., Zhu, Z.: An indicator-based many-objective evolutionary algorithm with boundary protection. IEEE Trans. on Cybernetics 51(9), 4553–4566 (2021)

    Article  Google Scholar 

  10. Liu, Y., Zhu, N., Li, M.: Solving many-objective optimization problems by a Pareto-based evolutionary algorithm with preprocessing and a penalty mechanism. IEEE Trans. on Cybernetics 51(11), 5585–5594 (2021)

    Article  Google Scholar 

  11. Zhang, K., Xu, Z., Xie, S., Yen, G.G.: Evolution strategy-based many-objective evolutionary algorithm through vector equilibrium. IEEE Trans. on Cybernetics 51(11), 5455–5467 (2021)

    Article  Google Scholar 

  12. Moreno, J., Rodriguez, D., Nebro, A.J., Lozano, J.A.: Merge nondominated sorting algorithm for many-objective optimization. IEEE Trans. on Cybernetics 51(12), 6154–6164 (2021)

    Article  Google Scholar 

  13. Chen, L., Deb, K., Liu, H.-L., Zhang, Q.: Effect of objective normalization and penalty parameter on penalty boundary intersection decomposition-based evolutionary many-objective optimization algorithms. Evol. Comput. 29(1), 157–186 (2021)

    Article  Google Scholar 

  14. Song, Z., Wang, H., He, C., Jin, Y.: A Kriging-assisted two-archive evolutionary algorithm for expensive many-objective optimization. IEEE Trans. on Evolutionary Computation 25(6), 1013–1027 (2021)

    Article  Google Scholar 

  15. Liu, S.-C., Zhan, Z.-H., Tan, K.C., Zhang, J.: A multiobjective framework for many-objective optimization. IEEE Trans. on Cybernetics 52(12), 13654–13668 (2022)

    Article  Google Scholar 

  16. Wang, Z., Zhang, Q., Ong, Y.-S., Yao, S., Liu, H., Luo, J.: Choose appropriate subproblems for collaborative modeling in expensive multiobjective optimization. IEEE Trans. on Cybernetics (Early Access Paper)

    Google Scholar 

  17. Sonoda, T., Nakata, M.: Multiple classifiers-assisted evolutionary algorithm based on decomposition for high-dimensional multi-objective problems. IEEE Trans. on Evolutionary Computation 26(6), 1581–1595 (2022)

    Article  Google Scholar 

  18. Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. on Evolutionary Computation 6(2), 182–197 (2002)

    Article  Google Scholar 

  19. Zhang, Q., Li, H.: MOEA/D: a multiobjective evolutionary algorithm based on decomposition. IEEE Trans. on Evolutionary Computation 11(6), 712–731 (2007)

    Article  Google Scholar 

  20. Beume, N., Naujoks, B., Emmerich, M.: SMS-EMOA: multiobjective selection based on dominated hypervolume. European J. of Operational Res. 181(3), 1653–1669 (2007)

    Article  MATH  Google Scholar 

  21. Bader, J., Zitzler, E.: HypE: an algorithm for fast hypervolume-based many-objective optimization. Evol. Comput. 19(1), 45–76 (2011)

    Article  Google Scholar 

  22. Deb, K., Jain, H.: An evolutionary many-objective optimization algorithm using reference-point-based non-dominated sorting approach, part I: solving problems with box constraints. IEEE Trans. on Evolutionary Computation 18(4), 577–601 (2014)

    Article  Google Scholar 

  23. Li, K., Deb, K., Zhang, Q., Kwong, S.: An evolutionary many-objective optimization algorithm based on dominance and decomposition. IEEE Trans. on Evolutionary Computation 19(5), 694–716 (2015)

    Article  Google Scholar 

  24. Cheng, R., Jin, Y., Olhofer, M., Sendhoff, B.: A reference vector guided evolutionary algorithm for many-objective optimization. IEEE Trans. on Evolutionary Computation 20(5), 773–791 (2016)

    Article  Google Scholar 

  25. Tian, Y., Zhang, X., Wang, C., Jin, Y.: An evolutionary algorithm for large-scale sparse multi-objective optimization problems. IEEE Trans. on Evolutionary Computation 24(2), 380–393 (2020)

    Article  Google Scholar 

  26. Liu, Y., Ishibuchi, H., Masuyama, N., Nojima, Y.: Adapting reference vectors and scalarizing functions by growing neural gas to handle irregular Pareto fronts. IEEE Trans. on Evolutionary Computation 24(3), 439–453 (2020)

    Google Scholar 

  27. Shang, K., Ishibuchi, H.: A new hypervolume-based evolutionary algorithm for many-objective optimization. IEEE Trans. on Evolutionary Computation 24(5), 839–852 (2020)

    Article  Google Scholar 

  28. Yuan, J., Liu, H.-L., Gu, F., Zhang, Q., He, Z.: Investigating the properties of indicators and an evolutionary many-objective algorithm using promising regions. IEEE Trans. on Evolutionary Computation 25(1), 75–86 (2021)

    Article  Google Scholar 

  29. Tanabe, R., Ishibuchi, H.: An easy-to-use real-world multi-objective optimization problem suite. Applied Soft Computing 89, 106078 (2020)

    Google Scholar 

  30. He, C., Tian, Y., Wang, H., Jin, Y.: A repository of real-world datasets for data-driven evolutionary multiobjective optimization. Complex & Intelligent Systems 6(1), 189–197 (2019)

    Article  Google Scholar 

  31. Kumar, A., et al.: A benchmark-suite of real-world constrained multi-objective optimization problems and some baseline results. Swarm and Evolutionary Computation 67, 100961 (2021)

    Google Scholar 

  32. Horn, J., Nafpliotis, N., Goldberg, D.E.: A niched Pareto genetic algorithm for multi-objective optimization. Proc. of IEEE ICEC 1994, 82–87 (1994)

    Google Scholar 

  33. Srinivas, N., Deb, K.: Multiobjective optimization using nondominated sorting in genetic algorithms. Evol. Comput. 2(3), 221–248 (1994)

    Article  Google Scholar 

  34. Ishibuchi, H., Masuda, H., Tanigaki, Y., Nojima, Y.: Review of coevolutionary developments of evolutionary multi-objective and many-objective algorithms and test problems. Proc. of IEEE SSCI 2014, 178–185 (2014)

    Google Scholar 

  35. Zitzler, E., Deb, K., Thiele, L.: Comparison of multiobjective evolutionary algorithms: empirical results. Evol. Comput. 8(2), 173–195 (2000)

    Article  Google Scholar 

  36. Ikeda, K., Kita, H., Kobayashi, S.: Failure of Pareto-based MOEAs: does non-dominated really mean near to optimal? Proc. of CEC 2001, 957–962 (2001)

    Google Scholar 

  37. Ishibuchi, H., Matsumoto, T., Masuyama, N., Nojima, Y.: Effects of dominance resistant solutions on the performance of evolutionary multi-objective and many-objective algorithms. Proc. of GECCO 2020, 507–515 (2020)

    Article  Google Scholar 

  38. Wagner, T., Beume, N., Naujoks, B.: Pareto-, aggregation-, and indicator-based methods in many-objective optimization. In: Obayashi, S., Deb, K., Poloni, C., Hiroyasu, T., Murata, T. (eds.) EMO 2007. LNCS, vol. 4403, pp. 742–756. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-70928-2_56

    Chapter  Google Scholar 

  39. Mostaghim, S., Schmeck, H.: Distance based ranking in many-objective particle swarm optimization. In: Rudolph, G., Jansen, T., Beume, N., Lucas, S., Poloni, C. (eds.) PPSN 2008. LNCS, vol. 5199, pp. 753–762. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-87700-4_75

    Chapter  Google Scholar 

  40. Tian, Y., Cheng, R., Zhang, X., Jin, Y.: PlatEMO: a MATLAB platform for evolutionary multi-objective optimization. IEEE Comput. Intell. Mag. 12(4), 73–87 (2017)

    Article  Google Scholar 

  41. Ishibuchi, H., Matsumoto, T., Masuyama, N., Nojima, Y.: Many-objective problems are not always difficult for Pareto dominance-based evolutionary algorithms. Proc. of ECAI 2020, 291–299 (2020)

    Google Scholar 

  42. Ishibuchi, H., Imada, R., Setoguchi, Y., Nojima, Y.: How to specify a reference point in hypervolume calculation for fair performance comparison. Evol. Comput. 26(3), 411–440 (2018)

    Article  Google Scholar 

  43. He, L., Shang, K., Nan, Y., Ishibuchi, H., Srinivasan, D.: Relation between objective space normalization and weight vector scaling in decomposition-based multi-objective evolutionary algorithms. IEEE Trans. on Evolutionary Computation (Early Access Paper)

    Google Scholar 

  44. Ishibuchi, H., Imada, R., Masuyama, N., Nojima, Y.: Use of two reference points in hypervolume-based evolutionary multiobjective optimization algorithms. In: Auger, A., Fonseca, C.M., Lourenço, N., Machado, P., Paquete, L., Whitley, D. (eds.) PPSN 2018. LNCS, vol. 11101, pp. 384–396. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-99253-2_31

    Chapter  Google Scholar 

  45. Ishibuchi, H., Pang, L.M., Shang, K.: Difficulties in fair performance comparison of multi-objective evolutionary algorithms. IEEE Comput. Intell. Mag. 17(1), 86–101 (2022)

    Article  Google Scholar 

  46. Cheng, R., Jin, Y., Olhofer, M., Sendhoff, B.: Test problems for large-scale multiobjective and many-objective optimization. IEEE Trans. on Cybernetics 47(12), 4108–4121 (2017)

    Article  Google Scholar 

  47. Pang, L.M., Ishibuchi, H., Shang, K.: Counterintuitive experimental results in evolutionary large-scale multi-objective optimization. IEEE Trans. on Evolutionary Computation 26(6), 1609–1616 (2022)

    Article  Google Scholar 

Download references

Acknowledgement

This work was supported by National Natural Science Foundation of China (Grant No. 61876075), Guangdong Provincial Key Laboratory (Grant No. 2020B121201001), the Program for Guangdong Introducing Innovative and Enterpreneurial Teams (Grant No. 2017ZT07X386), The Stable Support Plan Program of Shenzhen Natural Science Fund (Grant No. 20200925174447003), Shenzhen Science and Technology Program (Grant No. KQTD2016112514355531).

Author information

Authors and Affiliations

  1. Guangdong Provincial Key Laboratory of Brain-Inspired Intelligent Computation, Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, 518055, China

    Hisao Ishibuchi, Yang Nan & Lie Meng Pang

Authors
  1. Hisao Ishibuchi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yang Nan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lie Meng Pang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hisao Ishibuchi .

Editor information

Editors and Affiliations

  1. Leiden University, Leiden, The Netherlands

    Michael Emmerich

  2. Leiden University, Leiden, The Netherlands

    André Deutz

  3. Leiden University, Leiden, The Netherlands

    Hao Wang

  4. Leiden University, Leiden, The Netherlands

    Anna V. Kononova

  5. TH Köln, Gummersbach, Germany

    Boris Naujoks

  6. University of Exeter, Exeter, UK

    Ke Li

  7. University of Jyvaskyla, Jyvaskyla, Finland

    Kaisa Miettinen

  8. De Montfort University, Leicester, UK

    Iryna Yevseyeva

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ishibuchi, H., Nan, Y., Pang, L.M. (2023). Performance Evaluation of Multi-objective Evolutionary Algorithms Using Artificial and Real-world Problems. In: Emmerich, M., et al. Evolutionary Multi-Criterion Optimization. EMO 2023. Lecture Notes in Computer Science, vol 13970. Springer, Cham. https://doi.org/10.1007/978-3-031-27250-9_24

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-27250-9_24

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-27249-3

  • Online ISBN: 978-3-031-27250-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation