Skip to main content
Springer Nature Link
Log in

Feature-Based Benchmarking of Distance-Based Multi/Many-objective Optimisation Problems: A Machine Learning Perspective

  • Conference paper
  • First Online:
Evolutionary Multi-Criterion Optimization (EMO 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13970))

Included in the following conference series:

  • 1381 Accesses

Abstract

We consider the application of machine learning techniques to gain insights into the effect of problem features on algorithm performance, and to automate the task of algorithm selection for distance-based multi- and many-objective optimisation problems. This is the most extensive benchmark study of such problems to date. The problem features can be set directly by the problem generator, and include e.g. the number of variables, objectives, local fronts, and disconnected Pareto sets. Using 945 problem configurations (leading to \(28\,350\) instances) of varying complexity, we find that the problem features and the available optimisation budget (i) affect the considered algorithms (NSGA-II, IBEA, MOEA/D, and random search) in different ways and that (ii) it is possible to recommend a relevant algorithm based on problem features.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    Available in Matlab (https://github.com/fieldsend/DBMOPP_generator), and in Python (https://github.com/industrial-optimization-group/desdeo-problem/tree/master/desdeo_problem/testproblems/DBMOPP).

  2. 2.

    \(1\,000\) members drawn from the Pareto front plus all non-dominated points found by the union of the algorithms’ approximation sets for each instance. The reference point for hypervolume was \(1.1 \times \text {maximum of objective values on the Pareto front}\) and estimated via Monte Carlo [7] with \(50\,000\) samples for \(4+\) objectives.

References

  1. Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001). https://doi.org/10.1023/a:1010933404324

    Article  MATH  Google Scholar 

  2. Breiman, L., Friedman, J., Stone, C.J., Olshen, R.A.: Classification and Regression Trees. Taylor & Francis, Andover (1984)

    MATH  Google Scholar 

  3. Brockhoff, D., Tusar, T., Auger, A., Hansen, N.: Using well-understood single-objective functions in multiobjective black-box optimization test suites. arXiv CoRR (2016). https://doi.org/10.48550/arxiv.1604.00359

  4. Carnell, R.: LHS: Latin hypercube samples (2022). r package version 1.1.5

    Google Scholar 

  5. Deb, K., Prarap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evol. Comput. 6, 182–197 (2002)

    Article  Google Scholar 

  6. Deb, K., Thiele, L., Laumanns, M., Zitzler, E.: Scalable test problems for evolutionary multiobjective optimization. In: Abraham, A., Jain, L., Goldberg, R. (eds.) Evolutionary Multiobjective Optimization: Theoretical Advances and Applications, pp. 105–145. Springer, London (2005). https://doi.org/10.1007/1-84628-137-7_6

    Chapter  MATH  Google Scholar 

  7. Everson, R.M., Fieldsend, J.E., Singh, S.: Full elite sets for multi-objective optimisation. In: Parmee, I.C. (ed.) Adaptive Computing in Design and Manufacture V, pp. 343–354. Springer, London (2002). https://doi.org/10.1007/978-0-85729-345-9_29

    Chapter  Google Scholar 

  8. Fieldsend, J.E., Alyahya, K.: Visualising the landscape of multi-objective problems using local optima networks. In: Proceedings of the Genetic and Evolutionary Computation Conference Companion, pp. 1421–1429 (2019)

    Google Scholar 

  9. Fieldsend, J.E., Chugh, T., Allmendinger, R., Miettinen, K.: A feature rich distance-based many-objective visualisable test problem generator. In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 541–549 (2019)

    Google Scholar 

  10. Fieldsend, J.E., Chugh, T., Allmendinger, R., Miettinen, K.: A visualizable test problem generator for many-objective optimization. IEEE Trans. Evol. Comput. 26(1), 1–11 (2022)

    Article  Google Scholar 

  11. Huband, S., Hingston, P., Barone, L., While, L.: A review of multiobjective test problems and a scalable test problem toolkit. IEEE Trans. Evol. Comput. 10(5), 477–506 (2006)

    Article  MATH  Google Scholar 

  12. Kerschke, P., Hoos, H.H., Neumann, F., Trautmann, H.: Automated algorithm selection: survey and perspectives. Evol. Comput. 27(1), 3–45 (2019)

    Article  Google Scholar 

  13. Köppen, M., Vicente-Garcia, R., Nickolay, B.: Fuzzy-pareto-dominance and its application in evolutionary multi-objective optimization. In: Coello Coello, C.A., Hernández Aguirre, A., Zitzler, E. (eds.) EMO 2005. LNCS, vol. 3410, pp. 399–412. Springer, Heidelberg (2005). https://doi.org/10.1007/978-3-540-31880-4_28

    Chapter  MATH  Google Scholar 

  14. Köppen, M., Yoshida, K.: Substitute distance assignments in NSGA-II for handling many-objective optimization problems. In: Obayashi, S., Deb, K., Poloni, C., Hiroyasu, T., Murata, T. (eds.) EMO 2007. LNCS, vol. 4403, pp. 727–741. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-70928-2_55

    Chapter  Google Scholar 

  15. Kuhn, M.: Building predictive models in R using the caret package. J. Stat. Softw. Art. 28(5), 1–26 (2008)

    Google Scholar 

  16. Liaw, A., Wiener, M.: Classification and regression by randomforest. R News 2(3), 18–22 (2002)

    Google Scholar 

  17. Liefooghe, A., Daolio, F., Verel, S., Derbel, B., Aguirre, H., Tanaka, K.: Landscape-aware performance prediction for evolutionary multiobjective optimization. IEEE Trans. Evol. Comput. 24(6), 1063–1077 (2019)

    Article  Google Scholar 

  18. Liefooghe, A., Derbel, B., Verel, S., López-Ibáñez, M., Aguirre, H., Tanaka, K.: On pareto local optimal solutions networks. In: Auger, A., Fonseca, C.M., Lourenço, N., Machado, P., Paquete, L., Whitley, D. (eds.) PPSN 2018. LNCS, vol. 11102, pp. 232–244. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-99259-4_19

    Chapter  Google Scholar 

  19. Liefooghe, A., Verel, S., Lacroix, B., Zăvoianu, A.C., McCall, J.: Landscape features and automated algorithm selection for multi-objective interpolated continuous optimisation problems. In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 421–429 (2021)

    Google Scholar 

  20. Malan, K.M.: A survey of advances in landscape analysis for optimisation. Algorithms 14(2), 40 (2021)

    Article  MathSciNet  Google Scholar 

  21. Mersmann, O., Bischl, B., Trautmann, H., Preuss, M., Weihs, C., Rudolph, G.: Exploratory landscape analysis. In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 829–836 (2011)

    Google Scholar 

  22. Muñoz, M.A., Villanova, L., Baatar, D., Smith-Miles, K.: Instance spaces for machine learning classification. Mach. Learn. 107(1), 109–147 (2018). https://doi.org/10.1007/s10994-017-5629-5

    Article  MathSciNet  MATH  Google Scholar 

  23. R Core Team: R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria (2020)

    Google Scholar 

  24. Rice, J.R.: The algorithm selection problem. In: Advances in Computers, vol. 15, pp. 65–118. Elsevier (1976)

    Google Scholar 

  25. Shand, C., Allmendinger, R., Handl, J., Webb, A., Keane, J.: HAWKS: Evolving challenging benchmark sets for cluster analysis. IEEE Trans. Evol. Comput. 26(6), 1206–1220 (2022)

    Article  Google Scholar 

  26. Smith-Miles, K., Lopes, L.: Measuring instance difficulty for combinatorial optimization problems. Compute. Oper. Res. 39(5), 875–889 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  27. Smith-Miles, K.A.: Cross-disciplinary perspectives on meta-learning for algorithm selection. ACM Comput. Surv. 41(1), 1–25 (2009)

    Article  Google Scholar 

  28. Therneau, T., Atkinson, B.: rpart: Recursive partitioning and regression trees (2022). R package version 4.1.16

    Google Scholar 

  29. Wickham, H.: ggplot2: Elegant Graphics for Data Analysis. Springer, New York (2016)

    Book  MATH  Google Scholar 

  30. Zăvoianu, A.-C., Lacroix, B., McCall, J.: Comparative run-time performance of evolutionary algorithms on multi-objective interpolated continuous optimisation problems. In: Bäck, T., et al. (eds.) PPSN 2020. LNCS, vol. 12269, pp. 287–300. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58112-1_20

    Chapter  Google Scholar 

  31. Zhang, Q., Li, H.: MOEA/D: a multiobjective evolutionary algorithm based on decomposition. IEEE Trans. Evol. Comput. 11, 712–731 (2007)

    Article  Google Scholar 

  32. Zitzler, E., Künzli, S.: Indicator-based selection in multiobjective search. In: Yao, X., et al. (eds.) PPSN 2004. LNCS, vol. 3242, pp. 832–842. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-30217-9_84

    Chapter  Google Scholar 

Download references

Acknowledgements

This research is part of the thematic research area DEMO, Decision Analytics utilising Causal Models and Multiobjective Optimisation, jyu.fi/demo, at the University of Jyvaskyla.

Author information

Authors and Affiliations

  1. Univ. Lille, CNRS, Inria, Centrale Lille, UMR 9189 CRIStAL, 59000, Lille, France

    Arnaud Liefooghe

  2. Univ. Littoral Côte d’Opale, LISIC, 62100, Calais, France

    Sébastien Verel

  3. University of Exeter, Exeter, EX4 4QD, UK

    Tinkle Chugh & Jonathan Fieldsend

  4. The University of Manchester, Manchester, M15 6PB, UK

    Richard Allmendinger

  5. University of Jyvaskyla, Faculty of Information Technology, FI-40014, University of Jyvaskyla, Finland

    Kaisa Miettinen

Authors
  1. Arnaud Liefooghe
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sébastien Verel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tinkle Chugh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jonathan Fieldsend
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Richard Allmendinger
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Kaisa Miettinen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Arnaud Liefooghe .

Editor information

Editors and Affiliations

  1. Leiden University, Leiden, The Netherlands

    Michael Emmerich

  2. Leiden University, Leiden, The Netherlands

    André Deutz

  3. Leiden University, Leiden, The Netherlands

    Hao Wang

  4. Leiden University, Leiden, The Netherlands

    Anna V. Kononova

  5. TH Köln, Gummersbach, Germany

    Boris Naujoks

  6. University of Exeter, Exeter, UK

    Ke Li

  7. University of Jyvaskyla, Jyvaskyla, Finland

    Kaisa Miettinen

  8. De Montfort University, Leicester, UK

    Iryna Yevseyeva

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Liefooghe, A., Verel, S., Chugh, T., Fieldsend, J., Allmendinger, R., Miettinen, K. (2023). Feature-Based Benchmarking of Distance-Based Multi/Many-objective Optimisation Problems: A Machine Learning Perspective. In: Emmerich, M., et al. Evolutionary Multi-Criterion Optimization. EMO 2023. Lecture Notes in Computer Science, vol 13970. Springer, Cham. https://doi.org/10.1007/978-3-031-27250-9_19

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-27250-9_19

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-27249-3

  • Online ISBN: 978-3-031-27250-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics