Skip to main content
Springer Nature Link
Log in

Nullifying the Inherent Bias of Non-invariant Exploratory Landscape Analysis Features

  • Conference paper
  • First Online:
Applications of Evolutionary Computation (EvoApplications 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13989))

Abstract

Exploratory landscape analysis (ELA) in single-objective black-box optimization relies on a comprehensive and large set of numerical features characterizing problem instances. Those foster problem understanding and serve as basis for constructing automated algorithm selection models choosing the best suited algorithm for a problem at hand based on the aforementioned features computed prior to optimization. This work specifically points to the sensitivity of a substantial proportion of these features to absolute objective values, i.e., we observe a lack of shift and scale invariance. We show that this unfortunately induces bias within automated algorithm selection models, an overfitting to specific benchmark problem sets used for training and thereby hinders generalization capabilities to unseen problems. We tackle these issues by presenting an appropriate objective normalization to be used prior to ELA feature computation and empirically illustrate the respective effectiveness focusing on the BBOB benchmark set.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Bossek, J., Doerr, C., Kerschke, P.: Initial design strategies and their effects on sequential model-based optimization: an exploratory case study based on BBOB. In: Proceedings of the 2020 Genetic and Evolutionary Computation Conference, pp. 778–786. GECCO ’20, Association for Computing Machinery, New York, NY, USA (2020). https://doi.org/10.1145/3377930.3390155

  2. Hansen, N., Auger, A., Ros, R., Mersmann, O., Tušar, T., Brockhoff, D.: COCO: a platform for comparing continuous optimizers in a black-box setting. Optim. Methods Softw. 36, 114–144 (2021). https://doi.org/10.1080/10556788.2020.1808977

    Article  MathSciNet  MATH  Google Scholar 

  3. Hansen, N., Auger, A., Finck, S., Ros, R.: Real-parameter black-box optimization benchmarking 2010: experimental setup. Research Report RR-7215, INRIA (2010). https://hal.inria.fr/inria-00462481

  4. Hansen, N., Finck, S., Ros, R., Auger, A.: Real-parameter black-box optimization benchmarking 2009: noiseless functions definitions. Tech. Rep. RR-6829, INRIA (2009). https://hal.inria.fr/inria-00362633/document

  5. Heins, J., Bossek, J., Pohl, J., Seiler, M., Trautmann, H., Kerschke, P.: A study on the effects of normalized TSP features for automated algorithm selection. Theor. Comput. Sci. 940, 123–145 (2023). https://doi.org/10.1016/j.tcs.2022.10.019

    Article  MathSciNet  MATH  Google Scholar 

  6. Kerschke, P., Hoos, H.H., Neumann, F., Trautmann, H.: Automated algorithm selection: survey and perspectives. Evol. Comput. 27(1), 3–45 (2019). https://doi.org/10.1162/evco_a_00242

    Article  Google Scholar 

  7. Kerschke, P., Preuss, M., Wessing, S., Trautmann, H.: Detecting funnel structures by means of exploratory landscape analysis. In: Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation, pp. 265–272. GECCO ’15, Association for Computing Machinery, New York, NY, USA (2015). https://doi.org/10.1145/2739480.2754642

  8. Kerschke, P., Trautmann, H.: Automated algorithm selection on continuous black-box problems by combining exploratory landscape analysis and machine learning. Evol. Comput. 27(1), 99–127 (2019). https://doi.org/10.1162/evco_a_00236

    Article  Google Scholar 

  9. Kerschke, P., Trautmann, H.: Comprehensive feature-based landscape analysis of continuous and constrained optimization problems using the r-package flacco. In: Bauer, N., Ickstadt, K., Lübke, K., Szepannek, G., Trautmann, H., Vichi, M. (eds.) Applications in Statistical Computing. SCDAKO, pp. 93–123. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-25147-5_7

    Chapter  Google Scholar 

  10. Lunacek, M., Whitley, D.: The dispersion metric and the CMA evolution strategy. In: Proceedings of the 8th Annual Conference on Genetic and Evolutionary Computation. p. 477–484. GECCO ’06, Association for Computing Machinery, New York, NY, USA (2006). https://doi.org/10.1145/1143997.1144085

  11. Mersmann, O., Bischl, B., Trautmann, H., Preuss, M., Weihs, C., Rudolph, G.: Exploratory landscape analysis. In: Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation. p. 829–836. GECCO ’11, Association for Computing Machinery, New York, NY, USA (2011). https://doi.org/10.1145/2001576.2001690

  12. Muñoz Acosta, M.A., Kirley, M., Halgamuge, S.K.: Exploratory landscape analysis of continuous space optimization problems using information content. IEEE Trans. Evol. Comput. (TEVC) 19(1), 74–87 (2015). https://doi.org/10.1109/TEVC.2014.2302006

    Article  Google Scholar 

  13. Muñoz, M.A., Sun, Y., Kirley, M., Halgamuge, S.K.: Algorithm selection for black-box continuous optimization problems: a survey on methods and challenges. Inf. Sci. 317, 224–245 (2015). https://doi.org/10.1016/j.ins.2015.05.010

    Article  Google Scholar 

  14. Pedregosa, F., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)

    MathSciNet  MATH  Google Scholar 

  15. Prager, R.P.: pflacco: The R-Package flacco in Native Python Code (2022). https://github.com/Reiyan/pflacco, Python Package v1.1.0

  16. Prager, R.P., Seiler, M.V., Trautmann, H., Kerschke, P.: Automated algorithm selection in single-objective continuous optimization: a comparative study of deep learning and landscape analysis methods. In: Rudolph, G., Kononova, A.V., Aguirre, H., Kerschke, P., Ochoa, G., Tušar, T. (eds.) Parallel Problem Solving from Nature - PPSN XVII, pp. 3–17. Springer International Publishing, Cham (2022). https://doi.org/10.1007/978-3-031-14714-2_1

  17. Renau, Q., Doerr, C., Dreo, J., Doerr, B.: Exploratory landscape analysis is strongly sensitive to the sampling strategy. In: Bäck, T., et al. (eds.) PPSN 2020. LNCS, vol. 12270, pp. 139–153. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58115-2_10

    Chapter  Google Scholar 

  18. Renau, Q., Dreo, J., Doerr, C., Doerr, B.: Expressiveness and robustness of landscape features. In: Proceedings of the Genetic and Evolutionary Computation Conference Companion. p. 2048–2051. GECCO ’19, Association for Computing Machinery, New York, NY, USA (2019). https://doi.org/10.1145/3319619.3326913

  19. Renau, Q., Dreo, J., Doerr, C., Doerr, B.: Towards explainable exploratory landscape analysis: extreme feature selection for classifying BBOB functions. In: Castillo, P.A., Jiménez Laredo, J.L. (eds.) EvoApplications 2021. LNCS, vol. 12694, pp. 17–33. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-72699-7_2

    Chapter  Google Scholar 

  20. Škvorc, U., Eftimov, T., Korošec, P.: Understanding the problem space in single-objective numerical optimization using exploratory landscape analysis. Appl. Soft Comput. 90, 106138 (2020). https://doi.org/10.1016/j.asoc.2020.106138

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Data Science: Statistics and Optimization, University of Münster, Münster, Germany

    Raphael Patrick Prager & Heike Trautmann

  2. Data Management and Biometrics, University of Twente, Enschede, The Netherlands

    Heike Trautmann

Authors
  1. Raphael Patrick Prager
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Heike Trautmann
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Raphael Patrick Prager .

Editor information

Editors and Affiliations

  1. University of Coimbra, Coimbra, Portugal

    João Correia

  2. University of York, York, UK

    Stephen Smith

  3. Al Hussein Technical University, Amman, Jordan

    Raneem Qaddoura

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Prager, R.P., Trautmann, H. (2023). Nullifying the Inherent Bias of Non-invariant Exploratory Landscape Analysis Features. In: Correia, J., Smith, S., Qaddoura, R. (eds) Applications of Evolutionary Computation. EvoApplications 2023. Lecture Notes in Computer Science, vol 13989. Springer, Cham. https://doi.org/10.1007/978-3-031-30229-9_27

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-30229-9_27

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-30228-2

  • Online ISBN: 978-3-031-30229-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics