Abstract
Problem understanding and the ability to assign problems to distinct classes can improve the usability of metaheuristics. A popular problem-independent method for the characterization of optimization problems is exploratory landscape analysis (ELA). It consists of a sequence of operations that describe the hypersurfaces formed by fitness and other characteristic properties of the problem solutions on the basis of a limited number of samples. Sampling is the initial step of ELA that selects a limited number of candidate solutions for which are the characteristic properties evaluated. The solutions and the computed properties serve as the main inputs for the rest of ELA. In this work, we study the impact of different sampling strategies on machine learning-based classification of bi-objective problems on the basis of FLACCO features. A series of computational experiments demonstrates that different sampling strategies affect the value of the resulting landscape features, their suitability for problem classification, and the overhead of the sampling process. An in-depth analysis of the results also shows the relationship between classification accuracy and the structure of the training data set.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Andova, A., Vodopija, A., Krömer, P., Uher, V., Tušar, T., Filipič, B.: Initial results in predicting high-level features of constrained multi-objective optimization problems. In: Slovenian Conference on Artificial Intelligence, Proceedings of the 25th International Multiconference Information Society, vol. A, pp. 7–10 (2022)
Brockhoff, D., Auger, A., Hansen, N., Tušar, T.: Using well-understood single-objective functions in multiobjective black-box optimization test suites. Evol. Comput. 30(2), 165–193 (2022)
Demšar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)
Halton, J.H.: Algorithm 247: radical-inverse quasi-random point sequence. Commun. ACM 7(12), 701–702 (1964)
Hansen, N., Auger, A., Ros, R., Mersmann, O., Tušar, T., Brockhoff, D.: COCO: a platform for comparing continuous optimizers in a black-box setting. Optim. Methods Softw. 36(1), 114–144 (2021)
Hansen, N., Finck, S., Ros, R., Auger, A.: Real-parameter black-box optimization benchmarking 2009: noiseless functions definitions. Research report RR-6829, INRIA (2009)
Kerschke, P., Trautmann, H.: Automated algorithm selection on continuous black-box problems by combining exploratory landscape analysis and machine learning. Evol. Comput. 27(1), 99–127 (2019)
Krömer, P., Uher, V., Andova, A., Tušar, T., Filipič, B.: Sampling strategies for exploratory landscape analysis of bi-objective problems. In: Proceedings of the 2022 International Conference on Computational Science and Computational Intelligence (CSCI) (2022, in print)
Lemieux, C.: Monte Carlo and Quasi-Monte Carlo Sampling. Springer, New York (2009). https://doi.org/10.1007/978-0-387-78165-5
van der Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9(86), 2579–2605 (2008)
McKay, M.D., Beckman, R.J., Conover, W.J.: A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 42(1), 55–61 (2000)
Mersmann, O., Bischl, B., Trautmann, H., Preuss, M., Weihs, C., Rudolph, G.: Exploratory landscape analysis. In: Proceedings of the 13th Annual Genetic and Evolutionary Computation Conference (GECCO), pp. 829–836. ACM (2011)
Renau, Q., Doerr, C., Dreo, J., Doerr, B.: Exploratory landscape analysis is strongly sensitive to the sampling strategy. In: Bäck, T., et al. (eds.) PPSN 2020. LNCS, vol. 12270, pp. 139–153. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58115-2_10
Renau, Q., Dreo, J., Doerr, C., Doerr, B.: Towards explainable exploratory landscape analysis: extreme feature selection for classifying BBOB functions. In: Castillo, P.A., Jiménez Laredo, J.L. (eds.) EvoApplications 2021. LNCS, vol. 12694, pp. 17–33. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-72699-7_2
Skvorc, U., Eftimov, T., Korosec, P.: The effect of sampling methods on the invariance to function transformations when using exploratory landscape analysis. In: 2021 IEEE Congress on Evolutionary Computation (CEC), Kraków, Poland, pp. 1139–1146. IEEE (2021)
Sobol, I.M.: On the distribution of points in a cube and the approximate evaluation of integrals. Zhurnal Vychislitel’noi Matematiki i Matematicheskoi Fiziki 7(4), 784–802 (1967)
This work was supported by the project “Constrained multi-objective Optimization Based on Problem Landscape Analysis” funded by the Czech Science Foundation (grant no. GF22-34873K) and in part by the grant of the Student Grant System no. SP2023/12, VSB - Technical University of Ostrava.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Krömer, P., Uher, V. (2023). Comparing Sampling Strategies for the Classification of Bi-objective Problems by FLACCO Features. In: Barolli, L. (eds) Complex, Intelligent and Software Intensive Systems. CISIS 2023. Lecture Notes on Data Engineering and Communications Technologies, vol 176. Springer, Cham. https://doi.org/10.1007/978-3-031-35734-3_13
Download citation
DOI: https://doi.org/10.1007/978-3-031-35734-3_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-35733-6
Online ISBN: 978-3-031-35734-3
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)