research-article

Examining algorithm behavior using recurrence quantification and landscape analyses

Author:

Mario Andrés MuñozAuthors Info & Claims

GECCO '22: Proceedings of the Genetic and Evolutionary Computation Conference Companion

Pages 1658 - 1665

https://doi.org/10.1145/3520304.3534029

Published: 19 July 2022 Publication History

Abstract

Differences in performance between algorithms can be attributed to the interaction between their unique rule-sets and the characteristics of the instance's landscape. However, understanding this interaction can be difficult because algorithms are often composed of multiple elements, and in the worst cases are described using opaque notation and metaphors. In this paper, we introduce a methodology for the behavioral analysis of optimization algorithms, based on comparing algorithm dynamics in a given problem instance. At the methodology's core lays the hypothesis that if two algorithms, with the exact same initial conditions, have similar dynamics, then their rule-sets are also similar. An examination of Grey Wolf Optimization, shows that it exhibits bias leading to similar behavioral patterns regardless of the function.

References

[1]

T. Bartz-Beielstein, C. Doerr, D. van den Berg, J. Bossek, S. Chandrasekaran, T. Eftimov, A. Fischbach, P. Kerschke, W. La Cava, M. Lopez-Ibanez, K.M. Malan, J.H. Moore, B. Naujoks, P. Orzechowski, V. Volz, M. Wagner, and T. Weise. 2020. Benchmarking in Optimization: Best Practice and Open Issues. arXiv:2007.03488 [cs.NE]

[2]

N. Belkhir, J. Dréo, P. Savéant, and M. Schoenauer. 2017. Per instance algorithm configuration of CMA-ES with limited budget. In Proceedings of the Genetic and Evolutionary Computation Conference. ACM.

Digital Library

[3]

F. Biscani and D. Izzo. 2020. A parallel global multiobjective framework for optimization: pagmo. J. Open Source Softw. 5, 53 (2020), 2338.

[4]

C.L. Camacho-Villalón, T. Stützle, and M. Dorigo. 2020. Grey Wolf, Firefly and Bat Algorithms: Three Widespread Algorithms that Do Not Contain Any Novelty. In ANTS 2020. Springer International Publishing, 121--133.

[5]

J.-P Eckmann, S. Oliffson Kamphorst, and D Ruelle. 1987. Recurrence Plots of Dynamical Systems. Europhys. Lett. 4, 9 (nov 1987), 973--977.

[6]

N. Hansen, D. Brockhoff, O. Mersmann, T. Tusar, D. Tusar, O. Ait ElHara, P.R. Sampaio, A. Atamna, K. Varelas, U. Batu, D. Manh Nguyen, F. Matzner, and A. Auger. 2019. COmparing Continuous Optimizers: numbbo/COCO on Github.

[7]

P. Kerschke and H. Trautmann. 2019. Automated Algorithm Selection on Continuous Black-Box Problems by Combining Exploratory Landscape Analysis and Machine Learning. Evol. Comput. 27, 1 (mar 2019), 99--127.

Digital Library

[8]

K.M. Malan and A.P. Engelbrecht. 2014. Characterising the searchability of continuous optimisation problems for PSO. Swarm Intell. 8, 4 (2014), 1--28.

[9]

N. Marwan, M. Carmen Romano, M. Thiel, and Jügen Kurths. 2007. Recurrence plots for the analysis of complex systems. Phys. Rep. 438, 2007 (2007), 2370329.

[10]

N. Marwan, M. Thiel, and N. R. Nowaczyk. 2002. Cross recurrence plot based synchronization of time series. Nonlinear Process. Geophys. 9, 3/4 (2002), 325--331.

[11]

M.A. Muñoz and K. Smith-Miles. 2017. Performance analysis of continuous black-box optimization algorithms via footprints in instance space. Evol. Comput. 25, 4 (2017), 529--554.

Digital Library

[12]

M.A. Muñoz and K.A. Smith-Miles. 2020. Generating new space-filling test instances for continuous black-box optimization. Evol. Comput. 28, 3 (2020), 379--404.

Digital Library

[13]

M.A. Muñoz, Y. Sun, M. Kirley, and S.K. Halgamuge. 2015. Algorithm selection for black-box continuous optimization problems: a survey on methods and challenges. Inform. Sciences 317 (2015), 224--245.

Digital Library

[14]

Y. Shi and R.C. Eberhart. 1998. A modified particle swarm optimizer. In IEEE CEC '98. 69--73.

[15]

K. Smith-Miles, D. Baatar, B. Wreford, and R. Lewis. 2014. Towards objective measures of algorithm performance across instance space. Comput. Oper. Res. 45 (2014), 12--24.

Digital Library

[16]

K. Sörensen. 2015. Metaheuristics--- the metaphor exposed. Int. T. Oper. Res. 22, 1 (Jan. 2015), 3--18.

[17]

T. Vantuch, I. Zelinka, A. Adamatzky, and N. Marwan. 2018. Phase Transitions in Swarm Optimization Algorithms. In UCNC 2018. Springer International Publishing, 204--216.

[18]

T. Vantuch, I. Zelinka, A. Adamatzky, and N. Marwan. 2019. Perturbations and phase transitions in swarm optimization algorithms. Nat. Comput. 18, 3 (may 2019), 579--591.

Digital Library

[19]

H. Yang. 2011. Multiscale Recurrence Quantification Analysis of Spatial Cardiac Vectorcardiogram Signals. IEEE Trans. Biomed. Eng. 58, 2 (2011), 339--347.

[20]

J.P. Zbilut, A. Giuliani, and C.L. Webber. 1998. Detecting deterministic signals in exceptionally noisy environments using cross-recurrence quantification. Phys. Lett. A 246, 1--2 (1998), 122--128.

[21]

I. Zelinka. 2015. A survey on evolutionary algorithms dynamics and its complexity - Mutual relations, past, present and future. Swarm Evol. Comput. 25 (2015), 2--14.

[22]

I. Zelinka, L. Tomaszek, P. Vasant, T. Trong Dao, and Duy Vo Hoang. 2018. A novel approach on evolutionary dynamics analysis - A progress report. J Comput. Sci. 25 (2018), 437--445.

Cited By

Smith-Miles K(2025)Understanding Instance Hardness for Optimisation Algorithms: Methodologies, Open Challenges and Post-Quantum ImplicationsApplied Mathematical Modelling10.1016/j.apm.2025.115965(115965)Online publication date: Feb-2025
https://doi.org/10.1016/j.apm.2025.115965

Index Terms

Examining algorithm behavior using recurrence quantification and landscape analyses

Recommendations

BBOB Instance Analysis: Landscape Properties and Algorithm Performance Across Problem Instances
Applications of Evolutionary Computation
Abstract
Benchmarking is a key aspect of research into optimization algorithms, and as such the way in which the most popular benchmark suites are designed implicitly guides some parts of algorithm design. One of these suites is the black-box optimization ...
Landscape analysis of gaussian process surrogates for the covariance matrix adaptation evolution strategy
GECCO '19: Proceedings of the Genetic and Evolutionary Computation Conference

Gaussian processes modeling technique has been shown as a valuable surrogate model for the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) in continuous single-objective black-box optimization tasks, where the optimized function is expensive. ...
Measures in the time and frequency domains for fitness landscape analysis of dynamic optimization problems

Display Omitted Using Discrete-time Fourier transform (DTFT) and dynamic time warping (DTW) distance to analyze dynamic fitness landscape.The measures are extracted from the frequency spectrum and the time domain of fitness landscape.The criteria ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

GECCO '22: Proceedings of the Genetic and Evolutionary Computation Conference Companion

July 2022

2395 pages

ISBN:9781450392686

DOI:10.1145/3520304

Editor:
Jonathan E. Fieldsend
University of Exeter
,
General Chair:
Markus Wagner
The University of Adelaide

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGEVO: ACM Special Interest Group on Genetic and Evolutionary Computation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 July 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Australian Research Council

Conference

GECCO '22

Sponsor:

SIGEVO

GECCO '22: Genetic and Evolutionary Computation Conference

July 9 - 13, 2022

Massachusetts, Boston

Acceptance Rates

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
38
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)0

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Smith-Miles K(2025)Understanding Instance Hardness for Optimisation Algorithms: Methodologies, Open Challenges and Post-Quantum ImplicationsApplied Mathematical Modelling10.1016/j.apm.2025.115965(115965)Online publication date: Feb-2025
https://doi.org/10.1016/j.apm.2025.115965

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten