Skip to main content
SpringerLink
Log in

A Collection of Robotics Problems for Benchmarking Evolutionary Computation Methods

  • Conference paper
  • First Online:
Applications of Evolutionary Computation (EvoApplications 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13989))

Abstract

The utilization of benchmarking techniques has a crucial role in the development of novel optimization algorithms, and also in performing comparisons between already existing methods. This is especially true in the field of evolutionary computation, where the theoretical performance of the method is difficult to analyze. For these benchmarking purposes, artificial (or synthetic) functions are currently the most widely used ones. In this paper, we present a collection of real-world robotics problems that can be used for benchmarking evolutionary computation methods. The proposed benchmark problems are a combination of inverse kinematics and path planning in robotics that can be parameterized. We conducted an extensive numerical investigation that encompassed solving 200 benchmark problems by seven selected metaheuristic algorithms. The results of this investigation showed that the proposed benchmark problems are quite difficult (multimodal and non-separable) and that they can be successfully used for differentiating and ranking various metaheuristics.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://www.universal-robots.com/cb3/.

  2. 2.

    https://doi.org/10.5281/zenodo.7584647.

  3. 3.

    https://github.com/JakubKudela89/Robotics-Benchmarking.

References

  1. Batista, J., Souza, D., Silva, J., Ramos, K., Costa, J., dos Reis, L., Braga, A.: Trajectory planning using artificial potential fields with metaheuristics. IEEE Lat. Am. Trans. 18(05), 914–922 (2020)

    Article  Google Scholar 

  2. Belge, E., Altan, A., Hacıoğlu, R.: Metaheuristic optimization-based path planning and tracking of quadcopter for payload hold-release mission. Electronics 11(8), 1208 (2022)

    Article  Google Scholar 

  3. Camacho Villalón, C.L., Stützle, T., Dorigo, M.: Grey wolf, firefly and bat algorithms: three widespread algorithms that do not contain any novelty. In: Dorigo, M., et al. (eds.) ANTS 2020. LNCS, vol. 12421, pp. 121–133. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-60376-2_10

    Chapter  Google Scholar 

  4. Campelo, F., Aranha, C.D.C.: Sharks, zombies and volleyball: lessons from the evolutionary computation bestiary. In: CEUR Workshop Proceedings, vol. 3007, p. 6. CEUR Workshop Proceedings (2021)

    Google Scholar 

  5. Cenikj, G., Lang, R.D., Engelbrecht, A.P., Doerr, C., Korošec, P., Eftimov, T.: Selector: Selecting a representative benchmark suite for reproducible statistical comparison. In: Proceedings of the Genetic and Evolutionary Computation Conference. GECCO 2022, New York, NY, USA, pp. 620–629. Association for Computing Machinery (2022)

    Google Scholar 

  6. Croucamp, M., Grobler, J.: Metaheuristics for the robot part sequencing and allocation problem with collision avoidance. In: Marreiros, G., Melo, F.S., Lau, N., Lopes Cardoso, H., Reis, L.P. (eds.) EPIA 2021. LNCS (LNAI), vol. 12981, pp. 469–481. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-86230-5_37

    Chapter  Google Scholar 

  7. Denavit, J., Hartenberg, R.S.: A kinematic notation for lower-pair mechanisms based on matrices. J. Appl. Mech. 22(2), 215–221 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  8. Dereli, S., Köker, R.: A meta-heuristic proposal for inverse kinematics solution of 7-dof serial robotic manipulator: quantum behaved particle swarm algorithm. Artif. Intell. Rev. 53(2), 949–964 (2020)

    Article  Google Scholar 

  9. García-Martínez, C., Gutiérrez, P.D., Molina, D., Lozano, M., Herrera, F.: Since cec 2005 competition on real-parameter optimisation: a decade of research, progress and comparative analysis’s weakness. Soft. Comput. 21(19), 5573–5583 (2017)

    Article  Google Scholar 

  10. Garden, R.W., Engelbrecht, A.P.: Analysis and classification of optimisation benchmark functions and benchmark suites. In: 2014 IEEE Congress on Evolutionary Computation (CEC), pp. 1641–1649. IEEE (2014)

    Google Scholar 

  11. Hansen, N.: The CMA evolution strategy: A tutorial. arXiv preprint, arXiv:1604.00772 (2016). https://doi.org/10.48550/ARXIV.1604.00772

  12. Hansen, N., Auger, A., Ros, R., Mersmann, O., Tušar, T., Brockhoff, D.: Coco: A platform for comparing continuous optimizers in a black-box setting. Optim. Meth. Software 36(1), 114–144 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  13. Hellwig, M., Beyer, H.G.: Benchmarking evolutionary algorithms for single objective real-valued constrained optimization-a critical review. Swarm Evol. Comput. 44, 927–944 (2019)

    Article  Google Scholar 

  14. Hulka, T., Matoušek, R., Dobrovský, L., Dosoudilová, M., Nolle, L.: Optimization of snake-like robot locomotion using GA: Serpenoid design. Mendel J. 26(1), 1–6 (2020)

    Article  Google Scholar 

  15. Kanagaraj, G., Masthan, S.S., Vincent, F.Y.: Meta-heuristics based inverse kinematics of robot manipulator’s path tracking capability under joint limits. Mendel J. 28(1), 41–54 (2022)

    Article  Google Scholar 

  16. Karaboga, D.: An idea based on honey bee swarm for numerical optimization. Technical report TR06, Erciyes University (2005)

    Google Scholar 

  17. Kazikova, A., Pluhacek, M., Senkerik, R.: Why tuning the control parameters of metaheuristic algorithms is so important for fair comparison? In: Mendel. vol. 26, pp. 9–16 (2020)

    Google Scholar 

  18. Kennedy, J., Eberhart, R.: Particle swarm optimization. In: Proceedings of ICNN’95 - International Conference on Neural Networks, vol. 4, pp. 1942–1948 (1995)

    Google Scholar 

  19. Kerschke, P., Trautmann, H.: Comprehensive feature-based landscape analysis of continuous and constrained optimization problems using the r-package Flacco. In: Bauer, N., Ickstadt, K., Lübke, K., Szepannek, G., Trautmann, H., Vichi, M. (eds.) Applications in Statistical Computing. SCDAKO, pp. 93–123. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-25147-5_7

    Chapter  Google Scholar 

  20. Khan, A.H., Li, S., Chen, D., Liao, L.: Tracking control of redundant mobile manipulator: an RNN based metaheuristic approach. Neurocomputing 400, 272–284 (2020)

    Article  Google Scholar 

  21. Kudela, J.: A critical problem in benchmarking and analysis of evolutionary computation methods. Nature Mach. Intell. 4, 1238–1245 (2022)

    Article  Google Scholar 

  22. Kudela, J., Matousek, R.: New benchmark functions for single-objective optimization based on a zigzag pattern. IEEE Access 10, 8262–8278 (2022)

    Article  Google Scholar 

  23. Kudela, J., Matousek, R.: Recent advances and applications of surrogate models for finite element method computations: a review. Soft Comput. 1–25 (2022)

    Google Scholar 

  24. Kumar, R., Singh, L., Tiwari, R.: Comparison of two meta-heuristic algorithms for path planning in robotics. In: 2020 International Conference on Contemporary Computing and Applications (IC3A), pp. 159–162. IEEE (2020)

    Google Scholar 

  25. Mersmann, O., Preuss, M., Trautmann, H.: Benchmarking evolutionary algorithms: towards exploratory landscape analysis. In: Schaefer, R., Cotta, C., Kołodziej, J., Rudolph, G. (eds.) PPSN 2010. LNCS, vol. 6238, pp. 73–82. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15844-5_8

    Chapter  Google Scholar 

  26. Mohamed, A.W., Hadi, A.A., Mohamed, A.K., Awad, N.H.: Evaluating the performance of adaptive gainingsharing knowledge based algorithm on CEC 2020 benchmark problems. In: 2020 IEEE Congress on Evolutionary Computation (CEC), pp. 1–8. IEEE (2020)

    Google Scholar 

  27. Niu, P., Niu, S., Chang, L., et al.: The defect of the grey wolf optimization algorithm and its verification method. Knowl.-Based Syst. 171, 37–43 (2019)

    Article  Google Scholar 

  28. Nonoyama, K., Liu, Z., Fujiwara, T., Alam, M.M., Nishi, T.: Energy-efficient robot configuration and motion planning using genetic algorithm and particle swarm optimization. Energies 15(6), 2074 (2022)

    Article  Google Scholar 

  29. Parak, R., Matousek, R.: Comparison of multiple reinforcement learning and deep reinforcement learning methods for the task aimed at achieving the goal. Mendel J. 27(1), 1–8 (2021)

    Article  Google Scholar 

  30. Pattnaik, S., Mishra, D., Panda, S.: A comparative study of meta-heuristics for local path planning of a mobile robot. Eng. Optim. 54(1), 134–152 (2022)

    Article  Google Scholar 

  31. Piotrowski, A.P.: Regarding the rankings of optimization heuristics based on artificially-constructed benchmark functions. Inf. Sci. 297, 191–201 (2015)

    Article  Google Scholar 

  32. Qadir, Z., Zafar, M.H., Moosavi, S.K.R., Le, K.N., Mahmud, M.P.: Autonomous UAV path-planning optimization using metaheuristic approach for predisaster assessment. IEEE Internet Things J. 9(14), 12505–12514 (2021)

    Article  Google Scholar 

  33. Serrano-Pérez, O., Villarreal-Cervantes, M.G., González-Robles, J.C., Rodríguez-Molina, A.: Meta-heuristic algorithms for the control tuning of omnidirectional mobile robots. Eng. Optim. (2019)

    Google Scholar 

  34. Siciliano, B., Khatib, O. (eds.): Springer, Cham (2016). https://doi.org/10.1007/978-3-319-32552-1

    Book  MATH  Google Scholar 

  35. Škvorc, U., Eftimov, T., Korošec, P.: Understanding the problem space in single-objective numerical optimization using exploratory landscape analysis. Appl. Soft Comput. 90, 106138 (2020)

    Article  Google Scholar 

  36. Storn, R., Price, K.: Differential evolution-a simple and efficient heuristic for global optimization over continuous spaces. J. Global Optim. 11(4), 341–359 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  37. Tanabe, R.: Benchmarking feature-based algorithm selection systems for black-box numerical optimization. IEEE Trans. Evol. Comput. 26, 1321–1335 (2022)

    Article  Google Scholar 

  38. Tanabe, R., Fukunaga, A.S.: Improving the search performance of shade using linear population size reduction. In: 2014 IEEE Congress on Evolutionary Computation (CEC), pp. 1658–1665. IEEE (2014)

    Google Scholar 

  39. Tzanetos, A., Dounias, G.: Nature inspired optimization algorithms or simply variations of metaheuristics? Artif. Intell. Rev. 54(3), 1841–1862 (2021)

    Article  MATH  Google Scholar 

  40. Yadav, V., Botchway, R.K., Senkerik, R., Oplatkova, Z.K.: Robotic automation of software testing from a machine learning viewpoint. Mendel J. 27(2), 68–73 (2021)

    Article  Google Scholar 

  41. Yin, S., Luo, Q., Zhou, G., Zhou, Y., Zhu, B.: An equilibrium optimizer slime mould algorithm for inverse kinematics of the 7-dof robotic manipulator. Sci. Rep. 12(1), 1–28 (2022)

    Article  Google Scholar 

  42. Zhang, G., Shi, Y.: Hybrid sampling evolution strategy for solving single objective bound constrained problems. In: 2018 IEEE Congress on Evolutionary Computation (CEC), pp. 1–7. IEEE (2018)

    Google Scholar 

Download references

Acknowledgements

This work was supported by the IGA BUT No. FSI-S-23-8394 “Artificial intelligence methods in engineering tasks”.

Author information

Authors and Affiliations

  1. Institute of Automation and Computer Science, Brno University of Technology Technicka 2, 621 00, Brno, Czech Republic

    Jakub Kůdela, Martin Juříček & Roman Parák

Authors
  1. Jakub Kůdela
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Martin Juříček
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Roman Parák
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jakub Kůdela .

Editor information

Editors and Affiliations

  1. University of Coimbra, Coimbra, Portugal

    João Correia

  2. University of York, York, UK

    Stephen Smith

  3. Al Hussein Technical University, Amman, Jordan

    Raneem Qaddoura

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kůdela, J., Juříček, M., Parák, R. (2023). A Collection of Robotics Problems for Benchmarking Evolutionary Computation Methods. In: Correia, J., Smith, S., Qaddoura, R. (eds) Applications of Evolutionary Computation. EvoApplications 2023. Lecture Notes in Computer Science, vol 13989. Springer, Cham. https://doi.org/10.1007/978-3-031-30229-9_24

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-30229-9_24

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-30228-2

  • Online ISBN: 978-3-031-30229-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation