Skip to main content
SpringerLink
Log in

Multi-armed Bandit-Based Metaheuristic Operator Selection: The Pendulum Algorithm Binarization Case

  • Conference paper
  • First Online:
Optimization and Learning (OLA 2023)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1824))

Included in the following conference series:

  • 333 Accesses

Abstract

Multi-armed bandit (MAB) is a well-known reinforcement learning algorithm that has shown outstanding performance for recommendation systems and other areas. On the other hand, metaheuristic algorithms have gained much popularity due to their great performance in solving complex problems with endless search spaces. Pendulum Search Algorithm (PSA) is a recently created metaheuristic inspired by the harmonic motion of a pendulum. Its main limitation is to solve combinatorial optimization problems, characterized by using variables in the discrete domain. To overcome this limitation, we propose to use a two-step binarization technique, which offers a large number of possible options that we call scheme. For this, we use MAB as an algorithm that learns and recommends a binarization schemes during the execution of the iterations (online). With the experiments carried out, we show that it delivers better results in solving the Set Covering problem than using a fixed binarization scheme.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ab. Aziz, N.A., Ab. Aziz, K.: Pendulum search algorithm: an optimization algorithm based on simple harmonic motion and its application for a vaccine distribution problem. Algorithms 15(6) (2022)

    Google Scholar 

  2. Rahman, T.A., Ibrahim, Z., Ab. Aziz, N.A., Zhao, S., Aziz, N.H.A.: Single-agent finite impulse response optimizer for numerical optimization problems. IEEE Access 6, 9358–9374 (2018)

    Article  Google Scholar 

  3. Alizadeh, R., Nishi, T.: Hybrid set covering and dynamic modular covering location problem: Application to an emergency humanitarian logistics problem. Appl. Sci. 10(20), 7110 (2020)

    Article  Google Scholar 

  4. Audibert, J.-Y., Munos, R., Szepesvári, C.: Exploration-exploitation tradeoff using variance estimates in multi-armed bandits. Theoret. Comput. Sci. 410(19), 1876–1902 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  5. Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Mach. Learn. 47(2), 235–256 (2002)

    Article  MATH  Google Scholar 

  6. Becerra-Rozas, M., et al.: Continuous metaheuristics for binary optimization problems: an updated systematic literature review. Mathematics 11(1), 129 (2022)

    Article  MathSciNet  Google Scholar 

  7. Crawford, B., Soto, R., Astorga, G., García, J., Castro, C., Paredes, F.: Putting continuous metaheuristics to work in binary search spaces. Complexity 2017 (2017)

    Google Scholar 

  8. Crawford, B., Soto, R., Monfroy, E., Astorga, G., García, J., Cortes, E.: A meta-optimization approach for covering problems in facility location. In: Figueroa-García, J.C., López-Santana, E.R., Villa-Ramírez, J.L., Ferro-Escobar, R. (eds.) WEA 2017. CCIS, vol. 742, pp. 565–578. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66963-2_50

    Chapter  Google Scholar 

  9. DaCosta, L., Fialho, A., Schoenauer, M., Sebag, M.: Adaptive operator selection with dynamic multi-armed bandits. In: Proceedings of the 10th Annual Conference on Genetic and Evolutionary Computation, pp. 913–920 (2008)

    Google Scholar 

  10. Elena, G., Milos, K., Eugene, I.: Survey of multiarmed bandit algorithms applied to recommendation systems. Int. J. Open Inf. Technol. 9(4), 12–27 (2021)

    Google Scholar 

  11. Lanza-Gutierrez, J.M., Caballe, N.C., Crawford, B., Soto, R., Gomez-Pulido, J.A., Paredes, F.: Exploring further advantages in an alternative formulation for the set covering problem. Math. Probl. Eng. 2020 (2020)

    Google Scholar 

  12. Lanza-Gutierrez, J.M., Crawford, B., Soto, R., Berrios, N., Gomez-Pulido, J.A., Paredes, F.: Analyzing the effects of binarization techniques when solving the set covering problem through swarm optimization. Expert Syst. Appl. 70, 67–82 (2017)

    Article  Google Scholar 

  13. Lemus-Romani, J., et al.: A novel learning-based binarization scheme selector for swarm algorithms solving combinatorial problems. Mathematics 9(22), 2887 (2021)

    Article  Google Scholar 

  14. Mandal, S., Patra, N., Pal, M.: Covering problem on fuzzy graphs and its application in disaster management system. Soft. Comput. 25(4), 2545–2557 (2021)

    Article  MATH  Google Scholar 

  15. Patil, V., Ghalme, G., Nair, V., Narahari, Y.: Achieving fairness in the stochastic multi-armed bandit problem. In: AAAI, pp. 5379–5386 (2020)

    Google Scholar 

  16. Rodriguez-Tello, E., Narvaez-Teran, V., Lardeux, F.: Dynamic multi-armed bandit algorithm for the cyclic bandwidth sum problem. IEEE Access 7, 40258–40270 (2019)

    Article  MATH  Google Scholar 

  17. Song, H., Triguero, I., Özcan, E.: A review on the self and dual interactions between machine learning and optimisation. Progr. Artif. Intell. 8(2), 143–165 (2019). https://doi.org/10.1007/s13748-019-00185-z

    Article  Google Scholar 

  18. Soto, R., et al.: A reactive population approach on the dolphin echolocation algorithm for solving cell manufacturing systems. Mathematics 8(9), 1389 (2020)

    Article  Google Scholar 

  19. Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, 2nd edn. The MIT Press, Cambridge (2018)

    MATH  Google Scholar 

  20. Talbi, E.-G.: Metaheuristics: From Design to Implementation. Wiley, Hoboken (2009)

    Book  MATH  Google Scholar 

  21. Xiang, X., Qiu, J., Xiao, J., Zhang, X.: Demand coverage diversity based ant colony optimization for dynamic vehicle routing problems. Eng. Appl. Artif. Intell. 91, 103582 (2020)

    Google Scholar 

  22. Fialho, Á., Da Costa, L., Schoenauer, M., Sebag, M.: Dynamic multi-armed bandits and extreme value-based rewards for adaptive operator selection in evolutionary algorithms. LION 3, 176–190 (2009)

    Google Scholar 

Download references

Acknowledgements

Broderick Crawford, Ricardo Soto, Eduardo Rodriguez-Tello and Felipe Cisternas-Caneo are supported by Dirección de Investigación, VINCI-PUCV; Project: DI Investigación Asociativa Interdisciplinaria 2022 "SELECCIÓN DE CARACTERÍSTICAS USANDO METAHEURÍSTICAS PARA POTENCIAR MODELOS PREDICTIVOS EN SALUD”.

Broderick Crawford and Ricardo Soto are supported by Grant ANID/ FONDECYT/REGULAR/1210810.

Felipe Cisternas-Caneo is supported by Beca INF-PUCV.

Author information

Authors and Affiliations

  1. Pontificia Universidad Católica de Valparaíso, Valparaíso, Chile

    Pablo Ábrego-Calderón, Broderick Crawford, Ricardo Soto & Felipe Cisternas-Caneo

  2. Cinvestav, Unidad Tamaulipas, Km. 5.5 Carretera Victoria - Soto La Marina, 87130, Victoria, Tamaulipas, Mexico

    Eduardo Rodriguez-Tello

  3. Université d’ Angers, LERIA, Angers, France

    Eric Monfroy

  4. Universidad Andres Bello, Santiago, Chile

    Giovanni Giachetti

Authors
  1. Pablo Ábrego-Calderón
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Broderick Crawford
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ricardo Soto
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Eduardo Rodriguez-Tello
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Felipe Cisternas-Caneo
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Eric Monfroy
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Giovanni Giachetti
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pablo Ábrego-Calderón .

Editor information

Editors and Affiliations

  1. University of Cadiz, Cadiz, Spain

    Bernabé Dorronsoro

  2. University of Malaga, Malaga, Spain

    Francisco Chicano

  3. University of Luxembourg, Esch-sur-Alzette, Luxembourg

    Gregoire Danoy

  4. University of Lille, Lille, France

    El-Ghazali Talbi

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ábrego-Calderón, P. et al. (2023). Multi-armed Bandit-Based Metaheuristic Operator Selection: The Pendulum Algorithm Binarization Case. In: Dorronsoro, B., Chicano, F., Danoy, G., Talbi, EG. (eds) Optimization and Learning. OLA 2023. Communications in Computer and Information Science, vol 1824. Springer, Cham. https://doi.org/10.1007/978-3-031-34020-8_19

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-34020-8_19

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-34019-2

  • Online ISBN: 978-3-031-34020-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation