Skip to main content
SpringerLink
Log in

Monte Carlo Tree Search with Metaheuristics

  • Conference paper
  • First Online:
Artificial Intelligence and Soft Computing (ICAISC 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14126))

Included in the following conference series:

  • 286 Accesses

Abstract

Monte Carlo Tree Search/Upper Confidence bounds applied to Trees (MCTS/UCT) is a popular and powerful search technique applicable to many domains, most frequently to searching game trees. Even though the algorithm has been widely researched, there is still room for its improvement, especially when combined with metaheuristics or machine learning methods. In this paper, we revise and experimentally evaluate the idea of enhancing MCTS/UCT with game-specific heuristics that guide the playout (simulation) phase. MCTS/UCT with the proposed guiding mechanism is tested on two popular board games: Othello and Hex. The enhanced method clearly defeats the well-known Alpha-beta pruning algorithm in both games, and for the more complex game (Othello) is highly competitive to the vanilla MCTS/UCT formulation.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Arneson, B., Hayward, R., Henderson, P.: Mohex wins hex tournament. In: ICGA, vol. 32, pp. 114–116, September 2013. https://doi.org/10.3233/ICG-2009-32218

  2. Barratt, J., Pan, C.: Playing go without game tree search using convolutional neural networks. ArXiv abs/1907.04658 (2019)

    Google Scholar 

  3. Björnsson, Y., Finnsson, H.: Cadiaplayer: a simulation-based general game player. Comput. Intell. AI Games IEEE Trans. 1, 4–15 (2009). https://doi.org/10.1109/TCIAIG.2009.2018702

  4. Enzenberger, M., Müller, M., Arneson, B., Segal, R.: Fuego-an open-source framework for board games and go engine based on monte carlo tree search. IEEE Trans. Comput. Intell. AI Games 2(4), 259–270 (2010). https://doi.org/10.1109/TCIAIG.2010.2083662

    Article  Google Scholar 

  5. Gelly, S., Silver, D.: Combining online and offline knowledge in UCT. In: Proceedings of the 24th International Conference on Machine Learning, pp. 273–280 (2007). https://doi.org/10.1145/1273496.1273531

  6. Gelly, S., Silver, D.: Monte-carlo tree search and rapid action value estimation in computer go. Artif. Intell. 175(11), 1856–1875 (2011). https://doi.org/10.1016/j.artint.2011.03.007

  7. Karwowski, J., Mańdziuk, J.: A new approach to security games. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2015, Part II. LNCS (LNAI), vol. 9120, pp. 402–411. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-19369-4_36

    Chapter  Google Scholar 

  8. Karwowski, J., Mańdziuk, J.: A monte carlo tree search approach to finding efficient patrolling schemes on graphs. Eur. J. Oper. Res. 277(1), 255–268 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  9. Kocsis, L., Szepesvári, C.: Bandit based monte-carlo planning. In: Fürnkranz, J., Scheffer, T., Spiliopoulou, M. (eds.) ECML 2006. LNCS (LNAI), vol. 4212, pp. 282–293. Springer, Heidelberg (2006). https://doi.org/10.1007/11871842_29

    Chapter  Google Scholar 

  10. Maarup, T.: Everything you always wanted to know about hexbut were afraid to Ask. Ph.D. thesis (2005). www.maarup.net/thomas/hex/hex3.pdf

  11. Segler, M.H., Preuss, M., Waller, M.P.: Planning chemical syntheses with deep neural networks and symbolic AI. Nature 555(7698), 604–610 (2018)

    Article  Google Scholar 

  12. Silver, D., et al.: Mastering the game of Go without human knowledge. Nature 550(7676), 354–359 (2017)

    Article  Google Scholar 

  13. Świechowski, M., Park, H.S., Mańdziuk, J., Kim, K.J.: Recent advances in general game playing. The Scientific World Journal 2015, Article ID: 986262 (2015)

    Google Scholar 

  14. Świechowski, M., Godlewski, K., Sawicki, B., Mańdziuk, J.: Monte carlo tree search: a review of recent modifications and applications. Artif. Intell. Rev. 56, 2497–2562 (2023). https://doi.org/10.1007/s10462-022-10228-y

    Article  Google Scholar 

  15. Świechowski, M., Mańdziuk, J.: Self-adaptation of playing strategies in general game playing. IEEE Trans. Comput. Intell. AI Games 6(4), 367–381 (2014). https://doi.org/10.1109/TCIAIG.2013.2275163

    Article  MATH  Google Scholar 

  16. Walȩdzik, K., Mańdziuk, J.: Applying hybrid monte carlo tree search methods to risk-aware project scheduling problem. Inf. Sci. 460–461, 450–468 (2018)

    Article  Google Scholar 

  17. Łapa, K., Cpałka, K., Kisiel-Dorohinicki, M., Paszkowski, J., Dȩbski, M., Le, V.H.: Multi-population-based algorithm with an exchange of training plans based on population evaluation. J. Artif. Intell. Soft Comput. Rese. 12(4), 239–253 (2022). https://doi.org/10.2478/jaiscr-2022-0016

    Article  Google Scholar 

  18. Łapa, K., Cpałka, K., Laskowski, Ł, Cader, A., Zeng, Z.: Evolutionary algorithm with a configurable search mechanism. J. Artif. Intell. Soft Comput. Res. 10(3), 151–171 (2020). https://doi.org/10.2478/jaiscr-2020-0011

    Article  Google Scholar 

Download references

Acknowledgements.

This research was carried out with the support of the Laboratory of Bioinformatics and Computational Genomics and the HPC Center of the Faculty of Mathematics and Information Science, Warsaw University of Technology under computational grant number A-22-03.

Author information

Authors and Affiliations

  1. Warsaw University of Technology, Warsaw, Poland

    Jacek Mańdziuk & Patryk Walczak

Authors
  1. Jacek Mańdziuk
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Patryk Walczak
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jacek Mańdziuk .

Editor information

Editors and Affiliations

  1. Systems Research Institute of the Polish Academy of Sciences, Warsaw, Poland

    Leszek Rutkowski

  2. Częstochowa University of Technology, Częstochowa, Poland

    Rafał Scherer

  3. Częstochowa University of Technology, Częstochowa, Poland

    Marcin Korytkowski

  4. University of Alberta, Edmonton, AB, Canada

    Witold Pedrycz

  5. AGH University of Krakow, Kraków, Poland

    Ryszard Tadeusiewicz

  6. University of Louisville, Louisville, KY, USA

    Jacek M. Zurada

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Mańdziuk, J., Walczak, P. (2023). Monte Carlo Tree Search with Metaheuristics. In: Rutkowski, L., Scherer, R., Korytkowski, M., Pedrycz, W., Tadeusiewicz, R., Zurada, J.M. (eds) Artificial Intelligence and Soft Computing. ICAISC 2023. Lecture Notes in Computer Science(), vol 14126. Springer, Cham. https://doi.org/10.1007/978-3-031-42508-0_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-42508-0_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-42507-3

  • Online ISBN: 978-3-031-42508-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation