Abstract
As one of the main research areas in AI, General Game Playing (GGP) is concerned with creating intelligent agents that can play more than one game based on game rules without human intervention. Most recent work has successfully applied deep reinforcement learning to GGP. This paper continues this line of work by integrating the Memory-Augmented Monte Carlo Tree Search algorithm (M-MCTS) with deep reinforcement learning for General Game Playing. We first extend M-MCTS from playing the single game Go to multiple concurrent games so as to cater to the domain of GGP. Then inspired by Goldwaser and Thielscher (2020), we combine the extension with deep reinforcement learning for building a general game player. Finally, we have tested this player on several games compared with the benchmark UCT player, and the experimental results have confirmed the feasibility of applying M-MCTS and deep reinforcement learning to GGP.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Auer, P.: Using confidence bounds for exploitation-exploration trade-offs. J. Mach. Learn. Res. 3, 397–422 (2002)
Ayuso, J.L.B.: Integration of general game playing with RL-glue (2012). http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.224.7707&rep=rep1&type=pdf
Brown, N., Sandholm, T.: Libratus: the superhuman AI for no-limit poker. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence, pp. 5226–5228 (2017)
Clune, J.: Heuristic evaluation functions for general game playing. In: Proceedings of the 22nd AAAI Conference on Artificial Intelligence, pp. 1134–1139 (2007)
Cox, E., Schkufza, E., Madsen, R., Genesereth, M.: Factoring general games using propositional automata. In: Proceedings of the IJCAI Workshop on General Intelligence in Game-Playing Agents, pp. 13–20 (2009)
Finnsson, H., Björnsson, Y.: Simulation-based approach to general game playing. In: Proceedings of the 23rd AAAI Conference on Artificial Intelligence, pp. 259–264 (2008)
Finnsson, H., Björnsson, Y.: Learning simulation control in general game-playing agents. In: Proceedings of the 24th AAAI Conference on Artificial Intelligence (2010)
Genesereth, M., Björnsson, Y.: The international general game playing competition. AI Mag. 34(2), 107–107 (2013)
Genesereth, M., Love, N., Pell, B.: General game playing: overview of the AAAI competition. AI Mag. 26(2), 62–72 (2005)
Genesereth, M., Thielscher, M.: General game playing. Synthesis Lectures on Artificial Intelligence and Machine Learning 8(2), 1–229 (2014). https://doi.org/10.1007/978-981-4560-52-8_34-1
Goldwaser, A., Thielscher, M.: Deep reinforcement learning for general game playing. In: Proceedings of the 34th AAAI Conference on Artificial Intelligence, pp. 1701–1708 (2020)
Gunawan, A., Ruan, J., Thielscher, M., Narayanan, A.: Exploring a learning architecture for general game playing. In: Gallagher, M., Moustafa, N., Lakshika, E. (eds.) AI 2020. LNCS (LNAI), vol. 12576, pp. 294–306. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-64984-5_23
Hsu, F.H.: Behind Deep Blue: Building the Computer That Defeated the World Chess Champion. Princeton University Press, Princeton (2002)
Koriche, F., Lagrue, S., Piette, É., Tabary, S.: General game playing with stochastic CSP. Constraints 21(1), 95–114 (2015). https://doi.org/10.1007/s10601-015-9199-5
Méhat, J., Cazenave, T.: A parallel general game player. KI-künstliche Intelligenz 25(1), 43–47 (2011)
Moravčík, M., et al.: Deepstack: expert-level artificial intelligence in heads-up no-limit poker. Science 356(6337), 508–513 (2017)
Schiffel, S., Thielscher, M.: Fluxplayer: a successful general game player. In: Proceedings of the 22nd AAAI Conference on Artificial Intelligence, pp. 1191–1196 (2007)
Silver, D., et al.: Mastering the game of Go with deep neural networks and tree search. Nature 529(7587), 484–489 (2016)
Silver, D., et al.: A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play. Science 362(6419), 1140–1144 (2018)
Silver, D. et al.: Mastering the game of Go without human knowledge. Nature 550(7676), 354–359 (2017)
Świechowski, M., Park, H., Mańdziuk, J., Kim, K.J.: Recent advances in general game playing. Sci. World J. 2015 (2015)
Thielscher, M.: General game playing in AI research and education. In: Bach, J., Edelkamp, S. (eds.) KI 2011. LNCS (LNAI), vol. 7006, pp. 26–37. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-24455-1_3
Wang, H., Emmerich, M., Plaat, A.: Monte carlo Q-learning for general game playing. arXiv preprint arXiv:1802.05944 (2018)
Xiao, C., Mei, J., Müller, M.: Memory-augmented Monte Carlo tree search. In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence (2018)
Acknowledgments
We are grateful to the reviewers of this paper for their constructive and insightful comments. The research reported in this paper was partially supported by the National Natural Science Foundation of China (No. 61806102), the Major Program of the National Social Science Foundation of China (No. 17ZDA026), and the National Key Project of Social Science of China (No. 21AZX013).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Liang, S., Jiang, G., Zhang, Y. (2022). Combining M-MCTS and Deep Reinforcement Learning for General Game Playing. In: Chen, J., Lang, J., Amato, C., Zhao, D. (eds) Distributed Artificial Intelligence. DAI 2021. Lecture Notes in Computer Science(), vol 13170. Springer, Cham. https://doi.org/10.1007/978-3-030-94662-3_14
Download citation
DOI: https://doi.org/10.1007/978-3-030-94662-3_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-94661-6
Online ISBN: 978-3-030-94662-3
eBook Packages: Computer ScienceComputer Science (R0)