Abstract
Real-time fighting games are challenging for computer agents in that actions must be decided within a relatively short cycle of time, usually in milliseconds or less. That is only achievable by either very powerful machines or state-of-the-art algorithms. The former is usually a costly option while the latter remains an ongoing research topic despite countless research. This paper describes our algorithmic approach towards real-time fighting games via the fighting game AI challenge. The focus of our research is the LUD division, the most challenging category of the competition where action data is hidden to prevent methods that are dependent on prior training. In this paper, we propose several generic heuristics that can be used in combination with Monte-Carlo tree search. Our experimental results show that such an approach would provide an excellent AI outperforming pure Monte-Carlo tree search and classic algorithms such as evolutionary algorithms or deep reinforcement learning. Nonetheless, we believe that our proposed heuristics should be able to generalize to other domains beyond the scope of the fighting game AI challenge.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Back T (1996) Evolutionary algorithms in theory and practice: evolution strategies, evolutionary programming, genetic algorithms. Oxford University Press, Oxford
Balabanov K, Logofătu D (2019) Developing a general video game ai controller based on an evolutionary approach. In: Asian conference on intelligent information and database systems. Springer, pp 315–326
Bingmann T, Marianczuk J, Sanders P (2020) Engineering faster sorters for small sets of items. arXiv:2002.05599
Campbell M, Hoane AJ Jr, Hsu F (2002) Deep blue. Artif Intell 134(1–2):57–83
Cavazza M (2000) Al in computer games: survey and perspectives. Virtual Real 5(4):223–235
Chaslot G, Bakkes S, Szita I, Spronck P (2008) Monte-Carlo tree search: a new framework for game ai. In: AIIDE
Cho BH, Jung SH, Seong YR, Oh HR (2006) Exploiting intelligence in fighting action games using neural networks. IEICE Trans Inf Syst 89(3):1249–1256
Coulom R (2006) Efficient selectivity and backup operators in Monte-Carlo tree search. In: International conference on computers and games. Springer, pp 72–83
Demediuk S, Tamassia M, Raffe WL, Zambetta F, Li, X, Mueller F (2017) Monte Carlo tree search based algorithms for dynamic difficulty adjustment. In: 2017 IEEE conference on computational intelligence and games (CIG). IEEE, pp 53–59
Francisco-Valencia I, Marcial-Romero JR, Valdovinos-Rosas RM (2019) Some variations of upper confidence bound for general game playing. In: Mexican conference on pattern recognition. Springer, pp 68–79
Harper T (2013) The culture of digital fighting games: performance and practice. Routledge, Abingdon
Ishihara M, Miyazaki T, Chu CY, Harada T, Thawonmas R (2016) Applying and improving Monte-Carlo tree search in a fighting game ai. In: Proceedings of the 13th international conference on advances in computer entertainment technology, pp 1–6
Ishihara M, Ito S, Ishii R, Harada T, Thawonmas R (2018) Monte-Carlo tree search for implementation of dynamic difficulty adjustment fighting game ais having believable behaviors. In: 2018 IEEE conference on computational intelligence and games (CIG). IEEE, pp 1–8
Ishii R, Ito S, Ishihara M, Harada T, Thawonmas R (2018) Monte-Carlo tree search implementation of fighting game ais having personas. In: 2018 IEEE conference on computational intelligence and games (CIG). IEEE, pp 1–8
Kim MJ, Kim KJ (2017) Opponent modeling based on action table for mcts-based fighting game ai. In: 2017 IEEE conference on computational intelligence and games (CIG). IEEE, pp 178–180
Kim MJ, Ahn CW (2018) Hybrid fighting game ai using a genetic algorithm and Monte Carlo tree search. In: Proceedings of the genetic and evolutionary computation conference companion, pp 129–130
Kim MJ, Kim JS, Lee D, Kim SJ, Kim MJ, Ahn CW (2019) Integrating agent actions with genetic action sequence method. In: Proceedings of the genetic and evolutionary computation conference companion, pp 59–60
Lattimore T, Szepesvári C (2018) Bandit algorithms, p 28
Logofatu D, Leon F, Muharemi F (2019) General video game ai controller-integrating three algorithms to bring a new solution. In: 2019 23rd international conference on system theory, control and computing (ICSTCC). IEEE, pp 856–859
Lu F, Yamamoto K, Nomura LH, Mizuno S, Lee Y, Thawonmas R (2013) Fighting game artificial intelligence competition platform. In: 2013 IEEE 2nd global conference on consumer electronics (GCCE). IEEE, pp 320–323
Martinez-Arellano G, Cant R, Woods D (2016) Creating ai characters for fighting games using genetic programming. IEEE Trans Comput Intell AI Games 9(4):423–434
Nakagawa Y, Yamamoto K, Thawonmas R (2014) Online adjustment of the ai’s strength in a fighting game using the k-nearest neighbor algorithm and a game simulator. In: 2014 IEEE 3rd global conference on consumer electronics (GCCE). IEEE, pp 494–495
Nakagawa Y, Yamamoto K, Yin CC, Harada T, Thawonmas R (2015) Predicting the opponent’s action using the k-nearest neighbor algorithm and a substring tree structure. In: 2015 IEEE 4th global conference on consumer electronics (GCCE). IEEE, pp 533–534
Ng BW (2006) Street fighter and the king of fighters in Hong Kong: a study of cultural consumption and localization of Japanese games in an Asian context. Game Stud 6(1):2006
Nork B, Lengert GD, Litschel RU, Ahmad N, Lam GT, Logofătu D (2018) Machine learning with the pong game: a case study. In: International conference on engineering applications of neural networks. Springer, pp 106–117
Osaka S, Thawonmas R, Shibazaki T (2014) Investigation of various online adaptation methods of computer-game ai rulebase in dynamic scripting. In: Proceedings of the 1st international conference on digital interactive media entertainment and arts (DIME-ARTS 2006) (2006)
Perez-Liebana D, Samothrakis S, Togelius J, Schaul T, Lucas SM (2016) General video game ai: competition, challenges and opportunities. In: 13th AAAI conference on artificial intelligence
Pinto IP, Coutinho LR (2018) Hierarchical reinforcement learning with monte carlo tree search in computer fighting game. IEEE Trans Games 11(3):290–295
Robison AD (2017) Neural network ai for fighting ice. California Polytechnic State University, San Luis Obispo (Thesis)
Silver D, Huang A, Maddison CJ, Guez A, Sifre L, Van Den Driessche G, Schrittwieser J, Antonoglou I, Panneershelvam V, Lanctot M et al (2016) Mastering the game of go with deep neural networks and tree search. Nature 529(7587):484
Silver D, Schrittwieser J, Simonyan K, Antonoglou I, Huang A, Guez A, Hubert T, Baker L, Lai M, Bolton A et al (2017) Mastering the game of go without human knowledge. Nature 550(7676):354–359
Skinner G, Walmsley T (2019) Artificial intelligence and deep learning in video games a brief review. In: 2019 IEEE 4th international conference on computer and communication systems (ICCCS). IEEE, pp 404–408
Tamassia M (2017) Artificial intelligence techniques towards adaptive digital games. College of Science, Engineering and Health, RMIT University
Taylor ME, Carboni N, Fachantidis A, Vlahavas I, Torrey L (2014) Reinforcement learning agents providing advice in complex video games. Connect Sci 26(1):45–63
Thawonmas R, Osaka S (2006) A method for online adaptation of computer-game ai rulebase. In: Proceedings of the 2006 ACM SIGCHI international conference on Advances in computer entertainment technology, p 16
Thuan LG, Logofătu D, Badică C (2019) A hybrid approach for the fighting game ai challenge: balancing case analysis and Monte Carlo tree search for the ultimate performance in unknown environment. In: International conference on engineering applications of neural networks. Springer, pp 139–150
Vodopivec T, Samothrakis S, Ster B (2017) On monte carlo tree search and reinforcement learning. J Artif Intell Res 60:881–936
Williams PR, Perez-Liebana D, Lucas SM (2016) Ms. pac-man versus ghost team cig 2016 competition. In: 2016 IEEE conference on computational intelligence and games (CIG). IEEE, pp 1–8
Yoon S, Kim KJ (2017) Deep q networks for visual fighting game ai. In: 2017 IEEE conference on computational intelligence and games (CIG). IEEE, pp 306–308
Yoshida S, Ishihara M, Miyazaki T, Nakagawa Y., Harada T, Thawonmas R (2016) Application of Monte-Carlo tree search in a fighting game ai. In: 2016 IEEE 5th global conference on consumer electronics. IEEE, pp 1–2
Zook A, Harrison B, Riedl MO (2019) Monte-Carlo tree search for simulation-based strategy analysis. arXiv:1908.01423
Zuin GL, Macedo YP, Chaimowicz L, Pappa GL (2016) Discovering combos in fighting games with evolutionary algorithms. In: Proceedings of the genetic and evolutionary computation conference, pp 277–284
Acknowledgements
We appreciate the support of the Frankfurt University of Applied Sciences during the implementation of this project and the Intelligent Computer Entertainment Lab from Ritsumeikan University for their awesome fighting game AI platform which was indispensable for conducting our experiments.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Lam, G.T., Logofătu, D. & Bădică, C. A novel real-time design for fighting game AI. Evolving Systems 12, 169–176 (2021). https://doi.org/10.1007/s12530-020-09351-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12530-020-09351-4