Abstract
Turn-based multi-action adversarial games are challenging scenarios in which each player turn consists of a sequence of atomic actions. The order in which an AI agent runs these atomic actions may hugely impact the outcome of the turn. One of the main challenges of game artificial intelligence is to design a heuristic function to help agents to select the optimal turn to play, given a particular state of the game. In this paper, we report results using the recently developed N-Tuple Bandit Evolutionary Algorithm to tune the heuristic function parameters. For evaluation, we measure how the tuned heuristic function affects the performance of the state-of-the-art evolutionary algorithm Online Evolution Planning. The multi-action adversarial strategy card game Legends of Code and Magic was used as a testbed. Results indicate that the N-Tuple Bandit Evolutionary Algorithm can effectively tune the heuristic function parameters to improve the performance of the agent.
This work has been partially supported by the grant CAS18/00207 from the Spanish Ministry of Education, culture and sports.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
References
Baier, H., Cowling, P.I.: Evolutionary MCTS for multi-action adversarial games. In: Preceedings of the 2018 IEEE Conference on Computational Intelligence and Games, CIG 2018 (2018). https://doi.org/10.1109/CIG.2018.8490403
Bursztein, E.: I am a legend: hacking hearthstone using statistical learning methods. In: 2016 IEEE Conference on Computational Intelligence and Games (CIG) (2016). https://doi.org/10.1109/CIG.2016.7860416
Choe, J.S.B., Kim, J.K.: Enhancing Monte Carlo tree search for playing hearthstone. In: Proceedings of the 1st Conference on Games, GOG 2019 (2019)
Cowling, P., Powley, E., Whitehouse, D.: Information set Monte Carlo tree search. IEEE Trans. Comput. Intell. AI Games 4, 120–143 (2012). https://doi.org/10.1109/TCIAIG.2012.2200894
Dockhorn, A., Frick, M., Akkaya, Ü., Kruse, R.: Predicting opponent moves for improving hearthstone AI. In: Medina, J., et al. (eds.) IPMU 2018. CCIS, vol. 854, pp. 621–632. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-91476-3_51
Justesen, N., Mahlmann, T., Togelius, J.: Online evolution for multi-action adversarial games. In: Squillero, G., Burelli, P. (eds.) EvoApplications 2016. LNCS, vol. 9597, pp. 590–603. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-31204-0_38
Kowalski, J., Miernik, R.: Legends of code and magic (2019). https://jakubkowalski.tech/Projects/LOCM/. Accessed 26 May 2019
Lucas, S.M., et al.: Efficient evolutionary methods for game agent optimisation: model-based is best. In: AAAI Workshop on Games and Simulations for Artificial Intelligence (2019)
Lucas, S.M., Liu, J., Perez-Liebana, D.: The n-tuple bandit evolutionary algorithm for game agent optimisation. In: Proceedings of IEEE Congress on Evolutionary Computation, CEC 2018 (2018)
Myerson, R.: Game Theory: Analysis of Conflict. Harvard University Press, Cambridge (1997)
Perez Liebana, D., Samothrakis, S., Lucas, S., Rohlfshagen, P.: Rolling horizon evolution versus tree search for navigation in single-player real-time games. In: Proceedings of the 15th Annual Conference on Genetic and Evolutionary Computation, GECCO 2013, pp. 351–358 (2013). https://doi.org/10.1145/2463372.2463413
Santos, A., Santos, P.A., Melo, F.S.: Monte Carlo tree search experiments in hearthstone. In: 2017 IEEE Conference on Computational Intelligence and Games, CIG 2017, pp. 272–279 (2017)
Zhang, S., Buro, M.: Improving hearthstone AI by learning high-level rollout policies and bucketing chance node events. In: 2017 IEEE Conference on Computational Intelligence and Games, CIG 2017, pp. 309–316 (2017)
Acknowledgements
The authors want to thank J. Kowalski for his help solving doubts about the LOCM rules and N. Justensen and H. Baier for their help in better understanding their algorithms.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Montoliu, R., Gaina, R.D., Pérez-Liebana, D., Delgado, D., Lucas, S. (2020). Efficient Heuristic Policy Optimisation for a Challenging Strategic Card Game. In: Castillo, P.A., Jiménez Laredo, J.L., Fernández de Vega, F. (eds) Applications of Evolutionary Computation. EvoApplications 2020. Lecture Notes in Computer Science(), vol 12104. Springer, Cham. https://doi.org/10.1007/978-3-030-43722-0_26
Download citation
DOI: https://doi.org/10.1007/978-3-030-43722-0_26
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-43721-3
Online ISBN: 978-3-030-43722-0
eBook Packages: Computer ScienceComputer Science (R0)