Dongxin Tian
ABSTRACT
This study focuses on the application of Deep Q-Networks (DQN) to train AI agents to play bullet hell games. We built a training environment and utilized ray casting to collect input data for the network. Two similar network model architectures were evaluated and compared to maximize the learning efficiency of our AI agent. The trained AI demonstrates commendable performance and the ability to learn and adapt strategies into gameplay. However, while the AI agent displayed potential in mastering gameplay dynamics, there remain several challenges to integrating the agent to complete commercial bullet hell games. These challenges may provide directions for future research.
- M. Johnson, “Bullet hell: The globalized growth of Danmaku Games and the digital culture of High Scores and World Records,” Transnational Contexts of Culture, Gender, Class, and Colonialism in Play, pp. 17–42, 2016. doi:10.1007/978-3-319-43817-7_2Google Scholar
Cross Ref
- K. Shao, Z. Tang, Y. Zhu, N. Li, and D. Zhao, “A survey of deep reinforcement learning in video games,” arXiv.org, 2019, https://arxiv.org/abs/1912.10944Google Scholar
- V. Mnih , “Human-level control through deep reinforcement learning,” Nature, vol. 518, no. 7540, pp. 529–533, 2015. doi:10.1038/nature14236Google ScholarCross Ref
- Team Shanghai Alice, “Touhou Kaeidzuka ∼ Phantasmagoria of Flower View”, 2005.Google Scholar
- 4gamer.net, https://www.4gamer.net/specials/shanghai_alice/zone_k.shtml, 2005.Google Scholar
- D. Ostuni and E. T. Galante, "Towards an AI playing Touhou from pixels: a dataset for real-time semantic segmentation," 2021 IEEE Conference on Games (CoG), Copenhagen, Denmark, 2021, pp. 1-5, doi: 10.1109/CoG52621.2021.9619112.Google ScholarDigital Library
- C. J. Watkins and P. Dayan, “Q-learning,” Machine Learning, vol. 8, no. 3–4, pp. 279–292, 1992. doi:10.1007/bf00992698Google ScholarDigital Library
- N. Justesen, P. Bontrager, J. Togelius and S. Risi, "Deep Learning for Video Game Playing," in IEEE Transactions on Games, vol. 12, no. 1, pp. 1-20, March 2020, doi: 10.1109/TG.2019.2896986.Google ScholarCross Ref
- R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. MIT press, 2018.Google ScholarDigital Library
- I. Makarov, M. Tokmakov, and L. Tokmakova, Imitation of Human Behavior in 3D-Shooter Game, no. 1452, 2015.Google Scholar
- A. Aktaş and E. Orçun, “A survey of computer game development,” The Journal of Defense Modeling and Simulation: Applications, Methodology, Technology, vol. 13, no. 2, pp. 239–251, 2014. doi:10.1177/1548512914554405Google ScholarCross Ref
- A. Z. Aktaş and E. R. Orçun, “A real-time strategy game, ‘Gallipoli wars,’ as a centennial tribute to the gallipoli campaign (1915–2015),” The Journal of Defense Modeling and Simulation: Applications, Methodology, Technology, vol. 14, no. 3, pp. 285–301, 2016. doi:10.1177/1548512916649455Google ScholarCross Ref
- D. Wang and A. -H. Tan, "Creating Autonomous Adaptive Agents in a Real-Time First-Person Shooter Computer Game," in IEEE Transactions on Computational Intelligence and AI in Games, vol. 7, no. 2, pp. 123-138, June 2015, doi: 10.1109/TCIAIG.2014.2336702.Google ScholarCross Ref
- H. Li , “Research on overfitting of Deep Learning,” 2019 15th International Conference on Computational Intelligence and Security (CIS), 2019. doi:10.1109/cis.2019.00025Google ScholarCross Ref
- 2013 Luke McMillan, “The origin of the ‘shmup’ genre: A historical study,” Game Developer, https://www.gamedeveloper.com/design/the-origin-of-the-shmup-genre-a-historical-study (accessed Sep. 26, 2023).Google Scholar
Index Terms
- Artificial Intelligent Player for Bullet Hell Games Based on Deep Q-Networks
Recommendations
Iterated n-player games on small-world networks
GECCO '11: Proceedings of the 13th annual conference on Genetic and evolutionary computationThe evolution of strategies in iterated multi-player social dilemma games is studied on small-world networks. Two different games with varying reward values - the N-player Iterated Prisoner's Dilemma (N-IPD) and the N-player Iterated Snowdrift game (N-...
Analyzing n-player impartial games
AbstractCombinatorial game theory is the study of two player perfect information games. While work has been done in the past on expanding this field to include n-player games we present a unique method which guarantees a single winner. Specifically our ...
n-Player impartial combinatorial games with random players
In this paper we develop a method of analyzing n-player impartial combinatorial games where n - 1 players behave optimally while one of the players plays randomly. More specifically, we develop a function g r that maps impartial combinatorial games to n-...
Comments