Abstract
In this paper, we present a Q-learning-based multi-agent system designed for Dynamic Difficulty Adjustment (DDA) in a 3D fighting game. Our primary goal is to enhance the player’s gaming experience through dynamic game difficulty adjustments based on their performance. We leverage the Unity game development platform and the ML-Agents framework to implement the Q-learning algorithm, training intelligent agents to adapt the game’s difficulty. Our findings underscore the efficacy of Q-learning and multi-agent systems in improving DDA for video games. In the conclusion section, we discuss potential implications and future directions for our research.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Global Shipments of AR/VR Headsets Decline Sharply in 2022 Following the Prior Year’s Strong Results - IDC - https://www.idc.com/getdoc.jsp?containerId=prUS50467723.
- 2.
This is Meta’s AR/VR hardware roadmap for the next four years - The Verge - https://www.theverge.com/2023/2/28/23619730/meta-vr-oculus-ar-glasses-smartwatch-plans.
- 3.
Third Quarter 2020 U.S. Consumer Spend on Video Game Products Shattered Previous Record Highs - NPD - https://www.npd.com/news/press-releases/2020/the-npd-group-third-quarter-2020.
- 4.
Sony Research Inc - https://ai.sony/projects/gaming_ai/.
References
Asperti, A., Cortesi, D., Pieri, C.D., Pedrini, G., Sovrano, F.: Crawling in rogue’s dungeons with deep reinforcement techniques. IEEE Trans. Games 12(2), 177–186 (2020)
Bojanić, M., Bojanić, G.: Self-learning mechanism for mobile game adjustment towards a player. Appl. Sci. 11, 4412 (2021)
Fernandes, W.R., Levieux, G.: Difficulty pacing impact on player motivation. In: Göbl, B., van der Spek, E., Baalsrud Hauge, J., McCall, R. (eds.) ICEC. LNCS, vol. 13477, pp. 140–153. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-20212-4_11
Fernández-Samillán, D., Guizado-Díaz, C., Ugarte, W.: Story creation algorithm using Q- learning in a 2D action RPG video game. In: IEEE FRUCT, pp. 111–117 (2021)
Lazaridis, A., Vlahavas, I.P.: REIN-2: giving birth to prepared reinforcement learning agents using reinforcement learning agents. Neurocomputing 497, 86–93 (2022)
Leon-Urbano, C., Ugarte, W.: End-to-end electroencephalogram (EEG) motor imagery classification with long short-term. In: IEEE SSCI, pp. 2814–2820 (2020)
Li, Y., et al.: Learning adversarial policy in multiple scenes environment via multi-agent reinforcement learning. Connect. Sci. 33(3), 407–426 (2021)
Moon, J., Choi, Y., Park, T., Choi, J., Hong, J., Kim, K.: Diversifying dynamic difficulty adjustment agent by integrating player state models into Monte-Carlo tree search. Expert Syst. Appl. 205, 117677 (2022)
Oh, I., Rho, S., Moon, S., Son, S., Lee, H., Chung, J.: Creating pro-level AI for a real-time fighting game using deep reinforcement learning. IEEE Trans. Games 14(2), 212–220 (2022)
Pfau, J., Liapis, A., Yannakakis, G.N., Malaka, R.: Dungeons & replicants II: automated game balancing across multiple difficulty dimensions via deep player behavior modeling. IEEE Trans. Games 15(2), 217–227 (2023)
Reis, S., Reis, L.P., Lau, N.: Game adaptation by using reinforcement learning over meta games. Group Decis. Negot. 30(2), 321–340 (2021)
Ugarte, W., Boizumault, P., Loudni, S., Crémilleux, B., Lepailleur, A.: Soft constraints for pattern mining. J. Intell. Inf. Syst. 44(2), 193–221 (2015)
Zhang, Y., Goh, W.: Personalized task difficulty adaptation based on reinforcement learning. User Model. User Adapt. Interact. 31(4), 753–784 (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Maury-Castañeda, N., Villarruel-Vasquez, S., Ugarte, W. (2023). MAS4Games: A Reinforced Learning-Based Multi-agent System to Improve Player Retention in Virtual Reality Video Games. In: da Silva, H.P., Cipresso, P. (eds) Computer-Human Interaction Research and Applications. CHIRA 2023. Communications in Computer and Information Science, vol 1997. Springer, Cham. https://doi.org/10.1007/978-3-031-49368-3_7
Download citation
DOI: https://doi.org/10.1007/978-3-031-49368-3_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-49367-6
Online ISBN: 978-3-031-49368-3
eBook Packages: Computer ScienceComputer Science (R0)