skip to main content
10.1145/3440840.3440856acmotherconferencesArticle/Chapter ViewAbstractPublication PagesciisConference Proceedingsconference-collections
research-article
Open access

A Reinforcement Learning-Based Classification Symbiont Agent for Dynamic Difficulty Balancing

Published: 15 February 2021 Publication History

Abstract

AdaptiveSGA is a mechanism for achieving Adaptive Game AI-based Dynamic Difficulty Balancing in games. AdaptiveSGA is based on the Symbiotic Game Agent model and, therefore, leverages the advantages of biological symbiosis. Within the AdaptiveSGA architecture, the classification symbiont agent is responsible for the dynamic difficulty balancing component. Current work proposes the use of a classification symbiont agent that makes use of reinforcement learning to optimise dynamic difficulty balancing in order to match the opponent's skill. Current work also introduces three different types of decision-making algorithms that can be used by decision-making symbiont agents to display different kinds of behaviour. The ability to reproduce different kinds of NPC behaviour forms the adaptive game AI component of AdaptiveSGA. Experimental results showed that the reinforcement learning-based classification symbiont agent can achieve an even game with opponents and can further help minimise the number of draws.

References

[1]
Ashey Noblega, Aline Paes, and Esteban Clua. 2019. Towards Adaptive Deep Reinforcement Game Balancing. In ICAART, 693-700.
[2]
Maurice Hendrix, Tyrone Bellamy-Wood, Sam McKay, Victoria Bloom, and Ian Dunwell. 2019. Implementing Adaptive Game Difficulty Balancing in Serious Games. IEEE Transactions on Games 11, 4, 320-327.
[3]
Johannes Pfau, Jan David Smeddinck, Rainer Malaka. 2019. Deep Player Behavior Models: Evaluating a Novel Take on Dynamic Difficulty Adjustment. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, 1-6.
[4]
Johannes Pfau, Jan David Smeddinck, Rainer Malaka. 2020. Enemy Within: Long-Term Motivation Effects of Deep Player Behavior Models for Dynamic Difficulty Adjustment. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 1-10.
[5]
Mirna Paula Silva, Victor do Nascimento Silva, and Luiz Chaimowicz. 2015. Dynamic Difficulty Adjustment through an Adaptive AI. In 2015 14th Brazilian Symposium on Computer Games and Digital Entertainment (SBGames), 173-182.
[6]
Emanuel Carneiro, Adilson Cunha. 2012. An Adaptive Game AI Architecture. In SBC - Proceedings of SBGames 2012, 21-24.
[7]
Pieter Spronck, Marc Ponsen, Ida Sprinkhuizen-Kuyper, and Eric Postma. 2006. Adaptive game AI with dynamic scripting. Machine Learning 63, 3, 217-248.
[8]
Richard S. Sutton and Andrew G. Barto. 2015. Reinforcement Learning: An Introduction (Second edition, in progress). Retrieved July 14, 2020 from https://web.stanford.edu/class/psych209/Readings/SuttonBartoIPRLBook2ndEd.pdf
[9]
Christopher J.C.H. Watkins and Peter Dayan. 1992. Q-learning. Machine Learning 8, 3-4, 279-292.
[10]
Frank G. Glavin and Michael G. Madden. 2018. Skilled Experience Catalogue: A Skill-Balancing Mechanism for Non-Player Characters using Reinforcement Learning. In 2018 IEEE Conference on Computational Intelligence and Games (CIG), 1-8.
[11]
Simão Reis, Luís Paulo Reis, and Nuno Lau. 2020. Game Adaptation by Using Reinforcement Learning Over Meta Games. Group Decision and Negotiation, 1-20.
[12]
Takahiro Kusano, Yunshi Liu, Pujana Paliyawan, Ruck Thawonmas, and Tomohiro Harada. 2019. Motion Gaming AI using Time Series Forecasting and Dynamic Difficulty Adjustment for Improving Exercise Balance and Enjoyment. In 2019 IEEE Conference on Games.
[13]
Chin Hiong Tan, Kay Chen Tan, and Arthur Tay. 2011. Dynamic game difficulty scaling using adaptive behavior-based AI. IEEE Transactions on Computational Intelligence and AI in Games 3, 4, 289-301.
[14]
Siphesihle Philezwini Sithungu and Elizabeth Marie Ehlers. 2020. Adaptive Game AI-Based Dynamic Difficulty Scaling via the Symbiotic Game Agent. In Intelligent Information Processing X, 107-117.
[15]
Angela E. Douglas. 2010. The Symbiotic Habit. Princeton University Press.
[16]
Mohammed Abdullahi, Md Asri Ngadi, Salihu Idi Dishing, Shafi'i Muhammad Abdulhamid, and Mohammed Joda Usman. 2020. A survey of symbiotic organisms search algorithms and applications. Neural Computing and Applications 32, 2, 547-566.
[17]
Siphesihle Philezwini Sithungu, Duncan Anthony Coulter, and Elizabeth Marie Ehlers. 2019. Using Genetic Programming and Decision Trees for Team Evolution. In ACM International Conference Proceeding Series, 28-40.
[18]
Michael Littman. 2001. Mark Decision Processes. International Encyclopedia of the Social & Behavioral Sciences. Pergamon, Oxford, 9240-9242.
[19]
Deeplizard. What do Reinforcement Learning Algorithms Learn - Optimal Policies - deeplizard. Retrieved July 14, 2020 from https://deeplizard.com/learn/video/rP4oEpQbDm4

Cited By

View all
  • (2024)Technologies to Support Adaptable Game Design: A Systematic Mapping StudyJournal of the Brazilian Computer Society10.5753/jbcs.2024.309030:1(69-101)Online publication date: 26-Apr-2024
  • (2022)A systematic mapping study on digital game adaptation dimensionsProceedings of the 21st Brazilian Symposium on Human Factors in Computing Systems10.1145/3554364.3559122(1-14)Online publication date: 17-Oct-2022

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
CIIS '20: Proceedings of the 2020 3rd International Conference on Computational Intelligence and Intelligent Systems
November 2020
135 pages
ISBN:9781450388085
DOI:10.1145/3440840
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 February 2021

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. classification symbiont agent
  2. dynamic difficulty balancing
  3. q-learning
  4. symbiotic game agent

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

CIIS 2020

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)191
  • Downloads (Last 6 weeks)26
Reflects downloads up to 08 Mar 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Technologies to Support Adaptable Game Design: A Systematic Mapping StudyJournal of the Brazilian Computer Society10.5753/jbcs.2024.309030:1(69-101)Online publication date: 26-Apr-2024
  • (2022)A systematic mapping study on digital game adaptation dimensionsProceedings of the 21st Brazilian Symposium on Human Factors in Computing Systems10.1145/3554364.3559122(1-14)Online publication date: 17-Oct-2022

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Login options

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media