Elsevier

Expert Systems with Applications

Volume 40, Issue 16, 15 November 2013, Pages 6258-6265
Expert Systems with Applications

Online behavior change detection in computer games

https://doi.org/10.1016/j.eswa.2013.05.059Get rights and content

Highlights

  • We apply a change detection algorithm to the game’s domain.

  • Three extensions are proposed to the algorithm.

  • Results suggest the improvement provided by the extensions.

  • Results show our approach surpasses other change detection algorithms.

Abstract

Player Modelling has been receiving much attention from the game community in the recent years. The ability to build accurate models of player behavior can be useful in many aspects of a game. One important aspect is the tracking of a player’s behavior along time, informing every time a change is perceived. This way, the game Artificial Intelligence can adapt itself to better respond to this new behavior. In order to build models of player behavior, researchers frequently resort to Machine Learning techniques. Such methods work on previously recorded game metrics representing player’s interactions with the game environment. However, if the player changes styles over time, the constructed models get out of date. In order to address this drawback, this work proposes the use of and incremental learning technique to track a player’s behavior during his/her interaction with the game environment. Our approach attempts to automatically detect the moments in time when the player changes behavior. We apply a change detection technique from the area of Data Stream Mining that is based on incremental clustering and novelty detection. We also propose three modifications to the original technique, in order to formalize change detection, improve detection rate and reduce detection delay. Simulations were performed considering data produced by the Unreal Tournament game, showing the applicability of the method to online tracking of a player’s behavior and informing whenever behavior changes occur.

Introduction

Modeling a player’s behavior in a particular game scenario has been the focus of attention of many research work developed in the recent years (Smith, Lewis, Hullett, Smith, & Sullivan, 2011). The act of tracking game metrics and using them afterwards to construct representative models of players is generally known as Player Modeling (PM). In order to learn from the collected game metrics, PM usually relies on Machine Learning (ML) techniques. The types of models learned vary, because PM can be applied to different objectives.

Generally, there are three major objectives for the application of PM techniques. First, game metrics can be used to learn a player’s style or strategy. This is particularly useful for automatically building game characters that present a more human-like behavior (Galli et al., 2009, Weber and Mateas, 2009, Zanetti and Rhalibi, 2004). Second, it can be applied on metrics collected after a game has been launched in order to check whether the game design is appropriate and allows the existence of different player styles (Drachen, Canossa, & Yannakakis, 2009). In this case, the idea is to explore the types of players encountered and use this information to improve next games of the same series or genre. There is still another objective of PM, which is to be able to track or identify characteristics of a player while the game is in progress (Delalleau et al., 2011, Hunicke and Chapman, 2004, Martinez et al., 2010, Yannakakis and Hallam, 2009, Yannakakis and Maragoudakis, 2005). This is, perhaps, the most interesting application of PM, allowing for the adaptation of a game’s responses to different players’ needs along time.

Although they have different objectives, all the aforementioned approaches share the same basic idea: employ the statistics collected through the interaction of a player to improve the game’s quality and/or responses. The data collection process is generally separated from the model learning step, which means that models are induced in an offline manner, using data previously gathered from players. This allows the application of traditional ML techniques, which require a training set of examples and multiple iterations to induce an accurate model. Some exceptions to this methodology do exist, like the work by Hunicke and Chapman (2004), in which game metrics are online collected and used to induce models.

Following the same steps as Hunicke and Chapman (2004), this work proposes to track the player’s behavior during his/her interaction with the game environment. This means that model induction is done completely online. Our objective is to be able to identify moments of possible changes in the behavior of a player based on data collected due to player/environment interactions. In order to accomplish this objective, a change detection mechanism from the area of Data Stream Mining is applied (Vallim, Filho, de Mello, de Carvalho, & Gama, 2014). This method, called M-DBScan, is based on incremental online clustering and novelty detection. We apply the method on the Unreal Tournament 20041 (UT2004) game scenario with three proposed extensions. The first one attempts to improve the rate of truly detected changes, while the second defines a more formal way of detecting when a change happens. Finally, the third modification intends to reduce the delay for change detection. Our experimental results confirm these modifications increase the capability of the method to detect true behavior changes under a small delay. We also apply two other change detection techniques from the Data Stream Mining area and compare results with the extended version of M-DBScan proposed in this work.

This paper is organized as follows. Section 2 presents related work in the area of PM. Next, Section 3 brings the main concepts of M-DBScan, i.e., the change detection algorithm used in this work. Section 4 presents how M-DBScan tracks a player’s behavior, together with the extensions proposed to improve the algorithm’s performance. Experimental results obtained with the Unreal Tournament game are shown and discussed in Section 5. Finally, Section 6 concludes this work and draws some future research directions.

Section snippets

Player modeling

The term Player Modeling (PM) can be used in a variety of scenarios. In the work by Smith et al. (2011), a taxonomy is proposed in order to distinguish the major existing PM applications and techniques. This taxonomy is based on four independent facets that distinguish player models based on who they apply to, what they are used for, the details they model and, finally, how the models are induced. Generally, PM has been used for designing AI players with human-like behavior (Galli et al., 2009,

Micro-clustering DBScan

In order to build online models to represent the evolving behavior of players, the learning algorithm needs to constantly add new data to the induced model, updating its structure incrementally without the need to re-induce the whole model from scratch. In this scenario, traditional ML algorithms are not appropriate, since they need to be re-trained everytime new data becomes available. To deal with this scenario, algorithms from the area known as Data Stream Mining are more suitable.

Data

Detecting changing player behavior

In order to detect changes in the behavior of a player during gameplay, this work proposes the application of the M-DBScan algorithm. In this section, we first explain how to apply the original M-DBScan algorithm to game scenarios. Afterwards, we present the extensions proposed in this work.

Experiments

In order to validate the application of M-DBScan to the games domain, as well as the extensions presented in this work, simulations were created in the First Person Shooter (FPS) Unreal Tournament 2004 (UT2004) using the Pogamut IDE (Gemrot et al., 2009). Our hypothesis is that M-DBScan, and the extensions proposed in this work, will prove to be useful on the task of online learning of a player’s behavior and detecting possible behavior changes.

Conclusions

The online construction of player’s behavioral models is an interesting approach to PM. Being able of tracking the player in an online manner, and constantly updating the model that represents this player, makes it possible to understand how the player behavior is evolving and to capture subtle changes. However, only a few attempts have been made in this direction, with the majority of works aiming at using player models induced offline. In this work, we proposed the application of the M-DBScan

Acknowledgment

The authors thank the Brazilian Research Councils Fapesp (Fundação de Amparo à Pesquisa do Estado de São Paulo) and CNPq (Conselho Nacional de Desenvolvimento Científico e Tecnológico).

References (27)

  • C. Aggarwal et al.

    A framework for clustering evolving data streams

  • S. Marsland et al.

    A self-organising network that grows when required

    Neural Networks

    (2002)
  • M.K. Albertini et al.

    A self-organizing neural network to approach novelty detection

  • Bifet, A., & Gavaldà, R. (2007). Learning from time-changing data with adaptive windowing. In SIAM international...
  • Cao, F., Ester, M., Qian, W., & Zhou, A. (2006). Density-based clustering over and evolving data stream with noise. In...
  • Charles, D., & Black, M. (2004). Dynamic player modeling: a framework for player-centric digital games. In Proceedings...
  • Delalleau, O., Contal, E., Thibodeau-Laufer, E., Ferrari, R. C., Bengio, Y., & Zhang, F. (2011). Beyond skill rating:...
  • J. Demšar

    Statistical comparisons of classifiers over multiple data sets

    The Journal of Machine Learning Research

    (2006)
  • A. Drachen et al.

    Player modeling using self-organization in tomb raider: underworld

  • M. Ester et al.

    A density-based algorithm for discovering clusters in large spatial databases with noise

  • L. Galli et al.

    Learning a context-aware weapon selection policy for unreal tournament III

  • J. Gama

    Knowledge discovery from data streams

    (2010)
  • J. Gama et al.

    Data stream processing

  • Cited by (11)

    • Varying density method for data stream clustering

      2020, Applied Soft Computing Journal
    • Detecting pattern changes in individual travel behavior: A Bayesian approach

      2018, Transportation Research Part B: Methodological
      Citation Excerpt :

      Beyond travel behavior, a few studies have been done to detect pattern changes in other aspects of human behavior. For example, Vallim et al. (2013) also used an entropy-based measure to track changes in the behavior of computer game players. The use of entropy is a convenient choice for behavior change detection.

    • Automatic visual detection of human behavior: A review from 2000 to 2014

      2015, Expert Systems with Applications
      Citation Excerpt :

      Different machine learning techniques have been used over the time to build models of human behavior. Vallim, Andrade Filho, De Mello, and De Carvalho (2013) proposed a method that uses incremental learning technique to track a player’s behavior during his interaction with the game. A change detection technique from the area of stream mining was applied based on an incremental clustering and novelty detection method.

    • Proposal of a new stability concept to detect changes in unsupervised data streams

      2014, Expert Systems with Applications
      Citation Excerpt :

      In the particular case of learning from unsupervised data, change detection is even more challenging because there is no label information available in the stream. Attempting to accurately detect when data characteristics change, several recent works apply clustering techniques in the stream of data and, afterwards, monitor cluster evolution by means of novelty detection methods (Albertini & Mello, 2010; Marsland et al., 2002; Vallim et al., 2013). The rationale assumed by these algorithms consider that changes observed in induced clustering models correspond to changes in data characteristics.

    View all citing articles on Scopus
    View full text