skip to main content
10.1145/3450337.3483453acmconferencesArticle/Chapter ViewAbstractPublication Pageschi-playConference Proceedingsconference-collections
extended-abstract

Cognitive Shadowing for Learning Opponents in a Strategy Game Experiment: Using Machine Learning to Counter Players’ Strategies

Published: 15 October 2021 Publication History

Abstract

While non-player opponents in commercial video games often rely on simple artificial intelligence techniques, machine learning techniques that capture human strategies could make them more engaging. Cognitive Shadow is a prototype tool that combines several artificial intelligence techniques to continuously model human decision-making patterns during tasks that require categorical decision-making. The present study aims to assess the potential of Cognitive Shadow to create learning opponents that will counter the player's decisions in a strategy game, making it more challenging and engaging. The game developed to this end is a more complex version of rock-paper-scissors, set within the context of a wizards’ duel. Each participant (Player 1) took part in three game sessions of 12 battles (each including five rounds), only being told that they would face a non-player opponent. During Session 1, Cognitive Shadow was in learning mode, thus the non-player opponent (Player 2) chose its plays at random. During Session 2, Cognitive Shadow was active and helped counter participants’ decisions without their knowledge. Before Session 3, participants were informed that their opponent was using machine learning to anticipate and counter their strategy. The results showed that Player 2 was more effective with the help of Cognitive Shadow, having won significantly more battles in Sessions 2 and 3 than in Session 1. In addition, the level of engagement reported by human players increased significantly in Session 3. These results indicate that cognitive shadowing can be used in a strategy game to increase engagement when players are aware of the learning behavior.

References

[1]
Georgios N. Yannakakis, and Julian Togelius. 2015. A panorama of artificial and computational intelligence in games. IEEE Transactions on Computational Intelligence and AI in Games, 7(4), 317-335. https://doi.org/10.1109/TCIAIG.2014.2339221
[2]
Johannes Pfau, Jan David Smeddinck, and Rainer Malaka. 2018. Towards Deep Player Behavior Models in MMORPGs. Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in Play, 381 392. https://doi.org/10.1145/3242671.3242706
[3]
Christophe Berner 2019. Dota 2 with Large Scale Deep Reinforcement Learning. arXiv:1912.06680 [cs, stat]. http://arxiv.org/abs/1912.06680
[4]
Heather O'Brien. 2016. Theoretical perspectives on user engagement. In Heather O'Brien & Paul Cairns (Eds.), Why Engagement Matters (pp. 1-26). Springer International Publishing. https://doi.org/10.1007/978-3-319-27446-1_1
[5]
Johannes Pfau, Jan David Smeddinck, and Rainer Malaka. 2019. Deep player behavior models: Evaluating a novel take on dynamic difficulty adjustment. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, 1–6. https://doi.org/10.1145/3290607.3312899
[6]
Stefano Triberti, Anna Flavia Di Natale, and Andrea Gaggioli. 2021. Flowing Technologies: The Role of Flow and Related Constructs in Human-Computer Interaction. In Corinna Peifer & Stefan Engeser (Eds.), Advances in Flow Research (pp. 393–416). Springer International Publishing. https://doi.org/10.1007/978-3-030-53468-4_15
[7]
Christian López, and Conrad Tucker. 2018. Toward Personalized Adaptive Gamification: A Machine Learning Model for Predicting Performance. IEEE transactions on Games, 12(2), 155-168. https://doi.org/10.1109/TG.2018.2883661
[8]
Julian Frommel, Fabian Fischbach, Katja Rogers, and Michael Weber. 2018. Emotion-based Dynamic Difficulty Adjustment Using Parameterized Difficulty and Self-Reports of Emotion. Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in Play, 163 171. https://doi.org/10.1145/3242671.3242682
[9]
Johannes Pfau, Jan David Smeddinck, and Rainer Malaka. 2020. Enemy Within: Long-term Motivation Effects of Deep Player Behavior Models for Dynamic Difficulty Adjustment. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 1 10. https://doi.org/10.1145/3313831.3376423
[10]
Hendrik Baier, Adam Sattaur, Edward J. Powley, Sam Devlin, Jeff Rollason, and Peter I. Cowling. 2019. Emulating human play in a leading mobile card game. IEEE Transactions on Games, 11(4), 386-395. https://doi.org/10.1109/TG.2018.2835764
[11]
Juan Ortega, Noor Shaker, Julian Togelius, and Georgios N. Yannakakis. 2013. Imitating human playing styles in super mario bros. Entertainment Computing, 4(2), 93-104. https://doi.org/10.1016/j.entcom.2012.10.001
[12]
Philip Hingston (Ed.). 2012. Believable Bots: Can computers play like people? Springer.
[13]
Jeppe Theiss Kristensen, Arturo Valdivia, and Paolo Burelli. 2020. Estimating Player Completion Rate in Mobile Puzzle Games Using Reinforcement Learning. In 2020 IEEE Conference on Games (CoG) (pp. 636-639). https://doi.org/10.1109/CoG47356.2020.9231581
[14]
Lynda Aiman-Smith, Steven E. Scullen, and Steve H. Barr. 2002. Conducting studies of decision making in organizational contexts: A tutorial for policy-capturing and other regression-based techniques. Organizational Research Methods, 5(4), 388-414. https://doi.org/10.1177/109442802237117
[15]
Karen Nokes, and Gerard P. Hodgkinson. 2017. Chapter 5: Policy-capturing: An ingenious technique for exploring the cognitive bases of work-related decisions. In Robert. J. Galavan, Kristian. J. Sund, & Gerard P. Hodgkinson (Eds.), Methodological Challenges and Advances in Managerial and Organizational Cognition (New Horizons in Managerial and Organizational Cognition, Vol. 2, pp. 95–121). Emerald Publishing Limited. https://doi.org/10.1108/S2397-52102017005
[16]
Daniel Lafond, Katherine Labonté, Aren Hunter, Heather F. Neyedli, and Sébastien Tremblay. 2019. Judgment analysis for real-time decision support using the Cognitive Shadow policy-capturing system. In Tareq Ahram, Redha Taiar, Serge Colson, & Arnaud Choplin (Eds.), Human Interaction and Emerging Technologies (Vol. 1018, pp. 78–83). Springer International Publishing. https://doi.org/10.1007/978-3-030-25629-6_13
[17]
Daniel Lafond, Benoît Roberge-Vallières, François Vachon, and Sébastien Tremblay. 2017. Judgment analysis in a dynamic multitask environment: Capturing nonlinear policies using decision trees. Journal of Cognitive Engineering and Decision Making, 11(2), 122-135. https://doi.org/10.1177/1555343416661889
[18]
David H. Wolpert. 1996. The lack of a priori distinctions between learning algorithms. Neural computation, 8(7), 1341-1390. https://doi.org/10.1162/neco.1996.8.7.1341
[19]
Katherine Labonté, Daniel Lafond, Aren Hunter, Heather F. Neyedli, and Sébastien Tremblay. 2020. Comparing Two Decision Support Modes Using the Cognitive Shadow Online Policy-Capturing System. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 64(1), 1125-1129. https://doi.org/10.1177/1071181320641270
[20]
Katherine Labonté, Daniel Lafond, Bénédicte Chatelais, Aren Hunter, Folakemi Akpan, Heather F. Neyedli, and Sébastien Tremblay. in press. Combining Process Tracing and Policy Capturing Techniques for Judgment Analysis in an Anti-Submarine Warfare Simulation. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 65).
[21]
Heather L. O'Brien, Paul Cairns, and Mark Hall. 2018. A practical approach to measuring user engagement with the refined User Engagement Scale (UES) and new UES short form. International Journal of Human-Computer Studies, 112, 28–39. https://doi.org/10.1016/j.ijhcs.2018.01.004
[22]
Fons Van de Vijver and Ronald K. Hambledon. 1996. Translating tests: Some practical guidelines. European Psychologist, 1(2), 89-99.
[23]
Joseph A. Gliem and Rosemary R. Gliem. 2003. Calculating, interpreting, and reporting Cronbach's alpha reliability coefficient for Likert-type scales. Midwest Research-to-Practice Conference in Adult, Continuing, and Community Education. Columbus, OH.
[24]
Georgios N. Yannakakis, and John Hallam. 2004. Evolving opponents for interesting interactive computer games. 8th International Conference on the Simulation of Adaptive Behavior (SAB’04); From Animals to Animats 8, Los Angeles. 499-508.
[25]
Andrew K. Przybylski, C. Scott Rigby, and Richard M. Ryan. 2010. A motivational model of video game engagement. Review of General Psychology, 14(2), 154–166. https://doi.org/10.1037/a0019440
[26]
Florian Brühlmann, Philipp Baumgartner, Günter Wallner, Simone Kriglstein, and Elisa D. Mekler. 2020. Motivational profiling of league of legends players. Frontiers in Psychology, 11, 1307. https://doi.org/10.3389/fpsyg.2020.01307
[27]
Kellie Vella, Christopher James Koren, and Daniel Johnson. 2017. The impact of agency and familiarity in cooperative multiplayer games. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play (pp. 423-434). https://doi.org/10.1145/3116595.3116622

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CHI PLAY '21: Extended Abstracts of the 2021 Annual Symposium on Computer-Human Interaction in Play
October 2021
414 pages
ISBN:9781450383561
DOI:10.1145/3450337
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 October 2021

Check for updates

Author Tags

  1. Cognitive Shadow
  2. Games
  3. Player modeling
  4. Strategy game

Qualifiers

  • Extended-abstract
  • Research
  • Refereed limited

Funding Sources

Conference

CHI PLAY '21
Sponsor:

Acceptance Rates

Overall Acceptance Rate 421 of 1,386 submissions, 30%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 184
    Total Downloads
  • Downloads (Last 12 months)26
  • Downloads (Last 6 weeks)6
Reflects downloads up to 14 Feb 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media