skip to main content
research-article

Objective Difficulty-Skill Balance Impacts Perceived Balance but Not Behaviour: A Test of Flow and Self-Determination Theory Predictions

Published: 04 October 2023 Publication History

Abstract

Flow and self-determination theory predict that game difficulty in balance with player skill maximises enjoyment and engagement, mediated by attentive absorption or competence. Yet recent evidence and methodological concerns are challenging this view, and key theoretical predictions have remained untested, importantly which objective difficulty-skill ratio is perceived as most balanced. To test these, we ran a preregistered study (n=309) using a Go-like 2-player game with an AI opponent, randomly assigning players to one of three objective difficulty-skill ratios (AI plays to win, draw, or lose) over five matches. The AI successfully manipulated objective balance, with the draw condition perceived as most balanced. However, balance did not impact play behaviour, nor did we find the predicted uniform 'inverted-U' between balance and positive play experiences. Importantly, we found both theories too underspecified to severely test. We conclude that balance and competence likely matter less for behavioural engagement than commonly held. We propose alternative factors such as player appraisals, novelty, and progress, and debate the value and challenges of theory-testing work in games HCI.

References

[1]
Vero Vanden Abeele, Katta Spiel, Lennart Nacke, Daniel Johnson, and Kathrin Gerling. 2020. Development and validation of the player experience inventory: A scale to measure player experiences at the level of functional and psychosocial consequences. International Journal of Human-Computer Studies 135 ( 2020 ), 102370.
[2]
Sami Abuhamdeh. 2020. Investigating the “ ow” experience: Key conceptual and operational issues. Frontiers in psychology 11 ( 2020 ), 158.
[3]
Sami Abuhamdeh and Mihaly Csikszentmihalyi. 2012. Attentional involvement and intrinsic motivation. Motivation and Emotion 36, 3 ( 2012 ), 257-267. https://doi.org/10.1007/s11031-011-9252-7
[4]
Sami Abuhamdeh and Mihaly Csikszentmihalyi. 2012. The Importance of Challenge for the Enjoyment of Intrinsically Motivated, Goal-Directed Activities. Personality and Social Psychology Bulletin 38, 3 ( 2012 ), 317-330. https://doi.org/ 10.1177/0146167211427147
[5]
Sami Abuhamdeh, Mihaly Csikszentmihalyi, and Baland Jalal. 2015. Enjoying the possibility of defeat: Outcome uncertainty, suspense, and intrinsic motivation. Motivation and Emotion 39, 1 ( 2015 ), 1-10.
[6]
Sultan A Alharthi, Olaa Alsaedi, Phoebe O. Toups, Theresa Jean Tanenbaum, and Jessica Hammer. 2018. Playing to wait: A taxonomy of idle games. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1-15.
[7]
Nick Ballou. 2023. A Manifesto for More Productive Psychological Games Research. Games: Research and Practice 1, 1 ( 2023 ).
[8]
Nick Ballou, Sebastian Deterding, April Tyack, Elisa D Mekler, Rafael A Calvo, Dorian Peters, Gabriela VillalobosZúñiga, and Selen Turkay. 2022. Self-Determination Theory in HCI: Shaping a Research Agenda. In CHI Conference on Human Factors in Computing Systems Extended Abstracts. 1-6.
[9]
Alexander Becker and Daniel Görlich. 2020. What is Game Balancing?-An Examination of Concepts. ParadigmPlus 1, 1 ( 2020 ), 22-41. https://doi.org/10.55969/paradigmplus.v1n1a2
[10]
D. E. Berlyne. 1960. Con ict, arousal, and curiosity. McGraw-Hill, New York.
[11]
Mark Blythe and Marc Hassenzahl. 2003. The semantics of fun: Di erentiating enjoyable experiences. In Funology. Springer, 91-100.
[12]
Cameron B Browne, Edward Powley, Daniel Whitehouse, Simon M Lucas, Peter I Cowling, Philipp Rohlfshagen, Stephen Tavener, Diego Perez, Spyridon Samothrakis, and Simon Colton. 2012. A survey of monte carlo tree search methods. IEEE Transactions on Computational Intelligence and AI in games 4, 1 ( 2012 ), 1-43.
[13]
Jenova Chen. 2007. Flow in games (and everything else). Commun. ACM 50, 4 ( 2007 ), 31-34.
[14]
Andy Cockburn, Carl Gutwin, and Alan Dix. 2018. Hark no more: On the preregistration of chi experiments. In Proceedings of the 2018 chi conference on human factors in computing systems. 1-12.
[15]
Tom Cole, Paul Cairns, and Marco Gillies. 2015. Emotional and functional challenge in core and avant-garde games. In Proceedings of the 2015 annual symposium on computer-human interaction in play. 121-126.
[16]
Günter Cornett and Alvydas Jakeliunas. 2003. Hey, That's My Fish! https://boardgamegeek.com/boardgame/8203/heys-my-sh
[17]
Mihaly Csikszentmihalyi. 1991. Flow: The psychology of optimal experience. HarperPerennial New York.
[18]
Mihaly Csikszentmihalyi. 2014. Flow and the Foundations of Positive Psychology. Springer Netherlands, Dordrecht.
[19]
Mihaly Csikszentmihalyi and Jeanne Nakamura. 2010. E ortless attention in everyday life: A systematic phenomenology. E ortless attention: A new perspective in the cognitive science of attention and action ( 2010 ), 179-189.
[20]
Joe Cutting, Sebastian Deterding, Simon Demediuk, and Nick Sephton. 2023. Di culty-skill balance does not a ect engagement and enjoyment: a pre-registered study using arti cial intelligence-controlled di culty. Royal Society Open Science 10, 2 ( 2023 ), 220274.
[21]
Joe Cutting, David Gundry, and Paul Cairns. 2019. Busy doing nothing? What do players do in idle games? International journal of human-computer studies 122 ( 2019 ), 133-144.
[22]
Edward L. Deci and Richard M. Ryan. 1985. Intrinsic Motivation and Self-Determination in Human Behavior. Plenum Press, New York.
[23]
Simon Demediuk, Marco Tamassia, William L Ra e, Fabio Zambetta, Xiaodong Li, and Florian Mueller. 2017. Monte carlo tree search based algorithms for dynamic di culty adjustment. In 2017 IEEE conference on computational intelligence and games (CIG). IEEE, 53-59.
[24]
Alena Denisova, Paul Cairns, Christian Guckelsberger, and David Zendle. 2020. Measuring perceived challenge in digital games: Development & validation of the challenge originating from recent gameplay interaction scale (CORGIS). International Journal of Human-Computer Studies 137 ( 2020 ), 102383. https://doi.org/10.1016/j.ijhcs. 2019.102383
[25]
Alena Denisova, Christian Guckelsberger, and David Zendle. 2017. Challenge in Digital Games: Towards Developing a Measurement Tool. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems. ACM, Denver Colorado USA, 2511-2519. https://doi.org/10.1145/3027063.3053209
[26]
Sebastian Deterding, Marc Malmdorf Andersen, Julian Kiverstein, and Mark Miller. 2022. Mastering uncertainty: A predictive processing account of enjoying uncertain success in video game play. Frontiers in Psychology 13 ( 2022 ), 924953. https://doi.org/10.3389/fpsyg. 2022.924953
[27]
Florian Echtler and Maximilian Häußler. 2018. Open source, open science, and the replication crisis in HCI. In Extended abstracts of the 2018 CHI conference on human factors in computing systems. 1-8.
[28]
Arpad E Elo. 1978. The rating of chessplayers, past and present. Arco Pub.
[29]
Jessica Kay Flake and Eiko I Fried. 2020. Measurement Schmeasurement: Questionable Measurement Practices and How to Avoid Them. Advances in Methods and Practices in Psychological Science 3, 4 ( 2020 ), 456-465. https: //doi.org/10.1177/2515245920952393
[30]
Carlton J. Fong, Diana J. Zaleski, and Jennifer Kay Leach. 2015. The challenge-skill balance and antecedents of ow: A meta-analytic investigation. The Journal of Positive Psychology 10, 5 ( 2015 ), 425-446. https://doi.org/10.1080/17439760. 2014.967799
[31]
Tracy Fullerton. 2008. Game Design Workshop: A Playcentric Approach to Creating Innovative Games. Morgan Kaufman, Amsterdam et al.
[32]
Fantasy Flight Games. 2011. Hey, That's My Fish! https://www.fantasy ightgames.com/en/products/hey-thats-mysh-digital-device-application/
[33]
Jacqueline Gottlieb, Pierre Yves Oudeyer, Manuel Lopes, and Adrien Baranes. 2013. Information-seeking, curiosity, and attention: Computational and neural mechanisms. Trends in Cognitive Sciences 17, 11 ( 2013 ), 585-593. https: //doi.org/10.1016/j.tics. 2013. 09.001
[34]
Thore Graepel and Ralf Herbrich. 2006. Ranking and matchmaking. Game Developer Magazine 25 ( 2006 ), 34.
[35]
Ralf Herbrich, Tom Minka, and Thore Graepel. 2006. TrueSkill™ : A Bayesian Skill Rating System. In Advances in Neural Information Processing Systems, Vol. 19. MIT Press.
[36]
Susan A Jackson and Robert C Eklund. 2002. Assessing ow in physical activity: the ow state scale-2 and dispositional ow scale-2. Journal of sport & exercise psychology 24, 2 ( 2002 ).
[37]
Susan A Jackson, Andrew J Martin, and Robert C Eklund. 2008. Long and short measures of ow: The construct validity of the FSS-2, DFS-2, and new brief counterparts. Journal of Sport and Exercise Psychology 30, 5 ( 2008 ), 561-587.
[38]
Kyros Jalife, Casper Harteveld, and Christo er Holmgård. 2021. From Flow to Fuse: A Cognitive Perspective. Proceedings of the ACM on Human-Computer Interaction 5, CHI PLAY ( 2021 ), 1-30.
[39]
Daniel Johnson, M John Gardner, and Ryan Perry. 2018. Validation of two game experience scales: the player experience of need satisfaction (PENS) and game experience questionnaire (GEQ). International Journal of Human-Computer Studies 118 ( 2018 ), 38-46.
[40]
Robert M. Kaplan and Veronica L. Irvin. 2015. Likelihood of Null E ects of Large NHLBI Clinical Trials Has Increased over Time. PLOS ONE 10, 8 ( 2015 ), e0132382. https://doi.org/10.1371/journal.pone.0132382
[41]
Celeste Kidd and Benjamin Y. Hayden. 2015. The Psychology and Neuroscience of Curiosity. Neuron 88, 3 ( 2015 ), 449-460.
[42]
Madison Klarkowski, Daniel Johnson, Peta Wyeth, Mitchell McEwan, Cody Phillips, and Simon Smith. 2016. Operationalising and evaluating sub-optimal and optimal play experiences through challenge-skill manipulation. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 5583-5594.
[43]
Christoph Klimmt. 2006. Computerspielen als Handlung: Dimensionen und Determinanten des Erlebens interaktiver Unterhaltungsangebote. Vol. 2. Herbert von Halem Verlag.
[44]
Bart P Knijnenburg, Martijn C Willemsen, Zeno Gantner, Hakan Soncu, and Chris Newell. 2012. Explaining the user experience of recommender systems. User modeling and user-adapted interaction 22, 4 ( 2012 ), 441-504.
[45]
Shringi Kumari, Sebastian Deterding, and Jonathan Freeman. 2019. The Role of Uncertainty in Moment-to-Moment Player Motivation : A Grounded Theory. In CHI PLAY '19 Proceedings of the Annual Symposium on Computer-Human Interaction in Play. ACM Press, New York. https://doi.org/10.1145/3311350.3347148
[46]
Daniel Lakens. 2019. The value of preregistration for psychological science: A conceptual analysis. 62, 3 ( 2019 ), 221-230.
[47]
Derek Lomas, Kishan Patel, Jodi L Forlizzi, and Kenneth R Koedinger. 2013. Optimizing challenge in an educational game using large-scale design experiments. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 89-98.
[48]
J Derek Lomas, Kenneth Koedinger, Nirmal Patel, Sharan Shodhan, Nikhil Poonwala, and Jodi L Forlizzi. 2017. Is di culty overrated?: The e ects of choice, novelty and suspense on intrinsic motivation in educational games. In Proceedings of the 2017 CHI conference on human factors in computing systems. ACM, 1028-1039.
[49]
Colleen Macklin and John Sharp. 2016. Games, Design and Play: A Detailed Approach to Iterative Game Design. Addison-Wesley, Boston, MA.
[50]
Chris McEntee. 2012. Rational Design: The Core of Rayman Origins. http://www.gamasutra.com/view/feature/167214/ rational_design_the_core_of_.php
[51]
Elisa D Mekler, Julia Ayumi Bopp, Alexandre N Tuch, and Klaus Opwis. 2014. A systematic review of quantitative studies on the enjoyment of digital entertainment games. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 927-936.
[52]
Josh Menke. 2016. Skill, Matchmaking, and Ranking Systems Design. https://www.gdcvault.com/play/1023014/SkillMatchmaking-and-Ranking-Systems
[53]
Talya Miron-Shatz, Arthur Stone, and Daniel Kahneman. 2009. Memories of yesterday's emotions: Does the valence of experience a ect the memory-experience gap? Emotion 9, 6 ( 2009 ), 885.
[54]
Marcus R. Munafò, Brian A. Nosek, Dorothy V.M. Bishop, Katherine S. Button, Christopher D. Chambers, Nathalie Percie Du Sert, Uri Simonsohn, Eric Jan Wagenmakers, Jennifer J. Ware, and John P.A. Ioannidis. 2017. A manifesto for reproducible science. Nature Human Behaviour 1, 1 ( 2017 ), 1-9. https://doi.org/10.1038/s41562-016-0021
[55]
Fiona Fui-Hoon Nah. 2004. A study on tolerable waiting time: how long are web users willing to wait? Behaviour & Information Technology 23, 3 ( 2004 ), 153-163.
[56]
Nakamura, Jeanne and Csikszentmihalyi, Mihaly. 2002. The Concept of Flow. In Handbook of positive psychology. Oxford University Press, Oxford, 89-105.
[57]
Leif D Nelson, Joseph Simmons, and Uri Simonsohn. 2018. Psychology's renaissance. Annual review of psychology 69 ( 2018 ), 511-534.
[58]
Klaus Oberauer and Stephan Lewandowsky. 2019. Addressing the theory crisis in psychology. Psychonomic Bulletin & Review 26, 5 ( 2019 ), 1596-1618. https://doi.org/10.3758/s13423-019-01645-2
[59]
Amy Orben and Daniël Lakens. 2020. Crud (Re)De ned. Advances in Methods and Practices in Psychological Science 3, 2 ( 2020 ), 238-247. https://doi.org/10.1177/2515245920917961
[60]
Serge Petralito, Florian Brühlmann, Glena Iten, Elisa D Mekler, and Klaus Opwis. 2017. A good reason to die: how avatar death and high challenges enable positive experiences. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 5087-5097.
[61]
Travis Proulx and Richard D Morey. 2021. Beyond statistical ritual: Theory in psychological science. Perspectives on Psychological Science 16, 4 ( 2021 ), 671-681.
[62]
C. Scott Rigby and Richard M Ryan. 2007. The Player Experience of Need Satisfaction (PENS). Technical Report. Immersyve.
[63]
Scott Rigby and Richard M. Ryan. 2011. Glued to Games: How Video Games Draw Us in and Hold Us Spellbound. Praeger, Santa Barbara, Denver, Oxford.
[64]
Yvonne Rogers. 2012. HCI Theory: Classical, Modern, and Contemporary. Morgan & Claypool.
[65]
Miia Ronimus, Janne Kujala, Asko Tolvanen, and Heikki Lyytinen. 2014. Children's engagement during digital game-based learning of reading: The e ects of time, rewards, and challenge. Computers & Education 71 ( 2014 ), 237-246.
[66]
Richard M Ryan. 1982. Control and information in the intrapersonal sphere: An extension of cognitive evaluation theory. Journal of personality and social psychology 43, 3 ( 1982 ), 450.
[67]
Richard M. Ryan and Edward L. Deci. 2017. Self-Determination Theory: Basic Psychological Needs in Motivation, Development, and Wellness. Guilford Press, New York.
[68]
Richard M Ryan, C Scott Rigby, and Andrew Przybylski. 2006. The motivational pull of video games: A self-determination theory approach. Motivation and emotion 30, 4 ( 2006 ), 344-360.
[69]
Jesse Schell. 2009. The Art of Game Design: A Book of Lenses. Morgan Kaufman, Amsterdam et al.
[70]
Ian Schreiber and Brenda Romero. 2021. Game Balance. CRC Press, Boca Raton, London, New York.
[71]
J. L. Sherry. 2004. Flow and Media Enjoyment. Communication Theory 14, 4 (Nov. 2004 ), 328-347. https://doi.org/10. 1093/ct/14.4. 328
[72]
Patrick E Shrout and Joseph L Rodgers. 2018. Psychology, science, and knowledge construction: Broadening perspectives from the replication crisis. Annual review of psychology 69 ( 2018 ), 487-510.
[73]
David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al. 2016. Mastering the game of Go with deep neural networks and tree search. Nature 529, 7587 ( 2016 ), 484-489.
[74]
Joseph P. Simmons, Leif D. Nelson, and Uri Simonsohn. 2011. False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Signi cant. Psychological Science 22, 11 ( 2011 ), 1359-1366. https://doi.org/10.1177/0956797611417632
[75]
Nathan Sorenson, Philippe Pasquier, and Steve DiPaola. 2011. A generic approach to challenge modeling for the procedural creation of video game levels. IEEE Transactions on Computational Intelligence and AI in Games 3, 3 ( 2011 ), 229-244.
[76]
Alistair Sutcli e. 2009. Designing for user engagement: Aesthetic and attractive user interfaces. Synthesis lectures on human-centered informatics 2, 1 ( 2009 ), 1-55.
[77]
Penelope Sweetser, Daniel Johnson, Peta Wyeth, Aiman Anwar, Yan Meng, and Anne Ozdowska. 2017. GameFlow in Di erent Game Genres and Platforms. Computers in Entertainment 15, 3 (April 2017 ), 1-24. https://doi.org/10.1145/ 3034780
[78]
Penelope Sweetser and Peta Wyeth. 2005. GameFlow: a model for evaluating player enjoyment in games. Computers in Entertainment (CIE) 3, 3 ( 2005 ), 3-3.
[79]
April Tyack and Elisa D Mekler. 2020. Self-Determination Theory in HCI Games Research: Current Uses and Open Questions. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1-22.
[80]
Robert J Vallerand and Catherine F Ratelle. 2002. Intrinsic and extrinsic motivation: A hierarchical model. Handbook of self-determination research 128 ( 2002 ), 37-63.
[81]
Iris van Rooij and Giosuè Baggio. 2021. Theory before the test: How to build high-verisimilitude explanatory theories in psychological science. Perspectives on Psychological Science 16, 4 ( 2021 ), 682-697.
[82]
Max L Wilson, Ed H Chi, Stuart Reeves, and David Coyle. 2014. RepliCHI: the workshop II. In CHI'14 Extended Abstracts on Human Factors in Computing Systems. 33-36.
[83]
Max L Wilson, Wendy Mackay, Ed Chi, Michael Bernstein, Dan Russell, and Harold Thimbleby. 2011. RepliCHI-CHI should be replicating and validating results more: discuss. In CHI'11 Extended Abstracts on Human Factors in Computing Systems. 463-466.
[84]
Georgios N Yannakakis and Julian Togelius. 2011. Experience-driven procedural content generation. IEEE Transactions on A ective Computing 2, 3 ( 2011 ), 147-161.
[85]
Mohammad Zohaib. 2018. Dynamic di culty adjustment (DDA) in computer games: A review. Advances in HumanComputer Interaction 2018 ( 2018 ).
[86]
Robert Zubek. 2020. Elements of Game Design. MIT Press.

Cited By

View all
  • (2024)Disengagement From Games: Characterizing the Experience and Process of Exiting Play SessionsProceedings of the ACM on Human-Computer Interaction10.1145/36770668:CHI PLAY(1-27)Online publication date: 15-Oct-2024
  • (2024)The Basic Needs in Games Scale (BANGS)International Journal of Human-Computer Studies10.1016/j.ijhcs.2024.103289188:COnline publication date: 1-Aug-2024
  • (2024)Alternate realities in interactive digital narratives – understanding and improving design and prosocial effects through empirical methodsMultimedia Tools and Applications10.1007/s11042-024-18884-883:15(46757-46778)Online publication date: 3-Apr-2024
  • Show More Cited By

Index Terms

  1. Objective Difficulty-Skill Balance Impacts Perceived Balance but Not Behaviour: A Test of Flow and Self-Determination Theory Predictions

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image Proceedings of the ACM on Human-Computer Interaction
    Proceedings of the ACM on Human-Computer Interaction  Volume 7, Issue CHI PLAY
    November 2023
    1360 pages
    EISSN:2573-0142
    DOI:10.1145/3554313
    Issue’s Table of Contents
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 04 October 2023
    Published in PACMHCI Volume 7, Issue CHI PLAY

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Balance
    2. Challenge
    3. Competence
    4. Difficulty
    5. Engagement
    6. Flow
    7. Self-Determination Theory
    8. Skill

    Qualifiers

    • Research-article

    Funding Sources

    • EPSRC

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)178
    • Downloads (Last 6 weeks)13
    Reflects downloads up to 18 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Disengagement From Games: Characterizing the Experience and Process of Exiting Play SessionsProceedings of the ACM on Human-Computer Interaction10.1145/36770668:CHI PLAY(1-27)Online publication date: 15-Oct-2024
    • (2024)The Basic Needs in Games Scale (BANGS)International Journal of Human-Computer Studies10.1016/j.ijhcs.2024.103289188:COnline publication date: 1-Aug-2024
    • (2024)Alternate realities in interactive digital narratives – understanding and improving design and prosocial effects through empirical methodsMultimedia Tools and Applications10.1007/s11042-024-18884-883:15(46757-46778)Online publication date: 3-Apr-2024
    • (2024)What if We Educated Students Assuming They can Think? Introducing the Critical Education Framework (CEF) for Interactive Narratives, Games and Related FieldsInteractive Storytelling10.1007/978-3-031-78450-7_15(214-223)Online publication date: 2-Dec-2024

    View Options

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media