skip to main content
research-article

To Be or Not to Be Stuck, or Is It a Continuum?: A Systematic Literature Review on the Concept of Being Stuck in Games

Published: 06 October 2021 Publication History

Abstract

Players can get stuck in video games, which impedes their process to their goal and results in unfavorable outcomes like negative emotions, impediments of flow, and obstacles for learning. Currently, it is not easily possible to assess if a player is stuck, as no widely accepted definition of "being stuck" in games exists. We conducted 13 expert interviews and a systematic literature review with 104 relevant papers selected from 4022 candidates. We present a definition of "being stuck" that conceptualizes the state as a continuum and contextualize it within related concepts. Our stuck continuum can be applied to regulate the player's stuck level. We propose a taxonomy of measures that are useful for the detection of the level of stuckness and discuss the effectiveness of countermeasures. Our stuck concept is crucial for game developers creating an optimal player experience in games.

Supplemental Material

ZIP File
Our supplementary material consists of 2 files: 1) synthesis_being_stuck.xlsx This is the final spreadsheet with all raw data created during full-text screening and used for the synthesis.

References

[1]
? Amir Zaib Abbasi, Ding Hooi Ting, Helmut Hlavacs, Liliana Vale Costa, and Ana Isabel Veloso. 2019. An empirical validation of consumer video game engagement: A playful-consumption experience approach. Entertainment Computing 29 (2019), 43 -- 55. https://doi.org/10.1016/j.entcom.2018.12.002
[2]
Ernest Adams. 2014. Fundamentals of Game Design. Pearson Education. Proc. ACM Hum.-Comput. Interact., Vol. 5, No. CHI PLAY, Article 229. Publication date: September 2021. 229:26 Tobias Drey et al.
[3]
Amy L Alexander, Tad Brunyé, Jason Sidman, and Shawn A Weil. 2005. From gaming to training: A review of studies on fidelity, immersion, presence, and buy-in and their effects on transfer in pc-based simulations and games. DARWARS Training Impact Group 5 (2005), 1--14.
[4]
? T. Alves, S. Gama, and F. S. Melo. 2018. Flow adaptation in serious games for health. In 2018 IEEE 6th International Conference on Serious Games and Applications for Health (SeGAH). 1--8. https://doi.org/10.1109/SeGAH.2018.8401382
[5]
Erik Andersen. 2012. Optimizing Adaptivity in Educational Games. In Proceedings of the International Conference on the Foundations of Digital Games (Raleigh, North Carolina) (FDG '12). Association for Computing Machinery, New York, NY, USA, 279--281. https://doi.org/10.1145/2282338.2282398
[6]
" Craig G. Anderson, Kathryn Campbell, and Constance Steinkuehler. 2019. Building Persistence through Failure: The Role of Challenge in Video Games. In Proceedings of the 14th International Conference on the Foundations of Digital Games (San Luis Obispo, California, USA) (FDG '19). Association for Computing Machinery, New York, NY, USA, Article 34, 6 pages. https://doi.org/10.1145/3337722.3337741
[7]
" Craig G. Anderson, Jen Dalsen, Vishesh Kumar, Matthew Berland, and Constance Steinkuehler. 2018. Failing up: How failure in a game environment promotes learning through discourse. Thinking Skills and Creativity 30 (2018), 135--144. https://doi.org/10.1016/j.tsc.2018.03.002 The Role of Failure in Promoting Thinking Skills and Creativity.
[8]
" K. C. Apostolakis, K. Stefanidis, A. Psaltis, K. Dimitropoulos, and P. Daras. 2017. Offline and online adaptation in prosocial games. In 2017 9th International Conference on Virtual Worlds and Games for Serious Applications (VS-Games). 201--208. https://doi.org/10.1109/VS-GAMES.2017.8056602
[9]
Association for Computing Machinery. 2020. ACM Digital Library. https://dl.acm.org/ (Accessed on 08/23/2020).
[10]
" Youngkyun Baek and Achraf Touati. 2017. Exploring how individual traits influence enjoyment in a mobile learning game. Computers in Human Behavior 69 (2017), 347--357. https://doi.org/10.1016/j.chb.2016.12.053
[11]
Javier A. Bargas-Avila and Kasper Hornbæk. 2011. Old Wine in New Bottles or Novel Challenges: A Critical Analysis of Empirical Studies of User Experience. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Vancouver, BC, Canada) (CHI '11). Association for Computing Machinery, New York, NY, USA, 2689--2698. https://doi.org/10.1145/1978942.1979336
[12]
" Glen Berseth, M. Brandon Haworth, Mubbasir Kapadia, and Petros Faloutsos. 2014. Characterizing and Optimizing Game Level Difficulty. In Proceedings of the Seventh International Conference on Motion in Games (Playa Vista, California) (MIG '14). Association for Computing Machinery, New York, NY, USA, 153--160. https://doi.org/10.1145/ 2668064.2668100
[13]
" N. Beume, H. Danielsiek, C. Eichhorn, B. Naujoks, M. Preuss, K. Stiller, and S. Wessing. 2008. Measuring flow as concept for detecting game fun in the Pac-Man game. In 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence). 3448--3455. https://doi.org/10.1109/CEC.2008.4631264
[14]
" Fernando Bevilacqua, Henrik Engström, and Per Backlund. 2018. Changes in heart rate and facial actions during a gaming session with provoked boredom and stress. Entertainment Computing 24 (2018), 10--20. https://doi.org/10. 1016/j.entcom.2017.10.004
[15]
" Y. Bian, C. Yang, F. Gao, H. Li, X. Sun, X. Meng, and Y. Wang. 2015. A Physiological Evaluation Model for Flow- Experience in VR Games: Construction and Preliminary Test. In 2015 International Conference on Identification, Information, and Knowledge in the Internet of Things (IIKI). 244--249. https://doi.org/10.1109/IIKI.2015.59
[16]
" Francisco Bicho and Carlos Martinho. 2018. Multi-Dimensional Player Skill Progression Modelling for Procedural Content Generation. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3235765. 3235774
[17]
" G. Biswas, R. Rajendran, N. Mohammed, B. S. Goldberg, R. A. Sottilare, K. Brawner, and M. Hoffman. 2020. Multilevel Learner Modeling in Training Environments for Complex Decision Making. IEEE Transactions on Learning Technologies 13, 1 (2020), 172--185. https://doi.org/10.1109/TLT.2019.2923352
[18]
" Paula Bitrián, Isabel Buil, and Sara Catalán. 2020. Flow and business simulation games: A typology of students. The International Journal of Management Education 18, 1 (2020), 100365. https://doi.org/10.1016/j.ijme.2020.100365
[19]
" Paris Mavromoustakos Blom, Sander Bakkes, and Pieter Spronck. 2019. Modeling and adjusting in-game difficulty based on facial expression analysis. Entertainment Computing 31 (2019), 100307. https://doi.org/10.1016/j.entcom. 2019.100307
[20]
" Boyan Bontchev and Dessislava Vassileva. 2016. Assessing Engagement in an Emotionally-Adaptive Applied Game. In Proceedings of the Fourth International Conference on Technological Ecosystems for Enhancing Multiculturality (Salamanca, Spain) (TEEM '16). Association for Computing Machinery, New York, NY, USA, 747--754. https://doi.org/ 10.1145/3012430.3012602
[21]
Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative Research in Psychology 3, 2 (2006), 77--101. https://doi.org/10.1191/1478088706qp063oa arXiv:https://www.tandfonline.com/doi/pdf/10.1191/1478088706qp063oa Proc. ACM Hum.-Comput. Interact., Vol. 5, No. CHI PLAY, Article 229. Publication date: September 2021. To Be or Not to Be Stuck, or Is It a Continuum?: A Systematic Literature Review 229:27
[22]
Virginia Braun and Victoria Clarke. 2020. One size fits all? What counts as quality practice in (reflexive) thematic analysis? Qualitative Research in Psychology 0, 0 (2020), 1--25. https://doi.org/10.1080/14780887.2020.1769238 arXiv:https://doi.org/10.1080/14780887.2020.1769238
[23]
Pearl Brereton, Barbara A. Kitchenham, David Budgen, Mark Turner, and Mohamed Khalil. 2007. Lessons from applying the systematic literature review process within the software engineering domain. Journal of Systems and Software 80, 4 (2007), 571 -- 583. https://doi.org/10.1016/j.jss.2006.07.009 Software Performance.
[24]
? J. Brich, K. Rogers, J. Frommel, M. Weidhaas, A. Bruckner, S. Mirabile, T. Dorn, V. Riemer, C. Schrader, and M. Weber. 2015. LiverDefense: Using a Tower Defense Game as a Customisable Research Tool. In 2015 7th International Conference on Games and Virtual Worlds for Serious Applications (VS-Games). 1--8. https://doi.org/10.1109/VSGAMES. 2015.7295779
[25]
Jeanne H. Brockmyer, Christine M. Fox, Kathleen A. Curtiss, Evan McBroom, Kimberly M. Burkhart, and Jacquelyn N. Pidruzny. 2009. The development of the Game Engagement Questionnaire: A measure of engagement in video gameplaying. Journal of Experimental Social Psychology 45, 4 (2009), 624 -- 634. https://doi.org/10.1016/j.jesp.2009.02.016
[26]
? A. Burns and J. Tulip. 2017. Detecting flowin games using facial expressions. In 2017 IEEE Conference on Computational Intelligence and Games (CIG). 45--52. https://doi.org/10.1109/CIG.2017.8080414
[27]
? G. Chanel, C. Rebetez, M. Bétrancourt, and T. Pun. 2011. Emotion Assessment From Physiological Signals for Adaptation of Game Difficulty. IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans 41, 6 (2011), 1052--1063. https://doi.org/10.1109/TSMCA.2011.2116000
[28]
? Guillaume Chanel, Cyril Rebetez, Mireille Bétrancourt, and Thierry Pun. 2008. Boredom, Engagement and Anxiety as Indicators for Adaptation to Difficulty in Games. In Proceedings of the 12th International Conference on Entertainment and Media in the Ubiquitous Era (Tampere, Finland) (MindTrek '08). Association for Computing Machinery, New York, NY, USA, 13--17. https://doi.org/10.1145/1457199.1457203
[29]
? Li-Xian Chen and Chuen-Tsai Sun. 2016. Self-Regulation Influence on Game Play Flow State. Comput. Hum. Behav. 54, C (Jan. 2016), 341--350. https://doi.org/10.1016/j.chb.2015.08.020
[30]
Domenic V Cicchetti and Alvan R Feinstein. 1990. High agreement but low kappa: II. Resolving the paradoxes. Journal of clinical epidemiology 43, 6 (1990), 551--558.
[31]
Serdar Çiftci. 2018. Trends of serious games research from 2007 to 2017: A bibliometric analysis. Journal of Education and Training Studies 6, 2 (2018), 18--27.
[32]
? Douglas B. Clark, Satyugjit S. Virk, Jackie Barnes, and Deanne M. Adams. 2016. Self-explanation and digital games: Adaptively increasing abstraction. Computers & Education 103 (2016), 28--43. https://doi.org/10.1016/j.compedu.2016. 09.010
[33]
Oscar Clark. 2014. Games as a service: How free to play design can make better games. CRC Press.
[34]
Ruth Colvin Clark, Frank Nguyen, John Sweller, and Melissa Baddeley. 2006. Efficiency in learning: Evidence-based guidelines to manage cognitive load. Performance Improvement 45, 9 (2006), 46--47.
[35]
Jacob Cohen. 1960. A Coefficient of Agreement for Nominal Scales. Educational and Psychological Measurement 20, 1 (1960), 37--46. https://doi.org/10.1177/001316446002000104 arXiv:https://doi.org/10.1177/001316446002000104
[36]
Jacob Cohen. 1968. Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit. Psychological bulletin 70, 4 (1968), 213.
[37]
Corporation for Digital Scholarship. 2020. Zotero. https://www.zotero.org/ (Accessed on 08/23/2020).
[38]
Mihaly Csikszentmihalyi. 1975. Beyond boredom and anxiety: Experiencing flow in work and play. San Francisco/ Washington/London (1975).
[39]
Mihaly Csikszentmihalyi. 1990. Flow: The psychology of optimal experience. Vol. 1990. Harper & Row New York.
[40]
Mihaly Csikszentmihalyi. 1997. Finding flow: The psychology of engagement with everyday life. (1997).
[41]
? Simon Demediuk, Marco Tamassia, William L. Raffe, Fabio Zambetta, Florian "Floyd" Mueller, and Xiaodong Li. 2018. Measuring Player Skill Using Dynamic Difficulty Adjustment. In Proceedings of the Australasian Computer Science Week Multiconference (Brisband, Queensland, Australia) (ACSW '18). Association for Computing Machinery, New York, NY, USA, Article 41, 7 pages. https://doi.org/10.1145/3167918.3167939
[42]
Guillaume Denis and Pierre Jouvelot. 2005. Motivation-Driven Educational Game Design: Applying Best Practices to Music Education. In Proceedings of the 2005 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology (Valencia, Spain) (ACE '05). Association for Computing Machinery, New York, NY, USA, 462--465. https://doi.org/10.1145/1178477.1178581
[43]
Alena Denisova and Paul Cairns. 2015. Adaptation in Digital Games: The Effect of Challenge Adjustment on Player Performance and Experience. In Proceedings of the 2015 Annual Symposium on Computer-Human Interaction in Play (London, United Kingdom) (CHI PLAY '15). Association for Computing Machinery, New York, NY, USA, 97--101. https://doi.org/10.1145/2793107.2793141
[44]
Alena Denisova and Paul Cairns. 2019. Player experience and deceptive expectations of difficulty adaptation in digital games. Entertainment Computing 29 (2019), 56 -- 68. https://doi.org/10.1016/j.entcom.2018.12.001 Proc. ACM Hum.-Comput. Interact., Vol. 5, No. CHI PLAY, Article 229. Publication date: September 2021. 229:28 Tobias Drey et al.
[45]
Alena Denisova, Paul Cairns, Christian Guckelsberger, and David Zendle. 2020. Measuring perceived challenge in digital games: Development & validation of the challenge originating from recent gameplay interaction scale (CORGIS). International Journal of Human-Computer Studies 137 (2020), 102383. https://doi.org/10.1016/j.ijhcs.2019.102383
[46]
Alena Denisova, Christian Guckelsberger, and David Zendle. 2017. Challenge in Digital Games: Towards Developing a Measurement Tool. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI EA '17). Association for Computing Machinery, New York, NY, USA, 2511--2519. https://doi.org/10.1145/3027063.3053209
[47]
Tobias Drey, Jan Gugenheimer, Julian Karlbauer, Maximilian Milo, and Enrico Rukzio. 2020. VRSketchIn: Exploring the Design Space of Pen and Tablet Interaction for 3D Sketching in Virtual Reality. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '20). Association for Computing Machinery, New York, NY, USA, 1--14. https://doi.org/10.1145/3313831.3376628
[48]
Tobias Drey, Pascal Jansen, Fabian Fischbach, Julian Frommel, and Enrico Rukzio. 2020. Towards Progress Assessment for Adaptive Hints in Educational Virtual Reality Games. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI EA '20). Association for Computing Machinery, New York, NY, USA, 1--9. https://doi.org/10.1145/3334480.3382789
[49]
? I. Dunwell, P. Petridis, M. Hendrix, S. Arnab, M. AL-Smadi, and C. Guetl. 2012. Guiding Intuitive Learning in Serious Games: An Achievement-Based Approach to Externalized Feedback and Assessment. In 2012 Sixth International Conference on Complex, Intelligent, and Software Intensive Systems. 911--916. https://doi.org/10.1109/CISIS.2012.205
[50]
Panteleimon Ekkekakis. 2012. Affect, mood, and emotion. Measurement in sport and exercise psychology 321 (2012).
[51]
? Lucca Eloy, Angela E.B. Stewart, Mary Jean Amon, Caroline Reinhardt, Amanda Michaels, Chen Sun, Valerie Shute, Nicholas D. Duran, and Sidney D'Mello. 2019. Modeling Team-Level Multimodal Dynamics during Multiparty Collaboration. In 2019 International Conference on Multimodal Interaction (Suzhou, China) (ICMI '19). Association for Computing Machinery, New York, NY, USA, 244--258. https://doi.org/10.1145/3340555.3353748
[52]
Elsevier. 2020. ScienceDirect.com | Science, health and medical journals, full text articles and books. https://www. sciencedirect.com/ (Accessed on 08/23/2020).
[53]
? Séverine Erhel and Eric Jamet. 2019. Improving instructions in educational computer games: Exploring the relations between goal specificity, flow experience and learning outcomes. Computers in Human Behavior 91 (2019), 106 -- 114. https://doi.org/10.1016/j.chb.2018.09.020
[54]
Alvan R Feinstein and Domenic V Cicchetti. 1990. High agreement but low kappa: I. The problems of two paradoxes. Journal of clinical epidemiology 43, 6 (1990), 543--549.
[55]
Joseph L Fleiss. 1971. Measuring nominal scale agreement among many raters. Psychological bulletin 76, 5 (1971), 378.
[56]
? S. Foutsitzi, S. Asteriadis, and G. Caridakis. 2019. An overview of Affective Models and ICT in Education. In 2019 10th International Conference on Information, Intelligence, Systems and Applications (IISA). 1--8. https://doi.org/10. 1109/IISA.2019.8900783
[57]
? Julian Frommel, Fabian Fischbach, Katja Rogers, and Michael Weber. 2018. Emotion-Based Dynamic Difficulty Adjustment Using Parameterized Difficulty and Self-Reports of Emotion. In Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in Play (Melbourne, VIC, Australia) (CHI PLAY '18). Association for Computing Machinery, New York, NY, USA, 163--171. https://doi.org/10.1145/3242671.3242682
[58]
? Julian Frommel, Claudia Schrader, and Michael Weber. 2018. Towards Emotion-Based Adaptive Games: Emotion Recognition Via Input and Performance Features. In Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in Play (Melbourne, VIC, Australia) (CHI PLAY '18). Association for Computing Machinery, New York, NY, USA, 173--185. https://doi.org/10.1145/3242671.3242672
[59]
Fong-Ling Fu, Rong-Chang Su, and Sheng-Chin Yu. 2009. EGameFlow: A scale to measure learners' enjoyment of e-learning games. Computers & Education 52, 1 (2009), 101 -- 112. https://doi.org/10.1016/j.compedu.2008.07.004
[60]
? Xin Fu, Xi Chen, Yu-Tong Shi, Indranil Bose, and Shun Cai. 2017. User segmentation for retention management in online social games. Decision Support Systems 101 (2017), 51--68. https://doi.org/10.1016/j.dss.2017.05.015
[61]
Clive J Fullagar, Patrick A Knight, and Heather S Sovern. 2013. Challenge/skill balance, flow, and performance anxiety. Applied Psychology 62, 2 (2013), 236--259.
[62]
Tracy Fullerton. 2014. Game design workshop: a playcentric approach to creating innovative games. CRC press.
[63]
? M. Gavriushenko, L. Karilainen, and M. Kankaanranta. 2015. Adaptive systems as enablers of feedback in English language learning game-based environments. In 2015 IEEE Frontiers in Education Conference (FIE). 1--8. https: //doi.org/10.1109/FIE.2015.7344107
[64]
Kiel M Gilleade and Alan Dix. 2004. Using Frustration in the Design of Adaptive Videogames. In Proceedings of the 2004 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology (Singapore) (ACE '04). Association for Computing Machinery, New York, NY, USA, 228--232. https://doi.org/10.1145/1067343.1067372
[65]
? Benjamin Goldberg and Janis Cannon-Bowers. 2015. Feedback source modality effects on training outcomes in a serious game: Pedagogical agents make a difference. Computers in Human Behavior 52 (2015), 1--11. https: Proc. ACM Hum.-Comput. Interact., Vol. 5, No. CHI PLAY, Article 229. Publication date: September 2021. To Be or Not to Be Stuck, or Is It a Continuum?: A Systematic Literature Review 229:29 //doi.org/10.1016/j.chb.2015.05.008
[66]
Fateme Hajiarbabi, Hassan Ahadi, Ali Delavar, and Hasan Asadzadeh. 2012. Computer Games and Its Effect on Attribution of Learned Helplessness Students. Life Science Journal 9, 3 (2012).
[67]
" Sandro Hardy, Angelika Kern, Tim Dutz, Christoph Weber, Stefan Göbel, and Ralf Steinmetz. 2014. What Makes Games Challenging" Considerations on How to Determine the "Challenge" Posed by an Exergame for Balance Training. In Proceedings of the 2014 ACM International Workshop on Serious Games (Orlando, Florida, USA) (SeriousGames '14). Association for Computing Machinery, New York, NY, USA, 57--62. https://doi.org/10.1145/2656719.2656728
[68]
" László Harmat, Örjan de Manzano, Töres Theorell, Lennart Högman, Håkan Fischer, and Fredrik Ullén. 2015. Physiological correlates of the flow experience during computer game playing. International Journal of Psychophysiology 97, 1 (2015), 1--7. https://doi.org/10.1016/j.ijpsycho.2015.05.001
[69]
" B. Harrison and D. L. Roberts. 2015. An Analytic and Psychometric Evaluation of Dynamic Game Adaption for Increasing Session-Level Retention in Casual Games. IEEE Transactions on Computational Intelligence and AI in Games 7, 3 (2015), 207--219. https://doi.org/10.1109/TCIAIG.2015.2410757
[70]
" Drew Hicks, Michael Eagle, Elizabeth Rowe, Jodi Asbell-Clarke, Teon Edwards, and Tiffany Barnes. 2016. Using Game Analytics to Evaluate Puzzle Design and Level Progression in a Serious Game. In Proceedings of the Sixth International Conference on Learning Analytics and Knowledge (Edinburgh, United Kingdom) (LAK '16). Association for Computing Machinery, New York, NY, USA, 440--448. https://doi.org/10.1145/2883851.2883953
[71]
Teresa Hirzle, Jan Gugenheimer, Florian Geiselhart, Andreas Bulling, and Enrico Rukzio. 2019. A Design Space for Gaze Interaction on Head-Mounted Displays. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI '19). Association for Computing Machinery, New York, NY, USA, 1--12. https://doi.org/10.1145/3290605.3300855
[72]
" Jon-Chao Hong, Kai-Hsin Tai, Ming-Yueh Hwang, and Yen-Chun Kuo. 2016. Internet cognitive failure affects learning progress as mediated by cognitive anxiety and flow while playing a Chinese antonym synonym game with interacting verbal--analytical and motor-control. Computers & Education 100 (2016), 32--44. https://doi.org/10.1016/j. compedu.2016.04.009
[73]
" Huei-Tse Hou and Ming-Chaun Li. 2014. Evaluating multiple aspects of a digital educational problem-solving-based adventure game. Computers in Human Behavior 30 (2014), 29--38. https://doi.org/10.1016/j.chb.2013.07.052
[74]
" X. Hou, Y. Liu, O. Sourina, andW. Mueller-Wittig. 2015. CogniMeter: EEG-based Emotion, MentalWorkload and Stress Visual Monitoring. In 2015 International Conference on Cyberworlds (CW). 153--160. https://doi.org/10.1109/CW.2015.58
[75]
" Ting-Chia Hsu. 2017. Learning English with Augmented Reality: Do learning styles matter" Computers & Education 106 (2017), 137--149. https://doi.org/10.1016/j.compedu.2016.12.007
[76]
" C. Hung, F. Kuo, J. Chih-Yuan Sun, and P. Yu. 2014. An Interactive Game Approach for Improving Students' Learning Performance in Multi-Touch Game-Based Learning. IEEE Transactions on Learning Technologies 7, 1 (Jan 2014), 31--37. https://doi.org/10.1109/TLT.2013.2294806
[77]
" Ming-Yueh Hwang, Jon-Chao Hong, Jian-Hong Ye, Yu-Feng Wu, Kai-Hsin Tai, and Ming-Chi Kiu. 2019. Practicing abductive reasoning: The correlations between cognitive factors and learning effects. Computers & Education 138 (2019), 33--45. https://doi.org/10.1016/j.compedu.2019.04.014
[78]
IEEE. 2020. IEEE Xplore. https://ieeexplore.ieee.org/Xplore/home.jsp (Accessed on 08/23/2020).
[79]
Insilica LLC. 2020. Sysrev. https://sysrev.com/ (Accessed on 08/23/2020).
[80]
" A. Isaksen, D. Gopstein, J. Togelius, and A. Nealen. 2018. Exploring Game Space of Minimal Action Games via Parameter Tuning and Survival Analysis. IEEE Transactions on Games 10, 2 (2018), 182--194. https://doi.org/10.1109/ TCIAIG.2017.2750181
[81]
" A. Johnson, Y. Tang, and C. Franzwa. 2014. kNN-based adaptive virtual reality game system. In Proceedings of the 11th IEEE International Conference on Networking, Sensing and Control. 457--462. https://doi.org/10.1109/ICNSC.2014. 6819669
[82]
Jesper Juul. 2009. Fear of Failing" The Many Meanings of Difficulty in Video Games. The Video Game Theory Reader 2, 237--252 (2009).
[83]
Jesper Juul. 2013. The Art of Failure: An Essay on the Pain of Playing Video Games. MIT Press.
[84]
" Jina Kang, Min Liu, and Wen Qu. 2017. Using gameplay data to examine learning behavior patterns in a serious game. Computers in Human Behavior 72 (2017), 757--770.
[85]
" A. Katmada, M. Chatzakis, H. Apostolidis, A. Mavridis, and S. Panagiotis. 2015. An adaptive serious neuro-game using a mobile version of a bio-feedback device. In 2015 International Conference on Interactive Mobile Communication Technologies and Learning (IMCL). 416--420. https://doi.org/10.1109/IMCTL.2015.7359633
[86]
" A. Katmada, A. Mavridis, H. Apostolidis, and T. Tsiatsos. 2015. Developing an adaptive serious game based on students' bio-feedback. In 2015 6th International Conference on Information, Intelligence, Systems and Applications (IISA). 1--6. https://doi.org/10.1109/IISA.2015.7387975 Proc. ACM Hum.-Comput. Interact., Vol. 5, No. CHI PLAY, Article 229. Publication date: September 2021. 229:30 Tobias Drey et al.
[87]
Daniel Kepplinger, Günter Wallner, Simone Kriglstein, and Michael Lankes. 2020. See, Feel, Move: Player Behaviour Analysis through Combined Visualization of Gaze, Emotions, and Movement. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '20). Association for Computing Machinery, New York, NY, USA, 1--14. https://doi.org/10.1145/3313831.3376401
[88]
" Mohammad M. Khajah, Brett D. Roads, Robert V. Lindsey, Yun-En Liu, and Michael C. Mozer. 2016. Designing Engaging Games Using Bayesian Optimization. Association for Computing Machinery, New York, NY, USA, 5571--5582. https://doi.org/10.1145/2858036.2858253
[89]
" M. D. Kickmeier-Rust, C. M. Steiner, and D. Albert. 2009. Non-invasive Assessment and Adaptive Interventions in Learning Games. In 2009 International Conference on Intelligent Networking and Collaborative Systems. 301--305. https://doi.org/10.1109/INCOS.2009.30
[90]
Kristian Kiili. 2005. Digital game-based learning: Towards an experiential gaming model. The Internet and Higher Education 8, 1 (2005), 13 -- 24. https://doi.org/10.1016/j.iheduc.2004.12.001
[91]
Kristian Kiili, Sara de Freitas, Sylvester Arnab, and Timo Lainema. 2012. The Design Principles for Flow Experience in Educational Games. Procedia Computer Science 15 (2012), 78 -- 91. https://doi.org/10.1016/j.procs.2012.10.060 4th International Conference on Games and Virtual Worlds for Serious Applications(VS-GAMES'12).
[92]
" Kristian Kiili, Timo Lainema, Sara de Freitas, and Sylvester Arnab. 2014. Flow framework for analyzing the quality of educational games. Entertainment Computing 5, 4 (2014), 367 -- 377. https://doi.org/10.1016/j.entcom.2014.08.002
[93]
Seungwook Kim, Daeyoung Choi, Eunjung Lee, and Wonjong Rhee. 2017. Churn prediction of mobile and online casual games using play log data. PloS one 12, 7 (2017).
[94]
Barbara Kitchenham and Pearl Brereton. 2013. A systematic review of systematic review process research in software engineering. Information and Software Technology 55, 12 (2013), 2049 -- 2075. https://doi.org/10.1016/j.infsof.2013.07. 010
[95]
Madison Klarkowski, Daniel Johnson, Peta Wyeth, Mitchell McEwan, Cody Phillips, and Simon Smith. 2016. Operationalising and Evaluating Sub-Optimal and Optimal Play Experiences through Challenge-Skill Manipulation. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI '16). Association for Computing Machinery, New York, NY, USA, 5583--5594. https://doi.org/10.1145/2858036.2858563
[96]
" Julia Kneer, Malte Elson, and Florian Knapp. 2016. Fight fire with rainbows: The effects of displayed violence, difficulty, and performance in digital games on affect, aggression, and physiological arousal. Computers in Human Behavior 54 (2016), 142--148. https://doi.org/10.1016/j.chb.2015.07.034
[97]
" M. Köles, L. Szegletes, and B. Forstner. 2015. Towards a physiology based difficulty control system for serious games. In 2015 6th IEEE International Conference on Cognitive Infocommunications (CogInfoCom). 323--328. https: //doi.org/10.1109/CogInfoCom.2015.7390612
[98]
Effie Lai-Chong Law and Xu Sun. 2012. Evaluating user experience of adaptive digital educational games with Activity Theory. International Journal of Human-Computer Studies 70, 7 (2012), 478 -- 497. https://doi.org/10.1016/j.ijhcs.2012. 01.007 Special Issue on User Experience (UX) in Virtual Learning Environments.
[99]
" Michael J. Lee, Andrew J. Ko, and Irwin Kwan. 2013. In-Game Assessments Increase Novice Programmers' Engagement and Level Completion Speed. In Proceedings of the Ninth Annual International ACM Conference on International Computing Education Research (San Diego, San California, USA) (ICER '13). Association for Computing Machinery, New York, NY, USA, 153--160. https://doi.org/10.1145/2493394.2493410
[100]
Serena Lee-Cultura, Kshitij Sharma, Sofia Papavlasopoulou, Symeon Retalis, and Michail Giannakos. 2020. Using Sensing Technologies to Explain Children's Self-Representation in Motion-Based Educational Games. In Proceedings of the Interaction Design and Children Conference (London, United Kingdom) (IDC '20). Association for Computing Machinery, New York, NY, USA, 541--555. https://doi.org/10.1145/3392063.3394419
[101]
Hua Li, Hector Munoz-Avila, Lei Ke, Carl Symborski, and Rafael Alonso. 2013. Discovery of player strategies in a serious game. In First AAAI Conference on Human Computation and Crowdsourcing. Citeseer.
[102]
" Y. Li and A. S. Elmaghraby. 2014. A framework for using games for behavioral analysis of autistic children. In 2014 Computer Games: AI, Animation, Mobile, Multimedia, Educational and Serious Games (CGAMES). 1--4. https: //doi.org/10.1109/CGames.2014.6934157
[103]
" Chen-Chung Liu, Yuan-Bang Cheng, and Chia-Wen Huang. 2011. The effect of simulation games on the learning of computational problem solving. Computers & Education 57, 3 (2011), 1907--1918. https://doi.org/10.1016/j.compedu. 2011.04.002
[104]
Pramila Rani Nilanjan Sarkar Changchun Liu. 2005. Maintaining optimal challenge in computer games through real-time physiological feedback. Foundations of Augmented Cognition (2005), 184.
[105]
" Christian E. Lopez and Conrad S. Tucker. 2019. The effects of player type on performance: A gamification case study. Computers in Human Behavior 91 (2019), 333--345. https://doi.org/10.1016/j.chb.2018.10.005
[106]
" L. B. Martins Kummer, J. Cesar Nievola, and E. C. Paraiso. 2018. Applying Commitment to Churn and Remaining Players Lifetime Prediction. In 2018 IEEE Conference on Computational Intelligence and Games (CIG). 1--8. https: Proc. ACM Hum.-Comput. Interact., Vol. 5, No. CHI PLAY, Article 229. Publication date: September 2021. To Be or Not to Be Stuck, or Is It a Continuum?: A Systematic Literature Review 229:31 //doi.org/10.1109/CIG.2018.8490443
[107]
E. Masanobu, H. D. F. B., and K. Mikami. 2017. Dynamic Pressure Cycle Control: Dynamic Diffculty Adjustment beyond the Flow Zone. In 2017 Nicograph International (NicoInt). 9--14. https://doi.org/10.1109/NICOInt.2017.12
[108]
Deborah J. Mayhew. 1999. The Usability Engineering Lifecycle. In CHI '99 Extended Abstracts on Human Factors in Computing Systems (Pittsburgh, Pennsylvania) (CHI EA '99). Association for Computing Machinery, New York, NY, USA, 147--148. https://doi.org/10.1145/632716.632805
[109]
Elisa D. Mekler, Julia Ayumi Bopp, Alexandre N. Tuch, and Klaus Opwis. 2014. A Systematic Review of Quantitative Studies on the Enjoyment of Digital Entertainment Games. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Toronto, Ontario, Canada) (CHI '14). Association for Computing Machinery, New York, NY, USA, 927--936. https://doi.org/10.1145/2556288.2557078
[110]
Matthew K. Miller and Regan L. Mandryk. 2016. Differentiating In-Game Frustration from at-Game Frustration Using Touch Pressure. In Proceedings of the 2016 ACM International Conference on Interactive Surfaces and Spaces (Niagara Falls, Ontario, Canada) (ISS '16). Association for Computing Machinery, New York, NY, USA, 225--234. https://doi.org/10.1145/2992154.2992185
[111]
David Moher, Alessandro Liberati, Jennifer Tetzlaff, Douglas G Altman, Prisma Group, et al. 2009. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS med 6, 7 (2009), e1000097.
[112]
David Moher, Larissa Shamseer, Mike Clarke, Davina Ghersi, Alessandro Liberati, Mark Petticrew, Paul Shekelle, Lesley A Stewart, et al. 2015. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Systematic reviews 4, 1 (2015), 1.
[113]
D. Monteiro, H. Liang, A. Abel, N. Bahaei, and R. d. C. Monteiro. 2018. Evaluating Engagement of Virtual Reality Games Based on First and Third Person Perspective Using EEG and Subjective Metrics. In 2018 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR). 53--60. https://doi.org/10.1109/AIVR.2018.00015
[114]
Belkacem Mostefai, Amar Balla, and Philippe Trigano. 2019. A generic and efficient emotion-driven approach toward personalized assessment and adaptation in serious games. Cognitive Systems Research 56 (2019), 82--106. https://doi.org/10.1016/j.cogsys.2019.03.006
[115]
J. E. Muñoz, S. Cao, and J. Boger. 2019. Kinematically Adaptive Exergames: Personalizing Exercise Therapy Through Closed-Loop Systems. In 2019 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR). 118--1187. https://doi.org/10.1109/AIVR46125.2019.00026
[116]
L. Müller, A. Bernin, A. Kamenz, S. Ghose, K. von Luck, C. Grecos, Q. Wang, and F. Vogt. 2017. Emotional journey for an emotion provoking cycling exergame. In 2017 IEEE 4th International Conference on Soft Computing Machine Intelligence (ISCMI). 104--108. https://doi.org/10.1109/ISCMI.2017.8279607
[117]
S. C. Müller and T. Fritz. 2015. Stuck and Frustrated or in Flow and Happy: Sensing Developers' Emotions and Progress. In 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, Vol. 1. 688--699.
[118]
Lennart E. Nacke, Anna Cox, Regan L. Mandryk, and Paul Cairns. 2016. SIGCHI Games: The Scope of Games and Play Research at CHI. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (San Jose, California, USA) (CHI EA '16). Association for Computing Machinery, New York, NY, USA, 1088--1091. https://doi.org/10.1145/2851581.2886438
[119]
A. Nagle, D. Novak, P.Wolf, and R. Riener. 2014. The effect of different difficulty adaptation strategies on enjoyment and performance in a serious game for memory training. In 2014 IEEE 3nd International Conference on Serious Games and Applications for Health (SeGAH). 1--8. https://doi.org/10.1109/SeGAH.2014.7067088
[120]
V. Naik and V. Kamat. 2015. Adaptive and Gamified Learning Environment (AGLE). In 2015 IEEE Seventh International Conference on Technology for Education (T4E). 7--14. https://doi.org/10.1109/T4E.2015.23
[121]
Steve Nebel, Maik Beege, Sascha Schneider, and Günter Daniel Rey. 2020. Competitive Agents and Adaptive Difficulty Within Educational Video Games. Frontiers in Education 5 (2020), 129. https://doi.org/10.3389/feduc.2020.00129
[122]
Loc Nguyen and Phung Do. 2008. Learner model in adaptive learning. World Academy of Science, Engineering and Technology 45, 70 (2008), 395--400.
[123]
Ogechi Nnadi, Ute Fischer, Michael Boyce, and Michael Nitsche. 2008. Effect of Dynamic Camera Control on Spatial Reasoning in 3D Spaces. In Proceedings of the 2008 ACM SIGGRAPH Symposium on Video Games (Los Angeles, California) (Sandbox '08). Association for Computing Machinery, New York, NY, USA, 157--162. https://doi.org/10. 1145/1401843.1401873
[124]
Thomas P Novak and Donna L Hoffman. 1997. Measuring the flow experience among web users. (1997).
[125]
B. Omella Mainieri, V. L. Azevedo, P. H. Cacique Braga, and N. Omar. 2018. Development and Assessment of an Adaptive Difficulty Arithmetic Game Based Learning Object. In 2018 XIII Latin American Conference on Learning Technologies (LACLO). 232--239. https://doi.org/10.1109/LACLO.2018.00051
[126]
Eleanor O'Rourke, Christy Ballweber, and Zoran Popovií. 2014. Hint Systems May Negatively Impact Performance in Educational Games. In Proceedings of the First ACM Conference on Learning @ Scale Conference (Atlanta, Georgia, USA) (L@S '14). Association for Computing Machinery, New York, NY, USA, 51--60. https://doi.org/10.1145/2556325.2566248 Proc. ACM Hum.-Comput. Interact., Vol. 5, No. CHI PLAY, Article 229. Publication date: September 2021. 229:32 Tobias Drey et al.
[127]
Karin A. Orvis, Daniel B. Horn, and James Belanich. 2008. The roles of task difficulty and prior videogame experience on performance and motivation in instructional videogames. Computers in Human Behavior 24, 5 (2008), 2415--2433. https://doi.org/10.1016/j.chb.2008.02.016 Including the Special Issue: Internet Empowerment.
[128]
Fred G. W. C. Paas and Jeroen J. G. Van Merriënboer. 1993. The Efficiency of Instructional Conditions: An Approach to Combine Mental Effort and Performance Measures. Human Factors 35, 4 (1993), 737--743. https://doi.org/10.1177/ 001872089303500412 arXiv:https://doi.org/10.1177/001872089303500412
[129]
Madhukar Pai, Michael McCulloch, Jennifer D Gorman, Nitika Pai, Wayne Enanoria, Gail Kennedy, Prathap Tharyan, and John M Colford Jr. 2004. Systematic reviews and meta-analyses: an illustrated, step-by-step guide. The National medical journal of India 17, 2 (2004), 86--95.
[130]
K. Park, T. Kihl, S. Park, M. Kim, and J. Chang. 2016. Narratives and sensor driven cognitive behavior training game platform. In 2016 IEEE 14th International Conference on Software Engineering Research, Management and Applications (SERA). 125--131. https://doi.org/10.1109/SERA.2016.7516137
[131]
Reinhard Pekrun and Lisa Linnenbrink-Garcia. 2012. Academic Emotions and Student Engagement. Springer US, Boston, MA, 259--282. https://doi.org/10.1007/978--1--4614--2018--7_12
[132]
H. A. Pereira, A. F. D. Souza, and C. S. D. Menezes. 2018. Obtaining evidence of learning in digital games through a deep learning neural network to classify facial expressions of the players. In 2018 IEEE Frontiers in Education Conference (FIE). 1--8. https://doi.org/10.1109/FIE.2018.8659216
[133]
M. Pirovano, R. Mainetti, G. Baud-Bovy, P. L. Lanzi, and N. A. Borghese. 2012. Self-adaptive games for rehabilitation at home. In 2012 IEEE Conference on Computational Intelligence and Games (CIG). 179--186. https://doi.org/10.1109/ CIG.2012.6374154
[134]
J.L Plass, B.D. Homer, S. Pawar, C. Brenner, and A.P. MacNamara. 2019. The effect of adaptive difficulty adjustment on the effectiveness of a game to develop executive function skills for learners of different ages. Cognitive Development 49 (2019), 56--67. https://doi.org/10.1016/j.cogdev.2018.11.006
[135]
Michela Ponticorvo, Angelo Rega, Andrea Di Ferdinando, Davide Marocco, and Orazio Miglino. 2017. Approaches to Embed Bio-inspired Computational Algorithms in Educational and Serious Games. In CAID@ IJCAI. 8--14.
[136]
Anne Marie Porter and Paula Goolkasian. 2019. Video Games and Stress: How Stress Appraisals and Game Content Affect Cardiovascular and Emotion Outcomes. Frontiers in Psychology 10 (2019), 967. https://doi.org/10.3389/fpsyg. 2019.00967
[137]
Christopher Power, Paul Cairns, Alena Denisova, Themis Papaioannou, and Ruth Gultom. 2019. Lost at the Edge of Uncertainty: Measuring Player Uncertainty in Digital Games. International Journal of Human--Computer Interaction 35, 12 (2019), 1033--1045. https://doi.org/10.1080/10447318.2018.1507161 arXiv:https://doi.org/10.1080/10447318.2018.1507161
[138]
Christopher Power, Alena Denisova, Themis Papaioannou, and Paul Cairns. 2017. Measuring Uncertainty in Games: Design and Preliminary Validation. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI EA '17). Association for Computing Machinery, New York, NY, USA, 2839--2845. https://doi.org/10.1145/3027063.3053215
[139]
A. Psaltis, K. C. Apostolakis, K. Dimitropoulos, and P. Daras. 2018. Multimodal Student Engagement Recognition in Prosocial Games. IEEE Transactions on Games 10, 3 (2018), 292--303. https://doi.org/10.1109/TCIAIG.2017.2743341
[140]
Raquel Robinson, Katelyn Wiley, Amir Rezaeivahdati, Madison Klarkowski, and Regan L. Mandryk. 2020. "Let's Get Physiological, Physiological!": A Systematic Review of Affective Gaming. Association for Computing Machinery, New York, NY, USA, 132--147. https://doi.org/10.1145/3410404.3414227
[141]
Miia Ronimus, Janne Kujala, Asko Tolvanen, and Heikki Lyytinen. 2014. Children's engagement during digital game-based learning of reading: The effects of time, rewards, and challenge. Computers & Education 71 (2014), 237--246. https://doi.org/10.1016/j.compedu.2013.10.008
[142]
Shaghayegh Roohi, Elisa D. Mekler, Mikke Tavast, Tatu Blomqvist, and Perttu Hämäläinen. 2019. Recognizing Emotional Expression in Game Streams. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play (Barcelona, Spain) (CHI PLAY '19). Association for Computing Machinery, New York, NY, USA, 301--311. https://doi.org/10.1145/3311350.3347197
[143]
Ira J Roseman. 1984. Cognitive determinants of emotion: A structural theory. Review of personality & social psychology (1984).
[144]
F. Rozi, Y. Rosmansyah, and B. Dabarsyah. 2019. A Systematic Literature Review on Adaptive Gamification: Components, Methods, and Frameworks. In 2019 International Conference on Electrical Engineering and Informatics (ICEEI). 187--190. https://doi.org/10.1109/ICEEI47359.2019.8988857
[145]
James A Russell. 1980. A circumplex model of affect. Journal of personality and social psychology 39, 6 (1980), 1161.
[146]
Sandra Sampayo-Vargas, Chris J. Cope, Zhen He, and Graeme J. Byrne. 2013. The effectiveness of adaptive difficulty adjustments on students' motivation and learning in an educational computer game. Computers & Education 69 (2013), 452--462. https://doi.org/10.1016/j.compedu.2013.07.004 Proc. ACM Hum.-Comput. Interact., Vol. 5, No. CHI PLAY, Article 229. Publication date: September 2021. To Be or Not to Be Stuck, or Is It a Continuum: A Systematic Literature Review 229:33
[147]
Supat Sanjamsai and Darunee Phukao. 2018. Flow experience in computer game playing among Thai university students. Kasetsart Journal of Social Sciences 39, 2 (2018), 175 -- 182. https://doi.org/10.1016/j.kjss.2018.03.003
[148]
B.R. Schadenberg, M.A. Neerincx, F. Cnossen, and R. Looije. 2017. Personalising game difficulty to keep children motivated to play with a social robot: A Bayesian approach. Cognitive Systems Research 43 (2017), 222--231. https: //doi.org/10.1016/j.cogsys.2016.08.003
[149]
Ruben Schertler, Simone Kriglstein, and Günter Wallner. 2019. User Guided Movement Analysis in Games Using Semantic Trajectories. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play (Barcelona, Spain) (CHI PLAY '19). Association for Computing Machinery, New York, NY, USA, 613--623. https://doi.org/10.1145/ 3311350.3347156
[150]
Henrik Schoenau-Fog, Alexander Birke, and Lars Reng. 2012. Evaluation of Continuation Desire as an Iterative Game Development Method. In Proceeding of the 16th International Academic MindTrek Conference (Tampere, Finland) (MindTrek '12). Association for Computing Machinery, New York, NY, USA, 241--243. https://doi.org/10.1145/2393132. 2393185
[151]
Claudia Schrader, Julia Brich, Julian Frommel, Valentin Riemer, and Katja Rogers. 2017. Rising to the Challenge: An Emotion-Driven Approach Toward Adaptive Serious Games. Springer International Publishing, Cham, 3--28. https://doi.org/10.1007/978--3--319--51645--5_1
[152]
Claudia Schrader and Ulrike Nett. 2018. The perception of control as a predictor of emotional trends during gameplay. Learning and Instruction 54 (2018), 62--72.
[153]
Chen Shu-Hui, Wu Wann-Yih, and Jason Dennison. 2018. Validation of EGameFlow: A Self-Report Scale for Measuring User Experience in Video Game Play. Comput. Entertain. 16, 3, Article 6 (Sept. 2018), 15 pages. https: //doi.org/10.1145/3238249
[154]
Valerie J. Shute, Sidney D'Mello, Ryan Baker, Kyunghwa Cho, Nigel Bosch, Jaclyn Ocumpaugh, Matthew Ventura, and Victoria Almeda. 2015. Modeling how incoming knowledge, persistence, affective states, and in-game progress influence student learning from an educational game. Computers & Education 86 (2015), 224 -- 235. https://doi.org/10. 1016/j.compedu.2015.08.001
[155]
Andy P. Siddaway, Alex M. Wood, and Larry V. Hedges. 2019. How to Do a Systematic Review: A Best Practice Guide for Conducting and Reporting Narrative Reviews, Meta-Analyses, and Meta-Syntheses. Annual Review of Psychology 70, 1 (2019), 747--770. https://doi.org/10.1146/annurev-psych-010418--102803 arXiv:https://doi.org/10.1146/annurevpsych- 010418--102803 30089228.
[156]
Mirna Paula Silva, Victor do Nascimento Silva, and Luiz Chaimowicz. 2017. Dynamic difficulty adjustment on MOBA games. Entertainment Computing 18 (2017), 103--123. https://doi.org/10.1016/j.entcom.2016.10.002
[157]
A. Sinha, R. Gavas, D. Chatterjee, R. Das, and A. Sinharay. 2015. Dynamic assessment of learners' mental state for an improved learning experience. In 2015 IEEE Frontiers in Education Conference (FIE). 1--9.
[158]
Alexander Skulmowski and Günter Daniel Rey. 2017. Measuring cognitive load in embodied learning settings. Frontiers in Psychology 8 (2017), 1191.
[159]
M. Song, Z. Yang, A. Baird, E. Parada-Cabaleiro, Z. Zhang, Z. Zhao, and B. Schuller. 2019. Audiovisual Analysis for Recognising Frustration during Game-Play: Introducing the Multimodal Game Frustration Database. In 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII). 517--523. https://doi.org/10.1109/ ACII.2019.8925464
[160]
Jaakko Stenros. 2017. The Game Definition Game: A Review. Games and culture 12, 6 (2017), 499--520.
[161]
Chuen-Tsai Sun, Li-Xian Chen, and Hsiu-Mei Chu. 2018. Associations among scaffold presentation, reward mechanisms and problem-solving behaviors in game play. Computers & Education 119 (2018), 95--111. https: //doi.org/10.1016/j.compedu.2018.01.001
[162]
Penny Sweetser, Daniel Johnson, and Peta Wyeth. 2012. Revisiting the GameFlow model with detailed heuristics. Journal of Creative Technologies 2012, 3 (2012), 1--16. https://eprints.qut.edu.au/58216/
[163]
Penelope Sweetser, Daniel Johnson, Peta Wyeth, Aiman Anwar, Yan Meng, and Anne Ozdowska. 2017. GameFlow in Different Game Genres and Platforms. Comput. Entertain. 15, 3, Article 1 (April 2017), 24 pages. https://doi.org/10. 1145/3034780
[164]
Penelope Sweetser and Peta Wyeth. 2005. GameFlow: A Model for Evaluating Player Enjoyment in Games. Comput. Entertain. 3, 3 (July 2005), 3. https://doi.org/10.1145/1077246.1077253
[165]
John Sweller. 1988. Cognitive Load During Problem Solving: Effects on Learning. Cognitive Science 12, 2 (1988), 257--285. https://doi.org/10.1207/s15516709cog1202_4 arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1207/s15516709cog1202_4
[166]
John Sweller, Jeroen JG Van Merrienboer, and Fred GWC Paas. 1998. Cognitive architecture and instructional design. Educational psychology review 10, 3 (1998), 251--296.
[167]
Samira Syal and John L. Nietfeld. 2020. The impact of trace data and motivational self-reports in a game-based learning environment. Computers & Education 157 (2020), 103978. https://doi.org/10.1016/j.compedu.2020.103978 Proc. ACM Hum.-Comput. Interact., Vol. 5, No. CHI PLAY, Article 229. Publication date: September 2021. 229:34 Tobias Drey et al.
[168]
M. A. Syufagi, P. Mauridhi Hery, and M. Hariadi. 2011. Modeling serious games based on Cognitive Skill classification using Learning Vector Quantization with Petri net. In 2011 International Conference on Advanced Computer Science and Information Systems. 159--164.
[169]
R. Tadayon, A. Amresh, T. McDaniel, and S. Panchanathan. 2018. Real-time stealth intervention for motor learning using player flow-state. In 2018 IEEE 6th International Conference on Serious Games and Applications for Health (SeGAH). 1--8. https://doi.org/10.1109/SeGAH.2018.8401360
[170]
Y. Tang, J. Liang, R. Hare, and F. Wang. 2020. A Personalized Learning System for Parallel Intelligent Education. IEEE Transactions on Computational Social Systems 7, 2 (2020), 352--361. https://doi.org/10.1109/TCSS.2020.2965198
[171]
Y. Tang and S. Shetty. 2011. Adaptive virtual reality game system for personalized problem-based learning. In 2011 International Conference on Networking, Sensing and Control. 1--6. https://doi.org/10.1109/ICNSC.2011.5874957
[172]
Yu-Hui Tao, C. Rosa Yeh, and Kung Chin Hung. 2012. Effects of the heterogeneity of game complexity and user population in learning performance of business simulation games. Computers & Education 59, 4 (2012), 1350--1360. https://doi.org/10.1016/j.compedu.2012.06.003
[173]
Henrik Thovtrup and Jakob Nielsen. 1991. Assessing the usability of a user interface standard. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 335--341.
[174]
Guy Tsafnat, Paul Glasziou, Miew Keen Choong, Adam Dunn, Filippo Galgani, and Enrico Coiera. 2014. Systematic review automation technologies. Systematic reviews 3, 1 (2014), 74.
[175]
Tzu-Wei Tsai, Hsiao Yu Lo, and Kai-Shao Chen. 2012. An Affective Computing Approach to Develop the Game-Based Adaptive Learning Material for the Elementary Students. In Proceedings of the 2012 Joint International Conference on Human-Centered Computer Environments (Aizu-Wakamatsu, Japan) (HCCE '12). Association for Computing Machinery, New York, NY, USA, 8--13. https://doi.org/10.1145/2160749.2160752
[176]
April Tyack and Elisa D. Mekler. 2020. Self-Determination Theory in HCI Games Research: Current Uses and Open Questions. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '20). Association for Computing Machinery, New York, NY, USA, 1--22. https://doi.org/10.1145/3313831.3376723
[177]
Andika Y. Utomo and Harry B. Santoso. 2015. Development of Gamification-Enriched Pedagogical Agent for e-Learning System Based on Community of Inquiry. In Proceedings of the International HCI and UX Conference in Indonesia (Bandung, Indonesia) (CHIuXiD '15). Association for Computing Machinery, New York, NY, USA, 1--9. https://doi.org/10.1145/2742032.2742033
[178]
Giel van Lankveld, Pieter Spronck, H. Jaap van den Herik, and Matthias Rauterberg. 2010. Incongruity-Based Adaptive Game Balancing. In Advances in Computer Games, H. Jaap van den Herik and Pieter Spronck (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 208--220.
[179]
Rob van Roy and Bieke Zaman. 2018. Need-supporting gamification in education: An assessment of motivational effects over time. Computers & Education 127 (2018), 283--297. https://doi.org/10.1016/j.compedu.2018.08.018
[180]
Matti Vuorre and Janet Metcalfe. 2016. The relation between the sense of agency and the experience of flow. Consciousness and Cognition 43 (2016), 133--142. https://doi.org/10.1016/j.concog.2016.06.001
[181]
V. R. Warriar, J. R. Woodward, and L. Tokarchuk. 2019. Modelling Player Preferences in AR Mobile Games. In 2019 IEEE Conference on Games (CoG). 1--8. https://doi.org/10.1109/CIG.2019.8848082
[182]
R. M. Weerakoon, P. Chundi, and M. Subramaniam. 2011. Dynamically Adapting Training Systems Based on User Interactions. In Proceedings of the 2011 Workshop on Knowledge Discovery, Modeling and Simulation (San Diego, California, USA) (KDMS '11). Association for Computing Machinery, New York, NY, USA, 31--38. https://doi.org/10. 1145/2023568.2023578
[183]
Russell B. Williams. 2018. Conceptual models and mental models in operation: Frustration, performance and flow with two different video game controllers. Entertainment Computing 28 (2018), 2 -- 10. https://doi.org/10.1016/j. entcom.2018.07.004
[184]
Harry J. Witchel, Carlos P. Santos, James K. Ackah, Julian Tee, Nachiappan Chockalingam, and Carina E.I. Westling. 2016. The Complex Relationship Between Empathy, Engagement and Boredom. In Proceedings of the European Conference on Cognitive Ergonomics (Nottingham, United Kingdom) (ECCE '16). Association for Computing Machinery, New York, NY, USA, Article 4, 4 pages. https://doi.org/10.1145/2970930.2970935
[185]
Jacob O. Wobbrock and Julie A. Kientz. 2016. Research Contributions in Human-Computer Interaction. Interactions 23, 3 (April 2016), 38--44. https://doi.org/10.1145/2907069
[186]
S. Wu and T. Lin. 2011. Exploring the use of physiology in adaptive game design. In 2011 International Conference on Consumer Electronics, Communications and Networks (CECNet). 1280--1283.
[187]
Qi-Fan Yang, Shao-Chen Chang,Gwo-Jen Hwang, and Di Zou. 2020. Balancing cognitive complexity and gaming level: Effects of a cognitive complexity-based competition game on EFL students' English vocabulary learning performance, anxiety and behaviors. Computers & Education 148 (2020), 103808. https://doi.org/10.1016/j.compedu.2020.103808
[188]
W. Yang, M. Rifqi, C. Marsala, and A. Pinna. 2018. Physiological-Based Emotion Detection and Recognition in a Video Game Context. In 2018 International Joint Conference on Neural Networks (IJCNN). 1--8. Proc. ACM Hum.-Comput. Interact., Vol. 5, No. CHI PLAY, Article 229. Publication date: September 2021. To Be or Not to Be Stuck, or Is It a Continuum: A Systematic Literature Review 229:35
[189]
Yu-chu Yeh, Guey-Jen Lai, Chun Fu Lin, Chung-Wei Lin, and Hua-Chun Sun. 2015. How stress influences creativity in game-based situations: Analysis of stress hormones, negative emotions, and working memory. Computers & Education 81 (2015), 143--153. https://doi.org/10.1016/j.compedu.2014.09.011
[190]
Chang Yun, Dvijesh Shastri, Ioannis Pavlidis, and Zhigang Deng. 2009. O' Game, Can You Feel My Frustration Improving User's Gaming Experience via Stresscam. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Boston, MA, USA) (CHI '09). Association for Computing Machinery, New York, NY, USA, 2195--2204. https://doi.org/10.1145/1518701.1519036
[191]
Panagiotis Zaharias. 2009. Usability in the context of e-learning: A framework augmenting traditional'usability constructs with instructional design and motivation to learn. International Journal of Technology and Human Interaction (IJTHI) 5, 4 (2009), 37--59.
[192]
Olaf Zawacki-Richter, Victoria I Marín, Melissa Bond, and Franziska Gouverneur. 2019. Systematic review of research on artificial intelligence applications in higher education--where are the educators International Journal of Educational Technology in Higher Education 16, 1 (2019), 39.
[193]
J. Zhao and Y. Li. 2020. Experimental Study on Gamification Teaching of Mathematics Classroom Practice Based on Flow Experience. In 2020 IEEE 2nd International Conference on Computer Science and Educational Informatization (CSEI). 36--40. https://doi.org/10.1109/CSEI50228.2020.9142485
[194]
Yun Zhou, Tao Xu, Shaoqi Li, and Ruifeng Shi. 2019. Beyond engagement: an EEG-based methodology for assessing user's confusion in an educational game. Universal Access in the Information Society 18, 3 (2019), 551--563.
[195]
F. Zwicky. 1967. The Morphological Approach to Discovery, Invention, Research and Construction. In New Methods of Thought and Procedure, Fritz Zwicky and Albert G. Wilson (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 273--297.

Cited By

View all
  • (2025)Pixels and people: Exploring the dynamics of engagement and disengagement in Minecraft's multiplayer realmInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2025.103465198(103465)Online publication date: Apr-2025
  • (2024)VRCreatIn: Taking In-Situ Pen and Tablet Interaction Beyond Ideation to 3D Modeling Lighting and TexturingProceedings of the International Conference on Mobile and Ubiquitous Multimedia10.1145/3701571.3701580(24-35)Online publication date: 1-Dec-2024
  • (2024)LeARn at Home: Comparing Augmented Reality and Video Conferencing Remote TutoringProceedings of the International Conference on Mobile and Ubiquitous Multimedia10.1145/3701571.3701577(255-263)Online publication date: 1-Dec-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Human-Computer Interaction
Proceedings of the ACM on Human-Computer Interaction  Volume 5, Issue CHI PLAY
CHI PLAY
September 2021
1535 pages
EISSN:2573-0142
DOI:10.1145/3490463
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 October 2021
Published in PACMHCI Volume 5, Issue CHI PLAY

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. continuum
  2. games
  3. interviews
  4. stuck
  5. survey
  6. systematic literature review
  7. taxonomy

Qualifiers

  • Research-article

Data Availability

Our supplementary material consists of 2 files: 1) synthesis_being_stuck.xlsx This is the final spreadsheet with all raw data created during full-text screening and used for the synthesis. https://dl.acm.org/doi/10.1145/3474656#V5chip229aux.zip

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)118
  • Downloads (Last 6 weeks)17
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Pixels and people: Exploring the dynamics of engagement and disengagement in Minecraft's multiplayer realmInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2025.103465198(103465)Online publication date: Apr-2025
  • (2024)VRCreatIn: Taking In-Situ Pen and Tablet Interaction Beyond Ideation to 3D Modeling Lighting and TexturingProceedings of the International Conference on Mobile and Ubiquitous Multimedia10.1145/3701571.3701580(24-35)Online publication date: 1-Dec-2024
  • (2024)LeARn at Home: Comparing Augmented Reality and Video Conferencing Remote TutoringProceedings of the International Conference on Mobile and Ubiquitous Multimedia10.1145/3701571.3701577(255-263)Online publication date: 1-Dec-2024
  • (2024)Dropping Hints: Visual Hints for Improving Learning using Mobile Augmented RealityProceedings of the 2024 ACM Symposium on Spatial User Interaction10.1145/3677386.3688900(1-3)Online publication date: 7-Oct-2024
  • (2024)Effects of a Gaze-Based 2D Platform Game on User Enjoyment, Perceived Competence, and Digital Eye StrainProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641909(1-14)Online publication date: 11-May-2024
  • (2023)When Games Become Inaccessible: A Constructive Grounded Theory on Stuckness in VideogamesCompanion Proceedings of the Annual Symposium on Computer-Human Interaction in Play10.1145/3573382.3616036(333-336)Online publication date: 6-Oct-2023
  • (2023)Investigating the Effects of Individual Spatial Abilities on Virtual Reality Object ManipulationProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581004(1-24)Online publication date: 19-Apr-2023
  • (2023)Try Again?: A Macro-Level Taxonomy of the Challenge and Failure Process in Games2023 IEEE Conference on Games (CoG)10.1109/CoG57401.2023.10333199(1-8)Online publication date: 21-Aug-2023
  • (2022)SpARklingPaperProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35503376:3(1-29)Online publication date: 7-Sep-2022
  • (2022)Personalized Game Difficulty Prediction Using Factorization MachinesProceedings of the 35th Annual ACM Symposium on User Interface Software and Technology10.1145/3526113.3545624(1-13)Online publication date: 29-Oct-2022
  • Show More Cited By

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media