research-article

The Puzzle Challenge Analysis Tool. A Tool for Analysing the Cognitive Challenge Level of Puzzles in Video Games

Authors:

Natasha Anne RappaAuthors Info & Claims

Proceedings of the ACM on Human-Computer Interaction, Volume 5, Issue CHI PLAY

Article No.: 276, Pages 1 - 27

https://doi.org/10.1145/3474703

Published: 06 October 2021 Publication History

Abstract

Video games are often designed around puzzles and problem-solving, leading to challenging yet engaging experiences for players. However, it is hard to measure or compare the challenge level of puzzles in video games. This can make designing appropriately challenging puzzles problematic. This study collates previous work to present refined definitions for challenge and difficulty within the context of video games. We present the Puzzle Challenge Analysis tool which can be used to determine the best metrics for analysing the challenge level of puzzles within video games. Previous research has focused on measuring the difficulty of simple action video games, such as Pac-Man, which can be easily modified for research purposes. Existing methods to measure challenge or difficulty include measuring player brain activity, examining game features and player scores. However, some of these approaches cannot be applied to puzzles or puzzle games. For example, approaches relying on game scores will not work for puzzle games with no scoring system, where puzzles are either solved or not. This paper describes the design and development of the Puzzle Challenge Analysis tool using two case studies of commercial video games The Witness and Untitled Goose Game. The tool is also tested for generalisability on a third commercial puzzle video game, Baba Is You. This proposed tool can help game designers and researchers to objectively analyse and compare puzzle challenge and produce more in-depth insights into the player experience. This has implications for designing challenging and engaging games for a range of player abilities.

References

[1]

Ernest Adams, and Andrew Rollings. 2010. Fundamentals of game design (2nd. ed.). New Riders, Berkeley, CA.

[2]

Brendan Z. Allison, and John Polich. 2008. Workload assessment of computer gaming using a single-stimulus event-related potential paradigm. Biological psychology 77, 3, 277--283. https://doi.org/10.1016/j.biopsycho.2007.10.014

[3]

Craig G. Anderson, Kathryn Campbell, and Constance Steinkuehler. 2019. Building persistence through failure: the role of challenge in video games. In Proceedings of the 14th International Conference on the Foundations of Digital Games (FDG '19). ACM, San Luis Obispo, CA, USA. https://doi.org/10.1145/3337722.3337741

Digital Library

[4]

Maria-Virginia Aponte, Guillaume Levieux, and Stéphane Natkin. 2009. Scaling the level of difficulty in single player video games. In Proceedings of the International Conference on Entertainment Computing (ICEC '09). https://doi.org/10.1007/978--3--642-04052--8_3

Digital Library

[5]

Maria-Virginia Aponte, Guillaume Levieux, and Stéphane Natkin. 2011a. Difficulty in videogames: an experimental validation of a formal definition. In Proceedings of the 8th International Conference on Advances in Computer Entertainment Technology (ACE '11). ACM, Lisbon, Portugal. https://doi.org/10.1145/2071423.2071484

Digital Library

[6]

Maria-Virginia Aponte, Guillaume Levieux, and Stéphane Natkin. 2011b. Measuring the level of difficulty in single player video games. Entertainment Computing 2, 4, 205--213. https://doi.org/10.1016/j.entcom.2011.04.001

[7]

Carnegie Mellon University. 2012. Beanstalk. Game [Web browser]. Carnegie Mellon University -- Entertainment Technology Center, Pittsburgh, USA.

[8]

Mihaly Csikszentmihalyi, and Isabella S. Csikszentmihalyi. 1992. Optimal experience: Psychological studies of flow in consciousness. Cambridge University Press.

[9]

Alena Denisova, Paul Cairns, Christian Guckelsberger, and David Zendle. 2020. Measuring perceived challenge in digital games: Development & validation of the challenge originating from recent gameplay interaction scale (CORGIS). International Journal of Human-Computer Studies 137, 102383 (May 2020). https://doi.org/10.1016/j.ijhcs.2019.102383

Digital Library

[10]

DMA Design. 1991. Lemmings. Game [PC]. (February 1991). Psygnosis, Liverpool, UK.

[11]

James Fraser, Michael Katchabaw, and Robert E. Mercer. 2014. A methodological approach to identifying and quantifying video game difficulty factors. Entertainment Computing, 5, 4, 441--449. https://doi.org/10.1016/j.entcom.2014.08.004

[12]

James Paul Gee. 2007. Good Video Games + Good Learning: Collected Essays on Video Games, Learning, and Literacy. Edited by Colin Lankshear, Michele Knobel, Chris Bigum and Michael Peters. Vol. 27. New Literacies and Digital Epistemologies. Peter Lang. New York, USA.

[13]

Audrey Girouard, Erin Treacy Solovey, Leanne M. Hirshfield, Krysta Chauncey, Angelo Sassaroli, Sergio Fantini, and Robert J. K. Jacob. 2009. Distinguishing difficulty levels with non-invasive brain activity measurements. In Proceedings of INTERACT 2009. Uppsala, Sweden. https://doi.org/10.1007/978--3--642-03655--2_50

[14]

Juho Hamari, David J. Shernoff, Elizabeth Rowe, Brianno Coller, Jodi Asbell-Clarke, and Teon Edwards. 2016. Challenging games help students learn: An empirical study on engagement, flow and immersion in game-based learning. Computers in Human Behavior, 54, 170--179. https://doi.org/10.1016/j.chb.2015.07.045

Digital Library

[15]

Erik Harpstead, and Vincent Aleven. 2015. Using empirical learning curve analysis to inform design in an educational game. In Proceedings of the 2015 Annual Symposium on Computer-Human Interaction in Play (CHI Play '15). ACM, London, UK. https://doi.org/10.1145/2793107.2793128

Digital Library

[16]

House House. 2019. Untitled Goose Game. Game [PC]. (September 2019). Panic Inc, Portland, USA.

[17]

Jesper Juul. 2003. The game, the player, the world: Looking for a heart of gameness. In Proceedings of Level Up: Digital Games Research Conference. Utrecht, Netherlands.

[18]

Jesper Juul. 2013. The Art of Failure: An Essay on the Pain of Playing Video Games. The MIT Press.

[19]

Conor Linehan, George Bellord, Ben Kirman, Zachary H. Morford, and Bryan Roche. 2014. Learning curves: analysing pace and challenge in four successful puzzle games. In Proceedings of the first ACM SIGCHI annual symposium on Computer-human interaction in play (CHI Play '14). ACM, Toronto, Canada. https://doi.org/10.1145/2658537.2658695

Digital Library

[20]

J. Derek Lomas, Kenneth Koedinger, Nirmal Patel, Sharan Shodhan, Nikhil Poonwala, and Jodi L. Forlizzi. 2017. Is Difficulty Overrated? The Effects of Choice, Novelty and Suspense on Intrinsic Motivation in Educational Games. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, Denver, Colorado, USA. https://doi.org/10.1145/3025453.3025638

Digital Library

[21]

Matt Makes Games. 2018. Celeste. Game [PC]. (January 2018). Matt Makes Games, Canada.

[22]

Mossmouth, LLC. 2008. Spelunky. Game [Xbox 360]. (July 2012). Mossmouth, LLC, San Francisco, USA.

[23]

Namco. 1980. Pac-Man. Game [arcade]. (July 1980). Namco, Tokyo, Japan.

[24]

Laura Naumann, Matthias Schultze-Kraft, Sven Dähne, and Benjamin Blankertz. 2017. Prediction of difficulty levels in video games from ongoing EEG. In Proceedings of Symbiotic Interaction. https://doi.org/10.1007/978--3--319--57753--1_11

[25]

Nintendo EAD. 1985. Super Mario Bros. Game [Nintendo Entertainment System]. Nintendo, Kyoto, Japan.

[26]

Number None. 2009. Braid. Game [PC]. (April 2009). Number None, USA.

[27]

Alexey Pajitnov, and Vladimir Pokhilko. 1988. Tetris. Game [PC]. (January 1988). Nintendo, Kyoto, Japan.

[28]

Red Storm Entertainment. 2011. Tom Clancy's Rainbow Six: Rogue Spear Black Thorn. Game [PC]. (December 2011). Ubisoft, Montreuil, France.

[29]

Shaghayegh Roohi, Asko Relas, Jari Takatalo, Henri Heiskanen, and Perttu Hämäläinen. 2020. Predicting Game Difficulty and Churn Without Players. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play (CHI Play '20). ACM, Ottawa, Canada. https://doi.org/10.1145/3410404.3414235

Digital Library

[30]

Rovio Entertainment. 2019. Angry Birds Dream Blast. Game [iOS and Android]. (January 2019). Rovio Entertainment, Espoo, Finland.

[31]

Anurag Sarkar, and Seth Cooper. 2019. Transforming Game Difficulty Curves using Function Composition. In the Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). Glasgow, Scotland UK. https://doi.org/10.1145/3290605.3300781

Digital Library

[32]

Arvi Teikari. 2019. Baba Is You. Game [PC]. (March 2019). Hempuli Oy, Finland.

[33]

Thekla Inc. 2016. The Witness. Game [PC]. (January 2016). Thekla Inc, USA.

[34]

Dominicus Tornqvist, and Jennifer Tichon. 2019. Motivated to lose? Evaluating challenge and player motivations in games. Behaviour & Information Technology, 1--22. https://doi.org/10.1080/0144929X.2019.1672789

[35]

Valve. 2007. Portal. Game [PC]. (October 2007). Valve, Kirkland, USA.

[36]

Valve. 2011. Portal 2. Game [PC]. (April, 2011). Valve, Kirkland, USA.

[37]

Rokas Volkovas, Michael Fairbank, John R Woodward, and Simon Lucas. 2019. Extracting Learning Curves From Puzzle Games. In Proceedings of 11th Computer Science and Electronic Engineering (CEEC). Colchester, UK. https://doi.org/10.1109/CEEC47804.2019.8974318

[38]

Robert K. Yin. 2014. Case study research : design and methods. 5th ed. SAGE. California, USA.

[39]

Zachtronics. 2016. Shenzen I/O. Game [PC]. (November 2016). Zachtronics, Redmond, USA.

Cited By

Domingues JFilipe VCarita ACarvalho V(2024)Understanding the Impact of Perceived Challenge on Narrative Immersion in Video Games: The Role-Playing Game Genre as a Case StudyInformation10.3390/info1506029415:6(294)Online publication date: 22-May-2024
https://doi.org/10.3390/info15060294
Kristensen JBurelli P(2024)Difficulty Modelling in Mobile Puzzle GamesInternational Journal of Computer Games Technology10.1155/2024/55923732024Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1155/2024/5592373
Guo ZThawonmas RRen X(2024)Rethinking dynamic difficulty adjustment for video game designEntertainment Computing10.1016/j.entcom.2024.10066350(100663)Online publication date: May-2024
https://doi.org/10.1016/j.entcom.2024.100663
Show More Cited By

Index Terms

The Puzzle Challenge Analysis Tool. A Tool for Analysing the Cognitive Challenge Level of Puzzles in Video Games
1. Applied computing
  1. Computers in other domains
    1. Personal computers and PC applications
      1. Computer games

Recommendations

Challenge in Digital Games: Towards Developing a Measurement Tool
CHI EA '17: Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems

Challenge is arguably the most important experience that players seek in digital games. However, without a measure of how challenged players feel during the act of play, it is hard to design games that are neither too easy nor too hard and, therefore, ...
Scaling the Level of Difficulty in Single Player Video Games
ICEC '09: Proceedings of the 8th International Conference on Entertainment Computing

In this this paper, we discuss the interest and the need to evaluate the difficulty of single player video games. We first show the importance of difficulty, drawing from semiotics to explain the link between tension-resolution cycles, and challenge ...
Difficulty influence on motivation over time in video games using survival analysis
FDG '17: Proceedings of the 12th International Conference on the Foundations of Digital Games

In this paper, we study the link between difficulty and player's motivation in two games developed by Ubisoft®: Rayman®Legends and Tom Clancy's The Division®. We describe a method to estimate players' difficulty over time and link it's time varying ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Human-Computer Interaction

Proceedings of the ACM on Human-Computer Interaction Volume 5, Issue CHI PLAY

CHI PLAY

September 2021

1535 pages

EISSN:2573-0142

DOI:10.1145/3490463

Editors:
Kathrin Gerling
KU Leuven
,
Elisa Mekler
Aalto University
,
Regan Mandryk
University of Saskatchewan

Issue’s Table of Contents

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 October 2021

Published in PACMHCI Volume 5, Issue CHI PLAY

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Australian Government

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
452
Total Downloads

Downloads (Last 12 months)107
Downloads (Last 6 weeks)12

Reflects downloads up to 08 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Domingues JFilipe VCarita ACarvalho V(2024)Understanding the Impact of Perceived Challenge on Narrative Immersion in Video Games: The Role-Playing Game Genre as a Case StudyInformation10.3390/info1506029415:6(294)Online publication date: 22-May-2024
https://doi.org/10.3390/info15060294
Kristensen JBurelli P(2024)Difficulty Modelling in Mobile Puzzle GamesInternational Journal of Computer Games Technology10.1155/2024/55923732024Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1155/2024/5592373
Guo ZThawonmas RRen X(2024)Rethinking dynamic difficulty adjustment for video game designEntertainment Computing10.1016/j.entcom.2024.10066350(100663)Online publication date: May-2024
https://doi.org/10.1016/j.entcom.2024.100663
Wojtkiewicz KPalak RTelec ZLitwinienko F(2024)Modelling Cognitive Load of Computer Game Users - Case StudyEmerging Challenges in Intelligent Management Information Systems10.1007/978-3-031-78465-1_5(52-63)Online publication date: 21-Dec-2024
https://doi.org/10.1007/978-3-031-78465-1_5
Kristensen JGuckelsberger CBurelli PHämäläinen P(2022)Personalized Game Difficulty Prediction Using Factorization MachinesProceedings of the 35th Annual ACM Symposium on User Interface Software and Technology10.1145/3526113.3545624(1-13)Online publication date: 29-Oct-2022
https://dl.acm.org/doi/10.1145/3526113.3545624

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents