Skip to main content
Springer Nature Link
Log in

Gaming and Confrustion Explain Learning Advantages for a Math Digital Learning Game

  • Conference paper
  • First Online:
Artificial Intelligence in Education (AIED 2021)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 12748))

Included in the following conference series:

Abstract

Digital learning games are thought to support learning by increasing enjoyment and promoting deeper engagement with the content, but few studies have empirically tested hypothesized pathways between digital learning games and learning outcomes. Decimal Point, a digital learning game that teaches decimal operations and concepts to middle school students, has been shown in previous studies to support better learning outcomes than a non-game, computer-based instructional system covering the same content. To investigate the underlying causes for Decimal Point’s learning benefits, we developed log-based detectors using labels from text replay coding of the data from an earlier study. We focused on gaming the system, a form of behavioral disengagement that is frequently associated with worse learning outcomes, and confrustion, an affective state that combines confusion and frustration that has shown mixed results related to learning outcomes. Results indicated that students in the non-game condition engaged in gaming the system at nearly twice the level of students in the game condition, and gaming the system fully mediated the relation between learning condition and learning outcomes. Students in the game condition demonstrated higher levels of confrustion during the self-explanation phase of the game, and while confrustion was not related to learning outcomes in the game condition, it was associated with better learning outcomes in the non-game condition. These results provide evidence that digital learning games may support learning by reducing behavioral disengagement, and that the effects of confusion and frustration may vary depending on digital learning context.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Aleven, V., McLaren, B.M., Sewall, J.: Scaling up programming by demonstration for intelligent tutoring systems development: an open-access website for middle school mathematics learning. IEEE Trans. Learn. Technol. 2(2), 64–78 (2009)

    Article  Google Scholar 

  2. Aleven, V., et al.: Example-tracing tutors: Intelligent tutor development for non-programmers. Int. J. Artif. Intell. Educ. 26(1), 224–269 (2016)

    Article  Google Scholar 

  3. Almeda, M.V., Baker, R.S.: Predicting student participation in STEM careers: the role of affect and engagement during middle dchool. J. Educ. Data Mining 12(2), 33–47 (2020)

    Google Scholar 

  4. Bacher-Hicks, A., Goodman, J., Mulhern, C.: Inequality in household adaptation to schooling shocks: covid-induced online learning engagement in real time. Natl. Bureau Econ. Res. 193, w27555 (2020)

    Google Scholar 

  5. Baker, R.S.: Gaming the system: a retrospective look. Philippine Comput. J. 6(2), 9–13 (2011)

    Google Scholar 

  6. Baker, R.S., De Carvalho, A.M.J.A., Raspat, J., Aleven, V., Corbett, A.T., Koedinger, K.R.: Educational software features that encourage and discourage “gaming the system”. In: Proceedings of the 14th International Conference on Artificial Intelligence in Education, pp. 475–482 (2009)

    Google Scholar 

  7. Baker, R.S., Corbett, A.T., Koedinger, K.R., Wagner, A.Z.: Off-task behavior in the cognitive tutor classroom: when students “game the system.” In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 383–390 (2004)

    Google Scholar 

  8. Baker, R.S., Corbett, A.T., Wagner, A.Z.: Human classification of low-fidelity replays of student actions. In: Proceedings of the Educational Data Mining Workshop at the 8th International Conference on Intelligent Tutoring Systems, pp. 29–36 (2006)

    Google Scholar 

  9. Bedwell, W.L., Pavlas, D., Heyne, K., Lazzara, E.H., Salas, E.: Toward a taxonomy linking game attributes to learning: an empirical study. Simul. Gaming 43(6), 729–760 (2012)

    Article  Google Scholar 

  10. BrightBytes, Inc.: 2020 Remote learning survey research results (2020). https://www.brightbytes.net/rls-research. Accessed 12 Feb 2021

  11. Calvo, R.A., D’Mello, S.: Affect detection: an interdisciplinary review of models, methods, and their applications. IEEE Trans. Affect. Comput. 1(1), 18–37 (2010)

    Article  Google Scholar 

  12. Chen, T., Guestrin, C.: Xgboost: a scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 785–794 (2016)

    Google Scholar 

  13. Cheng, M.T., Chen, J.H., Chu, S.J., Chen, S.Y.: The use of serious games in science education: a review of selected empirical research from 2002 to 2013. J. Comput. Educ. 2(3), 353–375 (2015)

    Article  Google Scholar 

  14. Cheng, M.T., Rosenheck, L., Lin, C.Y., Klopfer, E.: Analyzing gameplay data to inform feedback loops in the radix endeavor. Comput. Educ. 111, 60–73 (2017)

    Article  Google Scholar 

  15. Chi, M.T., De Leeuw, N., Chiu, M.H., LaVancher, C.: Eliciting self-explanations improves understanding. Cogn. Sci. 18(3), 439–477 (1994)

    Google Scholar 

  16. Clark, D.B., Tanner-Smith, E., Killingsworth, S.: Digital games, design, and learning: a systematic review and meta-analysis. Rev. Educ. Res. 86(1), 79–122 (2016)

    Article  Google Scholar 

  17. Cocea, M., Hershkovitz, A., Baker, R.S.: The impact of off-task and gaming behaviors on learning: immediate or aggregate? In: Proceeding of the 2009 Conference on Artificial Intelligence in Education: Building Learning Systems that Care: From Knowledge Representation to Affective Modelling, pp. 507–514. IOS Press (2009)

    Google Scholar 

  18. Common Sense Media. The common sense census: Media use by tweens and teens. https://www.commonsensemedia.org/research/the-common-sense-census-media-use-by-tweens-and-teens. Accessed 12 Feb 2021

  19. Crocco, F., Offenholley, K., Hernandez, C.: A proof-of-concept study of game-based learning in higher education. Simul. Gaming 47(4), 403–422 (2016)

    Article  Google Scholar 

  20. D’Mello, S., Lehman, B., Pekrun, R., Graesser, A.: Confusion can be beneficial for learning. Learn. Instr. 29, 153–170 (2014)

    Article  Google Scholar 

  21. Di Leo, I., Muis, K.R., Singh, C.A., Psaradellis, C.: Curiosity… Confusion? Frustration! the role and sequencing of emotions during mathematics problem solving. Contemp. Educ. Psychol. 58, 121–137 (2019)

    Article  Google Scholar 

  22. Fishman, B., Riconscente, M., Snider, R., Tsai, T., Plass, J.: Empowering Educators: Supporting Student Progress in the Classroom with Digital Games. University of Michigan, Ann Arbor. gamesandlearning.umich.edu/agames (2014)

    Google Scholar 

  23. Forlizzi, J., McLaren, B., Ganoe, C., McLaren, P., Kihumba, G., Lister, K.: Decimal point: designing and developing an educational game to teach decimals to middle school students. In: Busch, C. (ed.) Proceedings of the 8th European Conference on Games Based Learning (ECGBL-2014), pp. 128–135 (2014)

    Google Scholar 

  24. Gagnon, D. J., Harpstead, E., Slater, S.: Comparison of off the shelf data mining methodologies in educational game analytics. In: Proceedings of EDM, pp. 38–43 (2019)

    Google Scholar 

  25. Gamesandlearning.org. http://www.gamesandlearning.org/2014/06/09/teachers-on-using-games-in-class/. Accessed 15 Feb 2021

  26. Glasgow, R., Ragan, G., Fields, W.M., Reys, R., Wasman, D.: The decimal dilemma. Teach. Child. Math. 7(2), 89-93 (2000)

    Google Scholar 

  27. Habgood, M.P.J., Ainsworth, S.E.: Motivating children to learn effectively: exploring the value of intrinsic integration in educational games. J. Learn. Sci. 20(2), 169–206 (2011)

    Article  Google Scholar 

  28. Harpstead, E., MacLellan, C.J., Aleven, V., Myers, B.A.: Using extracted features to inform alignment-driven design ideas in an educational game. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 3329–3338. ACM (2014)

    Google Scholar 

  29. Harpstead, E., Richey, J.E., Nguyen, H., McLaren, B.M.: Exploring the subtleties of agency and indirect control in digital learning games. In: Proceedings of the 9th International Conference on Learning Analytics Knowledge, pp. 121–129. ACM (2019)

    Google Scholar 

  30. Hayes, A.F.: Introduction to Mediation, Moderation, and Conditional Process Analysis: a Regression-Based Approach. Guilford Publications (2017)

    Google Scholar 

  31. Irwin, K.C.: Using everyday knowledge of decimals to enhance understanding. J. Res. Math. Educ. 32(4), 399–420 (2001)

    Article  Google Scholar 

  32. Johnson, C.I., Mayer, R.E.: Adding the self-explanation principle to multimedia learning in a computer-based game-like environment. Comput. Hum. Behav. 26, 1246–1252 (2010)

    Article  Google Scholar 

  33. Ke, F.: Designing and integrating purposeful learning in game play: a systematic review. Educ. Tech. Research Dev. 64(2), 219–244 (2016)

    Article  Google Scholar 

  34. Lee, D.M.C., Rodrigo, M.M. T., Baker, R.S., Sugay, J.O., Coronel, A.: Exploring the relationship between novice programmer confusion and achievement. In: International conference on affective computing and intelligent interaction, pp. 175–184. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-24600-5_21

  35. Lehman, B., et al.: Inducing and tracking confusion with contradictions during complex learning. Int. J. Artif. Intell. Educ. 22(1–2), 85–105 (2013)

    Google Scholar 

  36. Liu, Z., Pataranutaporn, V., Ocumpaugh, J., Baker, R.: Sequences of frustration and confusion, and learning. In: Educational Data Mining (2013)

    Google Scholar 

  37. Mayer, R.E.: Computer games in education. Ann. Rev. Psychol. 70, 531–549 (2019)

    Article  Google Scholar 

  38. McLaren, B.M., Adams, D.M., Mayer, R.E., Forlizzi, J.: A computer-based game that promotes mathematics learning more than a conventional approach. Int. J. Game-Based Learn. (IJGBL) 7(1), 36–56 (2017)

    Article  Google Scholar 

  39. Mogessie, M., Richey, J.E., McLaren, B.M., Andres-Bray, JM.L., Baker, R.S.: Confrustion and gaming while learning with erroneous examples in a decimals game. In International Conference on Artificial Intelligence in Education, pp. 208–213, Springer, Cham (2020). https://doi.org/10.1007/978-3-030-52240-7_38

  40. Nicholson, S.: A user-centered theoretical framework for meaningful gamification, vol. 8, no. 1, pp. 223–230. Games+ Learning + Society (2012)

    Google Scholar 

  41. Nicholson, S.: Two paths to motivation through game design elements: reward-based gamification and meaningful gamification. In: Proceedings of the iConference 2013, pp. 671-672 (2013)

    Google Scholar 

  42. Paquette, L., Baker, R.S., Moskal, M.: A system-general model for the detection of gaming the system behavior in CTAT and LearnSphere. In: International Conference on Artificial Intelligence in Education, pp. 257–260, Springer, Cham (2018). https://doi.org/10.1007/978-3-319-93846-2_47

  43. Paquette, L., de Carvahlo, A., Baker, R., Ocumpaugh, J.: Reengineering the feature distillation process: a case study in detection of gaming the system. In: Educational Data Mining (2014)

    Google Scholar 

  44. Parong, J., Wells, A., Mayer, R.E.: Replicated evidence towards a cognitive theory of game-based training. J. Educ. Psychol. 112(5), 922–937 (2020)

    Article  Google Scholar 

  45. Richey, J.E., et al.: More confusion and frustration, better learning: the impact of erroneous examples. Comput. Educ. 139, 173–190 (2019)

    Article  Google Scholar 

  46. Richey, J.E., et al.: Confrustion in learning from erroneous examples: does type of prompted self-explanation make a difference? In International Conference on Artificial Intelligence in Education, pp. 445–457, Springer, Cham (2019b). https://doi.org/10.1007/978-3-030-23204-7_37

  47. Riconscente, M.M.: Results from a controlled study of the iPad fractions game motion math. Games Cult. 8(4), 186–214 (2013)

    Article  Google Scholar 

  48. Rodrigo, M.M.T., et al.: Affective and behavioral predictors of novice programmer achievement. In: Proceedings of the 14th ACM-SIGCSE Annual Conference on Innovation and Technology in Computer Science Education, pp. 156–160 (2009)

    Google Scholar 

  49. San Pedro, M.O.Z., Baker, R.S.J.D., Bowers, A.J., Heffernan, N.T.: Predicting college enrollment from student interaction with an intelligent tutoring system in middle school. In: Proceedings of the 6th International Conference on Educational Data Mining, pp. 177–184 (2013)

    Google Scholar 

  50. Schneider, B., Krajcik, J., Lavonen, J., Salmela-Aro, K., Broda, M., Spicer, J., et al.: Investigating optimal learning moments in U.S. and finnish science classes. J. Res. Sci. Teach. 53(3), 400–421 (2015)

    Google Scholar 

  51. Seaborn, K., Fels, D.I.: Gamification in theory and action: a survey. Int. J. Hum Comput Stud. 74, 14–31 (2015)

    Article  Google Scholar 

  52. Serrano-Laguna, Á., Torrente, J., Moreno-Ger, P., Fernández-Manjón, B.: Application of learning analytics in educational videogames. Entertainment Comput. 5(4), 313–322 (2014)

    Article  Google Scholar 

  53. Siew, N.M., Geofrey, J., Lee, B.N.: Students’ algebraic thinking and attitudes towards algebra: the effects of game-based learning using Dragonbox 12+ app. Electron. J. Math. Technol. 10(2), 66–79 (2016)

    Google Scholar 

  54. Sitzmann, T.: A meta-analytic examination of the instructional effectiveness of computer-based simulation games. Pers. Psychol. 64, 489–528 (2011)

    Article  Google Scholar 

  55. Slater, S., Ocumpaugh, J., Baker, R., Scupelli, P., Inventado, P.S., Heffernan, N.: Semantic features of math problems: relationships to student learning and engagement. In: Proceedings of the 9th International Conference on Educational Data Mining, pp. 223–230 (2016)

    Google Scholar 

  56. Stacey, K., Helme, S., Steinle, V.: Confusions between decimals, fractions and negative numbers: a consequence of the mirror as a conceptual metaphor in three different ways. In: Heuvel-Panhuizen, M.V.D. (ed.) Proceedings of the 25th Conference of the International Group for the Psychology of Mathematics Education, pp. 217–224. Utrecht, PME (2001)

    Google Scholar 

  57. Suh, S., Kim, S.W., Kim, N.J.: Effectiveness of MMORPG-based instruction in elementary English education in Korea. J. Comput. Assist. Learn. 26, 370–378 (2010)

    Article  Google Scholar 

  58. Tokac, U., Novak, E., Thompson, C.G.: Effects of game-based learning on students’ mathematics achievement: a meta-analysis. J. Comput. Assist. Learn. 35(3), 407–420 (2019)

    Article  Google Scholar 

  59. Vogel, J.J., Vogel, D.S., Cannon-Bowers, J., Bowers, C.A., Muse, K., Wright, M.: Computer gaming and interactive simulations for learning: a meta-analysis. J. Educ. Comput. Res. 34(3), 229–243 (2006)

    Article  Google Scholar 

  60. Wouters, P., van Oostendorp, H. (eds.): Instructional Techniques to Facilitate Learning and Motivation of Serious Games. Springer, New York (2017)

    Google Scholar 

  61. Wu, C.H., Huang, Y.M., Hwang, J.P.: Review of affective computing in education/learning: trends and challenges. Br. J. Edu. Technol. 47(6), 1304–1323 (2015)

    Article  Google Scholar 

  62. Yip, F.W.M., Kwan, A.C.M.: Online vocabulary games as a tool for teaching and learning English vocabulary. Educ. Media Int. 43, 233–249 (2006)

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Science Foundation Award #DRL-1661121. The opinions expressed are those of the authors and do not represent the views of NSF. Thanks to Jimit Bhalani, John Choi, Kevin Dhou, Darlan Santana Farias, Rosta Farzan, Jodi Forlizzi, Craig Ganoe, Rick Henkel, Scott Herbst, Grace Kihumba, Kim Lister, Patrick Bruce Gonçalves McLaren, and Jon Star for important contributions to the development and early experimentation with Decimal Point.

Author information

Authors and Affiliations

  1. Carnegie Mellon University, Pittsburgh, PA, 15213, USA

    J. Elizabeth Richey, Michael Mogessie & Bruce M. McLaren

  2. University of Pennsylvania, Philadelphia, PA, 19104, USA

    Jiayi Zhang, Rohini Das, Juan Miguel Andres-Bray, Richard Scruggs & Ryan S. Baker

Authors
  1. J. Elizabeth Richey
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiayi Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rohini Das
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Juan Miguel Andres-Bray
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Richard Scruggs
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Michael Mogessie
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Ryan S. Baker
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Bruce M. McLaren
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to J. Elizabeth Richey .

Editor information

Editors and Affiliations

  1. Technion – Israel Institute of Technology, Haifa, Israel

    Ido Roll

  2. Arizona State University, Tempe, AZ, USA

    Danielle McNamara

  3. Utrecht University, Utrecht, The Netherlands

    Sergey Sosnovsky

  4. London Knowledge Lab, London, UK

    Rose Luckin

  5. University of Leeds, Leeds, UK

    Vania Dimitrova

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Richey, J.E. et al. (2021). Gaming and Confrustion Explain Learning Advantages for a Math Digital Learning Game. In: Roll, I., McNamara, D., Sosnovsky, S., Luckin, R., Dimitrova, V. (eds) Artificial Intelligence in Education. AIED 2021. Lecture Notes in Computer Science(), vol 12748. Springer, Cham. https://doi.org/10.1007/978-3-030-78292-4_28

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-78292-4_28

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-78291-7

  • Online ISBN: 978-3-030-78292-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics