Skip to main content
SpringerLink
Log in

An Introduction to Game-Based Competence Assessment Based on Cognitive Diagnostic Models

  • Conference paper
  • First Online:
Games and Learning Alliance (GALA 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14475))

Included in the following conference series:

  • 402 Accesses

Abstract

In Serious Games, in particular in learning and training games, the assessment of competencies and skills is crucial for monitoring learning progress, tailoring learning experiences, and providing individual formative feedback. A sound psychometric diagnostic of competencies is not trivial, however. Conventional scoring techniques have severe shortcomings in terms of accuracy and the degree to which actionable information can be drawn from them. In this paper we introduce Cognitive Diagnostic Models and in particular Competence-based Knowledge Space Theory as theoretical underpinnings of in-game competence assessments. We exemplify the approach by a gamified mathematics learning scenario named Mathiade and illustrate the steps of developing, implementing, and evaluating competence models.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Clark, D.B., Tanner-Smith, E.E., Killingsworth, S.S.: Digital games, design, and learning: a systematic review and meta-analysis. Rev. Educ. Res. 86(1), 79–122 (2016)

    Article  Google Scholar 

  2. Wouters, P.J.M., van Oostendorp, H.: A meta-analytic review of the role of instructional support in game-based learning. Comput. Educ. 60, 412–425 (2013)

    Article  Google Scholar 

  3. Wouters, P.J.M., Van Nimwegen, C., Van Oostendorp, H., Van Der Spek, E.D.: A meta-analysis of the cognitive and motivational effects of serious games. J. Educ. Psychol. 105, 249–265 (2013)

    Article  Google Scholar 

  4. Wouters, P., van Oostendorp, H.: Overview of instructional techniques to facilitate learning and motivation of serious games. In: Wouters, P., van Oostendorp, H. (eds.) Instructional Techniques to Facilitate Learning and Motivation of Serious Games. AGL, pp. 1–16. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-39298-1_1

    Chapter  Google Scholar 

  5. Kickmeier-Rust, M.D., Albert, D.: Educationally adaptive: balancing serious games. Int. J. Comput. Sci. Sport 11(1), 15–28 (2012)

    Google Scholar 

  6. Bellotti, F., Kapralos, B., Lee, K., Moreno-Ger, P., Berta, R.: Assessment in and of serious games: an overview. Adv. Hum.-Comput. Interact. 2013, 1–11 (2013)

    Article  Google Scholar 

  7. Shute, V., Ke, F., Wang, L.: Assessment and adaptation in games. In: Wouters, P., van Oostendorp, H. (eds.) Instructional Techniques to Facilitate Learning and Motivation of Serious Games. AGL, pp. 59–78. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-39298-1_4

    Chapter  Google Scholar 

  8. Akram, B., Wookhee, M., Wiebe, E., Mott, B., Boyer, K.E., Lester, J.: Improving stealth assessment in game-based learning with LSTM-based analytics. In: Proceedings of the Eleventh International Conference on Educational Data Mining, Buffalo, New York (2018)

    Google Scholar 

  9. Nyamsuren, E., van der Maas, H.L.J., Maurer, M.: Set-theoretical and combinatorial instruments for problem space analysis in adaptive serious games. Int. J. Serious Games 5(1), 5–18 (2018)

    Article  Google Scholar 

  10. Heller, J., Stefanutti, L., Anselmi, P., Robusto, E.: On the link between cognitive diagnostic models and knowledge space theory. Psychometrika 80, 995–1019 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  11. Käser, T., Klingler, S., Schwing, A.G., Gross, M.: Dynamic Bayesian Networks for student modeling. IEEE Trans. Learn. Technol. 10(4), 450–462 (2017)

    Article  Google Scholar 

  12. Mislevy, R.J., Oranje, A., Bauer, M., von Davier, A.A., Hao, J., et al.: Psychometric Considerations in Game-Based Assessment. GlassLab, Redwood City, CA (2014)

    Google Scholar 

  13. Kickmeier-Rust, M.D.: Learning analytics for an in serious games. In: Proceedings of the Joint Workshop of the GALA Network of Excellence and the LEA’s BOX Project at EC-TEL 2014, 17 September 2014, Graz, Austria (2014)

    Google Scholar 

  14. Rowe, J., Lester. J.: Improving student problem solving in narrative-centered learning environments: a Modular Reinforcement Learning Framework. In: Proceedings of the Seventeenth International Conference on Artificial Intelligence in Education, Madrid, Spain (2015)

    Google Scholar 

  15. de la Torre, J., Carmona, G., Kieftenbeld, V., Tjoe, H., Lima, C.: Diagnostic classification models and mathematics education research: opportunities and Challenges. In: Izsák, A., Remillard, J.T., Templin, J. (Eds.), Psychometric Methods in Mathematics Education: Opportunities, Challenges, and Interdisciplinary Collaborations, pp. 53–72. National Council of Teachers of Mathematics, Reston, VA (2016)

    Google Scholar 

  16. Xin, T., Wang, C., Chen, P., Liu, Y.: Editorial: cognitive diagnostic models: methods for practical applications. Front. Psychol. 13, 895399 (2021)

    Article  Google Scholar 

  17. Falmagne, J.-C.: Thirty years of knowledge space theory: the beginning, the core ideas, and the assessment spaces. Paper presented at the EMPG Meeting 2015 at Padua, Italy, 01–03 September (2015)

    Google Scholar 

  18. Heller, J., Stefanutti, L., Anselmi, P., Robusto, E.: Erratum to: on the link between cognitive diagnostic models and knowledge space theory. Psychometrika 81, 250–251 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  19. Doignon, J.: Probabilistic Assessment of Knowledge. In: Albert, D. (ed.) Knowledge Structures, pp. 1–56. Springer, New York (1994). https://doi.org/10.1007/978-3-642-52064-8_1

    Chapter  Google Scholar 

  20. Düntsch, I., Gediga, G.: Skills and knowledge structures. Br. J. Math. Stat. Psychol. 48, 9–27 (1995)

    Article  MATH  Google Scholar 

  21. Korossy, K.: Modelling knowledge as competence and performance. In: Albert, D., Lukas, J. (eds.) Knowledge Spaces: Theories, Empirical Research Applications, pp. 103–132. Lawrence Erlbaum Associates, Mahwah, NJ (1999)

    Google Scholar 

  22. Kambouri, M., Koppen, M., Villano, M., Falmagne, J.-C.: Knowledge assessment: tapping human expertise by the QUERY routine. Int. J. Hum. Comput. Stud. 40, 119–151 (1994)

    Article  Google Scholar 

  23. Schrepp, M., Held, T., Albert, D.: Component–based construction of surmise relations for chess problems. In: Albert, D., Lukas, J. (eds.) Knowledge Spaces: Theories, Empirical Research, and Applications, pp. 41–66. Lawrence Erlbaum Associates, Mahwah, NJ (1999)

    Google Scholar 

  24. Schrepp, M.: A method for comparing knowledge structures concerning their adequacy. J. Math. Psychol. 45, 480–496 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  25. Albert, D., Kickmeier-Rust, M.D., Matsuda, F.: A formal framework for modelling the developmental course of competence and performance in the distance, speed, and time domain. Dev. Rev. 28, 401–420 (2008)

    Article  Google Scholar 

  26. Obersteiner, A., Dresler, T., Bieck, S.M., Moeller, K.: Understanding fractions: Integrating results from mathematics education, cognitive psychology, and neuroscience. Construct. Num.: Merg. Perspect. Psychol. Math. Educ., 135–162 (2019)

    Google Scholar 

  27. Padberg, F.: Didaktik der Bruchrechnung für Lehrerausbildung und Lehrerfortbildung, 4th edn. Spektrum, Heidelberg (2009)

    Book  MATH  Google Scholar 

  28. Oliveira, W., Hamari, J., Shi, L., et al.: Tailored gamification in education: a literature review and future agenda. Educ. Inf. Technol. 28, 373–406 (2023)

    Article  Google Scholar 

  29. Nebel, S., Beege, M., Schneider, S., Rey, G.D.: The higher the score, the higher the learning outcome? Heterogeneous impacts of leaderboards and choice within educational videogames. Comput. Hum. Behav. 65, 391–401 (2016)

    Article  Google Scholar 

  30. Landers, R.N., Bauer, K.N., Callan, R.C.: Gamification of task performance with leaderboards: a goal setting experiment. Comput. Hum. Behav. 71, 508–515 (2017)

    Article  Google Scholar 

  31. Erickson, L.V., Sammons-Lohse, D.: Learning through video games: the impacts of competition and cooperation. E-Learn. Digit. Med. 18(1), 1–17 (2021)

    Article  Google Scholar 

  32. Reza Keramati, M., Gillies, R.M.: Teaching cooperative learning through cooperative learning environment: a qualitative follow-up of an experimental study. Interact. Learn. Environ., 1–13 (2022)

    Google Scholar 

  33. Hockemeyer, C.: Documentation of the libsrbi Library (2000). https://kst.cord-hockemeyer.info/techreports/libsrbi_TechRep_FWF00.pdf

  34. Kickmeier-Rust, M.D., Albert, D. (eds.): An Alien’s Guide to Multi-adaptive Educational Games. Informing Science Press, Santa Rosa, CA (2012)

    Google Scholar 

  35. Kim, Y.J., Almond, R.J., Shute, V.: Applying evidence-centered design for the development of game-based assessments in physics playground. Int. J. Test., 1–22 (2016)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. St.Gallen University of Teacher Education, St.Gallen, Switzerland

    Michael D. Kickmeier-Rust, Corsin Niggli & Katharina Richter

Authors
  1. Michael D. Kickmeier-Rust
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Corsin Niggli
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Katharina Richter
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael D. Kickmeier-Rust .

Editor information

Editors and Affiliations

  1. Technological University Dublin, Dublin, Ireland

    Pierpaolo Dondio

  2. Technological University Dublin, Dublin, Ireland

    Mariana Rocha

  3. University of Galway, Galway, Ireland

    Attracta Brennan

  4. Berlin School of Economics and Law, Berlin, Germany

    Avo Schönbohm

  5. CAPTRS, Austin, TX, USA

    Francesca de Rosa

  6. Tampere University, Tampere, Finland

    Antti Koskinen

  7. University of Genova, Genoa, Italy

    Francesco Bellotti

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kickmeier-Rust, M.D., Niggli, C., Richter, K. (2024). An Introduction to Game-Based Competence Assessment Based on Cognitive Diagnostic Models. In: Dondio, P., et al. Games and Learning Alliance. GALA 2023. Lecture Notes in Computer Science, vol 14475. Springer, Cham. https://doi.org/10.1007/978-3-031-49065-1_24

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-49065-1_24

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-49064-4

  • Online ISBN: 978-3-031-49065-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation