Abstract
A distinctive feature of intelligent game-based learning environments is their capacity for enabling stealth assessment. Stealth assessments gather information about student competencies in a manner that is invisible, and enable drawing valid inferences about student knowledge. We present a framework for stealth assessment that leverages deep learning, a family of machine learning methods that utilize deep artificial neural networks, to infer student competencies in a game-based learning environment for middle grade computational thinking, Engage. Students’ interaction data, collected during a classroom study with Engage, as well as prior knowledge scores, are utilized to train deep networks for predicting students’ post-test performance. Results indicate deep networks that are pre-trained using stacked denoising autoencoders achieve high predictive accuracy, significantly outperforming standard classification techniques such as support vector machines and naïve Bayes. The findings suggest that deep learning shows considerable promise for automatically inducing stealth assessment models for intelligent game-based learning environments.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Lester, J., Ha, E., Lee, S., Mott, B., Rowe, J., Sabourin, J.: Serious games get smart: Intelligent game-based learning environments. AI Magazine 34(4), 31–45 (2013)
Jackson, T., McNamara, D.: Motivation and Performance in a Game-based Intelligent Tutoring System. Journal of Educational Psychology 105(4), 1036–1049 (2013)
Johnson, L.: Serious use of a serious game for language learning. International Journal of Artificial Intelligence in Education 20(2), 175–195 (2010)
Garris, R., Ahlers, R., Driskell, E.: Games, motivation, and learning: a research and practice model. Simulation & Gaming 33(4), 441–467 (2002)
Rowe, J., Shores, L., Mott, B., Lester, J.: Integrating learning, problem solving, and engagement in narrative-centered learning environments. International Journal of Artificial Intelligence in Education 21(2), 115–133 (2011)
Shute, V.: Stealth assessment in computer-based games to support learning. Computer games and instruction 55(2), 503–524 (2011)
Mislevy, R., Steinberg, L., Almond, R.: On the structure of educational assessment. Measurement: Interdisciplinary research and perspective 1(1), 3–62 (2003)
Baker, R., Corbett, A., Koedinger, K., Wagner, A.: Off-task behavior in the cognitive tutor classroom: when students game the system. In: SIGCHI conference on Human factors in computing systems, pp. 383-390 (2004)
Bengio, Y.: Learning deep architectures for AI. Foundations and Trends in Machine Learning 2(1), 1–127 (2009)
Kebritchi, M., Hirumi, A., Bai, H.: The effects of modern mathematics computer games on mathematics achievement and class motivation. Computers & Education 55(2), 427–443 (2010)
Quellmalz, E., Timms, M., Silberglitt, M., Buckley, B.: Science assessments for all: Integrating science simulations into balanced state science assessment systems. Journal of Research in Science Teaching 49(3), 363–393 (2012)
Sahebi, S., Huang, Y., Brusilovsky, P.: Predicting student performance in solving parameterized exercises. In: Trausan-Matu, S., Boyer, K.E., Crosby, M., Panourgia, K. (eds.) ITS 2014. LNCS, vol. 8474, pp. 496–503. Springer, Heidelberg (2014)
Min, W., Rowe, J.P., Mott, B.W., Lester, J.C.: Personalizing embedded assessment sequences in narrative-centered learning environments: a collaborative filtering approach. In: Lane, H., Yacef, K., Mostow, J., Pavlik, P. (eds.) AIED 2013. LNCS, vol. 7926, pp. 369–378. Springer, Heidelberg (2013)
Corbett, A., Anderson, J.: Knowledge tracing: Modeling the acquisition of procedural knowledge. User Modeling and User-Adapted Interaction 4(4), 253–278 (1994)
Mott, B., Rowe, J., Min, W., Taylor, R., Lester, J.: FLARE: an open source toolkit for creating expressive user interfaces for serious games. In: the 9th International Conference on the Foundations of Digital Games (2014)
AP® Computer Science Principles Draft Curriculum Framework: 2014. http://www.csprinciples.org/. accessed: 2014-09-05
Barr, V., Stephenson, C.: Bringing Computational Thinking to K-12: What is Involved and What is the Role of the Computer Science Education Community? ACM Inroads 2(1), 48–54 (2011)
Fraley, C., Raftery, A.: How many clusters? Which clustering method? Answers via model-based cluster analysis. The computer journal 41(8), 578–588 (1998)
Marx, J., Cummings, K.: Normalized change. American Journal of Physics 75(1), 87–91 (2007)
Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y., Manzagol, P.: Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. The Journal of Machine Learning Research 11, 3371–3408 (2010)
Keerthi, S., Lin, C.: Asymptotic behaviors of support vector machines with Gaussian kernel. Neural computation 15(7), 1667–1689 (2003)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Min, W. et al. (2015). DeepStealth: Leveraging Deep Learning Models for Stealth Assessment in Game-Based Learning Environments. In: Conati, C., Heffernan, N., Mitrovic, A., Verdejo, M. (eds) Artificial Intelligence in Education. AIED 2015. Lecture Notes in Computer Science(), vol 9112. Springer, Cham. https://doi.org/10.1007/978-3-319-19773-9_28
Download citation
DOI: https://doi.org/10.1007/978-3-319-19773-9_28
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-19772-2
Online ISBN: 978-3-319-19773-9
eBook Packages: Computer ScienceComputer Science (R0)