Abstract
Collaborative game-based learning environments offer the promise of combining the strengths of computer-supported collaborative learning and game-based learning to enable students to work collectively towards achieving problem-solving goals in engaging storyworlds. Group chat plays an important role in such environments, enabling students to communicate with team members while exploring the learning environment and collaborating on problem solving. However, students may engage in chat behavior that negatively affects learning. To help address this problem, we introduce a multidimensional stealth assessment model for jointly predicting students’ out-of-domain contributions to group chat as well as their learning outcomes with multi-task learning. Results from evaluating the model indicate that multi-task learning, which simultaneously performs the multidimensional stealth assessment, utilizing predictive features extracted from in-game actions and group chat data outperforms single-task variants and suggest that multi-task learning can effectively support stealth assessment in collaborative game-based learning environments.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Dillenbourg, P., Järvelä, S., Fischer, F.: The evolution of research on computer-supported collaborative learning. In: Technology-enhanced learning, pp. 3–19. Springer, Dordrecht (2009)
Hmelo-Silver, C.E., Chernobilsky, E.: Understanding collaborative activity systems: the relation of tools and discourse in mediating learning. In: Embracing Diversity in the Learning Sciences: Proceedings of the Sixth International Conference of the Learning Sciences, p. 254. Psychology Press (2004) October)
Jeong, H., Hmelo-Silver, C.E., Jo, K.T.: Ten years of computer-supported collaborative learning: a meta-analysis of CSCL in STEM education during 2005–2014. Educ. Res. Rev. 28, 100284 (2019)
Engle, R.A., Conant, F.R.: Guiding principles for fostering productive disciplinary engagement: explaining an emergent argument in a community of learners classroom. Cogn. Instr. 20(4), 399–483 (2002)
Saleh, A., Chen, Y., Hmelo-Silver, C.E., Glazewski, K.D., Mott, B.W., Lester, J.C.: Coordinating scaffolds for collaborative inquiry in a game-based learning environment. J. Res. Sci. Teach. 57(9), 1490–1518 (2020)
Park, K., et al.: Detecting disruptive talk in student chat-based discussion within collaborative game-based learning environments. In: LAK21: 11th International Learning Analytics and Knowledge Conference, pp. 405–415. (2021) April
Jeong, H., Hmelo-Silver, C.: Technology supports in CSCL (2012)
Carpenter, D., et al.: Detecting off-task behavior from student dialogue in game-based collaborative learning. In: International Conference on Artificial Intelligence in Education, pp. 55–66. Springer, Cham (2020) July
Sabourin, J.L., Rowe, J.P., Mott, B.W., Lester, J.C.: Considering alternate futures to classify off-task behavior as emotion self-regulation: a supervised learning approach. J. Educ. Data Min. 5(1), 9–38 (2013)
Mislevy, R.J., Steinberg, L.S., Almond, R.G.: Focus article: on the structure of educational assessments. Meas. Interdisc. Res. Perspect. 1(1), 3–62 (2003)
Shute, V.J.: Stealth assessment in computer-based games to support learning. Comput. Games Instruction 55(2), 503–524 (2011)
Henderson, N., et al.: Enhancing stealth assessment in game-based learning environments with generative zero-shot learning. International Educational Data Mining Society (2022)
Kim, Y.J., Almond, R.G., Shute, V.J.: Applying evidence-centered design for the development of game-based assessments in physics playground. Int. J. Test. 16(2), 142–163 (2016)
Zhao, W., Shute, V., Wang, L.: Stealth assessment of problem-solving skills from gameplay. In: Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC), (15212) (2015)
Shute, V.J., Rahimi, S.: Stealth assessment of creativity in a physics video game. Comput. Hum. Behav. 116, 106647 (2021)
Min, W., et al.: DeepStealth: game-based learning stealth assessment with deep neural networks. IEEE Trans. Learn. Technol. 13(2), 312–325 (2019)
Zhang, Y., Yang, Q.: A survey on multi-task learning. IEEE Transactions on Knowledge and Data Engineering (2021)
Gupta, A., Carpenter, D., Min, W., Rowe, J.P., Azevedo, R., Lester, J.C.: Multimodal multi-task stealth assessment for reflection-enriched game-based learning. In MAIED@ AIED, pp. 93–102 (2021)
Dillenbourg, P., Fischer, F.: Computer-supported collaborative learning: the basics. Zeitschrift für Berufs-und Wirtschaftspädagogik 21, 111–130 (2007)
Sung, H.Y., Hwang, G.J.: A collaborative game-based learning approach to improving students’ learning performance in science courses. Comput. Educ. 63, 43–51 (2013)
Carpenter, D., et al.: Detecting off-task behavior from student dialogue in game-based collaborative learning. In: Artificial Intelligence in Education: 21st International Conference, AIED 2020, Ifrane, Morocco, July 6–10, 2020, Proceedings, Part I 21 pp. 55-66. Springer International Publishing (2020)
Sanh, V., Debut, L., Chaumond, J., Wolf, T.: DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108 (2019)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Min, W., et al.: A generalized multidimensional evaluation framework for player goal recognition. In: Twelfth Artificial Intelligence and Interactive Digital Entertainment Conference (2016) September)
Blaylock, N., Allen, J.: Corpus-based, statistical goal recognition. In: IJCAI, vol. 3. pp. 1303–1308 (2003) August
Acknowledgements
This research was supported by the National Science Foundation under Grants DRL-1561655, DRL-1561486, IIS-1839966, and SES- 1840120. Any opinions, findings, and conclusions expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Gupta, A. et al. (2023). Enhancing Stealth Assessment in Collaborative Game-Based Learning with Multi-task Learning. In: Wang, N., Rebolledo-Mendez, G., Matsuda, N., Santos, O.C., Dimitrova, V. (eds) Artificial Intelligence in Education. AIED 2023. Lecture Notes in Computer Science(), vol 13916. Springer, Cham. https://doi.org/10.1007/978-3-031-36272-9_25
Download citation
DOI: https://doi.org/10.1007/978-3-031-36272-9_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-36271-2
Online ISBN: 978-3-031-36272-9
eBook Packages: Computer ScienceComputer Science (R0)