Abstract
Stealth assessment is a principled assessment methodology proposed for serious games that uses statistical models and machine learning technology to infer players’ mastery levels from logged gameplay data. Although stealth assessment has been proven to be valid and reliable, its application is complex, laborious, and time-consuming. A generic stealth assessment tool (GSAT), proven for its robustness with simulation data, has been proposed to resolve these issues. In this study, GSAT’s robustness is further investigated by using real-world data collected from a serious game on personality traits and validated with an associated personality questionnaire (NEO PI-R). To achieve this, (a) a stepwise regression approach was followed for generating statistical models from logged data for the big five personality traits (OCEAN model), (b) the statistical models are then used with GSAT to produce inferences regarding learners’ mastery level on these personality traits, and (c) the validity of GSAT’s outcomes are examined through a correlation analysis using the results of the NEO PI-R questionnaire. Despite the small dataset GSAT was capable of making inferences on players’ personality traits. This study has demonstrated the practicable feasibility of the SA methodology with GSAT and provides a showcase for its wider application in serious games.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Larson, L.C., Miller, T.N.: 21st century skills: prepare students for the future. Kappa Delta Pi Rec. 47(3), 121–123 (2011)
Shute, V.J.: Stealth assessment in computer-based games to support learning. Comput. Games Instr. 55(2), 503–524 (2011)
Mislevy, R.J.: Evidence-centered design for simulation-based assessment. CRESST Report 800. National Center for Research on Evaluation, Standards, and Student Testing (CRESST) (2011)
Shute, V.J., Ventura, M., Kim, Y.J.: Assessment and learning of qualitative physics in newton’s playground. J. Educ. Res. 106(6), 423–430 (2013)
Ventura, M., Shute, V., Small, M.: Assessing persistence in educational games. Design Recomm. Adapt. Intell. Tutor. Syst.: Learn. Model. 2, 93–101 (2014)
Shute, V.J., Wang, L., Greiff, S., Zhao, W., Moore, G.: Measuring problem solving skills via stealth assessment in an engaging video game. Comput. Hum. Behav. 63, 106–117 (2016)
Moore, G.R., Shute, V.J.: Improving learning through stealth assessment of conscientiousness. In: Marcus-Quinn, A., Hourigan, T. (eds.) Handbook on Digital Learning for K-12 Schools, pp. 355–368. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-33808-8_21
Georgiadis, K., Van Lankveld, G., Bahreini, K., Westera, W.: Accommodating stealth assessment in serious games: towards developing a generic tool. In 2018 10th International Conference on Virtual Worlds and Games for Serious Applications (VS-Games), pp. 1–4. IEEE (2018)
Georgiadis, K., Van Lankveld, G., Bahreini, K., Westera, W.: Learning analytics should analyse the learning: proposing a generic stealth assessment tool. Accepted at the IEEE Conference on Games (CoG) (2019)
Georgiadis, K., Van Lankveld, G., Bahreini, K., Westera, W.: On the robustness of steath assessment. IEEE Trans. Games (2019, submitted)
Van Lankveld, G., Spronck, P., Van den Herik, J., Arntz, A.: Games as personality profiling tools. In: 2011 IEEE Conference on Computational Intelligence and Games (CIG 2011), pp. 197–202. IEEE (2011)
McCrae, R.R., Costa Jr., P.T.: Personality trait structure as a human universal. Am. Psychol. 52(5), 509 (1997)
Costa, P.T., McCrae, R.R.: The revised neo personality inventory (neo-pi-r). SAGE Handb. Pers. Theory Assess. 2(2), 179–198 (2008)
Sabourin, J.L.: Stealth assessment of self-regulated learning in game-based learning environments (2013)
Min, W., et al.: DeepStealth: leveraging deep learning models for stealth assessment in game-based learning environments. In: Conati, C., Heffernan, N., Mitrovic, A., Verdejo, M.F. (eds.) AIED 2015. LNCS (LNAI), vol. 9112, pp. 277–286. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-19773-9_28
Wiggins, J.S. (ed.): The Five-Factor Model of Personality: Theoretical Perspectives. Guilford Press, New York (1996)
Domingos, P.M.: A few useful things to know about machine learning. Commun. ACM 55(10), 78–87 (2012)
Field, A.: Discovering Statistics Using SPSS. Sage Publications, London (2009)
Green, S.B.: How many subjects does it take to do a regression analysis. Multivar. Behav. Res. 26(3), 499–510 (1991)
Maxwell, S.E.: Sample size and multiple regression analysis. Psychol. Methods 5(4), 434 (2000)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Georgiadis, K., van Lankveld, G., Bahreini, K., Westera, W. (2019). Reinforcing Stealth Assessment in Serious Games. In: Liapis, A., Yannakakis, G., Gentile, M., Ninaus, M. (eds) Games and Learning Alliance. GALA 2019. Lecture Notes in Computer Science(), vol 11899. Springer, Cham. https://doi.org/10.1007/978-3-030-34350-7_49
Download citation
DOI: https://doi.org/10.1007/978-3-030-34350-7_49
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-34349-1
Online ISBN: 978-3-030-34350-7
eBook Packages: Computer ScienceComputer Science (R0)