Abstract
Metacognitive monitoring and regulation are key dynamic psychological processes in predicting learning, reasoning, and problem solving across AIEd systems. They are impacted by one’s metacognitive knowledge, skills, and experiences. Understanding the dynamical processes underlying metacognition are critical to design intelligent adaptive scaffolding. Metacognition may be assumed to function at hierarchical levels of abstraction including a local level (i.e., isolated metacognitive judgements) and global level (i.e., self-beliefs). However, metacognitive research for complex learning has traditionally disregarded the concept of a general metacognitive ability due to local level fluctuations in temporal metacognitive accuracy. In our study, we shift our analyses to reflect a global-level approach to study metacognitive judgment ability by measuring both accuracy and bias across a series of metacognitive judgments. Using hierarchical clustering on undergraduates’ (n = 58) metacognitive judgments’ accuracy and bias while learning about nine human biological systems with MetaTutor-IVH, a multimedia-based learning environment, we show that some learners show patterns of global-level metacognitive ability. Specifically, we find that learners who tended to have low metacognitive accuracy across all judgment types performed worse overall on learning outcomes. Examining the bias of these learners, we found they tended to be under-confident across all judgment types. Our work suggests that considering multiple metrics of local-level metacognitive judgments’ accuracy and bias, can be aggregated to depict a global-level metacognitive ability of learners that is correlated to learning outcomes. We further discuss the impact of our findings for the design of adaptive scaffolding of AIEd systems to foster metacognition.
The research presented in this paper is supported by funding from the National Science Foundation (DRL 1431532).
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Azevedo, R., Wiedbusch, M.: Theories of metacognition and pedagogy applied in aied systems. In: du Boulay, B., Mitrovic, A., Yacef, K. (eds.) Handbook of Artificial Intelligence in Education. (In Press)
Flavell, J.H.: Metacognition and cognitive monitoring: a new area of cognitive-developmental inquiry. Am. Psychol. 34(10), 906 (1979)
Koriat, A.: When two heads are better than one and when they can be worse: the amplification hypothesis. J. Exp. Psychol. Gen. 144(5), 934 (2015)
Tarricone, P.: The Taxonomy of Metacognition. Psychology Press, New York (2011)
Winne, P.H.: Cognition and metacognition within self-regulated learning. In: Schunk, D., Greene, J. (eds.) Handbook of Self-regulation of Learning and Performance, pp. 36–48. Routledge, New York (2018)
Nelson, T.O., Narens, L.: Metamemory: A theoretical framework and new findings. Psychology of Learning and Motivation, p. 125 (1990)
Veenman, M.V., Hout-Wolters, V., Bernadette, H., Afflerbach, P.: Metacognition and learning: conceptual and methodological considerations. Metacogn. Learn. 1(1), 3–14 (2006)
Seow, T.X., Rouault, M., Gillan, C.M., Fleming, S.M.: How local and global metacognition shape mental health. Biol. Psychiat. 90(7), 436–446 (2021)
Jemstedt, A., Kubik, V., Jönsson, F.U.: What moderates the accuracy of ease of learning judgments? Metacogn. Learn. 12(3), 337–355 (2017). https://doi.org/10.1007/s11409-017-9172-3
Leonesio, R.J., Nelson, T.O.: Do different metamemory judgments tap the same underlying aspects of memory? J. Exp. Psychol. Learn. Mem. Cogn. 16(3), 464 (1990)
Mazzoni, G., Cornoldi, C., Tomat, L., Vecchi, T.: Remembering the grocery shopping list: A study on metacognitive biases. Applied Cognitive Psychology: The Official Journal of the Society for Applied Research in Memory and Cognition 11(3), 253–267 (1997)
McCarley, J.S., Gosney, J.: Metacognitive judgments in a simulated luggage screening task. In: Proceedings of the human factors and ergonomics society annual meeting. vol. 49, pp. 1620–1624. SAGE Publications Sage CA: Los Angeles, CA (2005)
Dever, D.A., Wiedbusch, M., Azevedo, R.: Learners’ gaze behaviors and metacognitive judgments with an agent-based multimedia environment. In: Isotani, S., Millán, E., Ogan, A., Hastings, P., McLaren, B., Luckin, R. (eds.) AIED 2019. LNCS (LNAI), vol. 11626, pp. 58–61. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-23207-8_11
Brewer, W.F., Sampaio, C.: The metamemory approach to confidence: a test using semantic memory. J. Mem. Lang. 67(1), 59–77 (2012)
Chua, E.F., Schacter, D.L., Rand-Giovannetti, E., Sperling, R.A.: Understanding metamemory: neural correlates of the cognitive process and subjective level of confidence in recognition memory. Neuroimage 229(4), 1150–1160 (2006)
Dougherty, M.R., Scheck, P., Nelson, T.O., Narens, L.: Using the past to predict the future. Memory Cogn. 33(6), 1096–1115 (2005)
Hines, J.C., Touron, D.R., Hertzog, C.: Metacognitive influences on study time allocation in an associative recognition task: an analysis of adult age differences. Psychol. Aging 24(2), 462 (2009)
Ryals, A.J., Rogers, L.M., Gross, E.Z., Polnaszek, K.L., Voss, J.L.: Associative recognition memory awareness improved by theta-burst stimulation of frontopolar cortex. Cereb. Cortex 26(3), 1200–1210 (2016)
Biswas, G., Segedy, J.R., Bunchongchit, K.: From design to implementation to practice a learning by teaching system: betty’s brain. Int. J. Artif. Intell. Educ. 26(1), 350–364 (2016)
Britton, B.K., Van Dusen, L., Gülgöz, S., Glynn, S.M., Sharp, L.: Accuracy of learnability judgments for instructional texts. J. Educ. Psychol. 83(1), 43 (1991)
Jönsson, F.U., Kerimi, N.: An investigation of students’ knowledge of the delayed judgements of learning effect. J. Cogn. Psychol. 23(3), 358–373 (2011)
Koriat, A.: How do we know that we know? the accessibility model of the feeling of knowing. Psychol. Rev. 100(4), 609 (1993)
Kelemen, W.L., Frost, P.J., Weaver, C.A.: Individual differences in metacognition: evidence against a general metacognitive ability. Memory Cogn. 28(1), 92–107 (2000)
Schraw, G., Dunkle, M.E., Bendixen, L.D., Roedel, T.D.: Does a general monitoring skill exist? J. Educ. Psychol. 87(3), 433 (1995)
Schraw, G., Nietfeld, J.: A further test of the general monitoring skill hypothesis. J. Educ. Psychol. 90(2), 236 (1998)
Glaser, R., Schauble, L., Raghavan, K., Zeitz, C.: Scientific reasoning across different domains. In: De Corte, E., Linn, M.C., Mandl, H., Verschaffel, L. (eds.) Computer-Based Learning Environments and Problem Solving. NATO ASI Series, vol. 84, pp. 345–371. Springer, Heidelberg (1992). https://doi.org/10.1007/978-3-642-77228-3_16
Veenman, M.V., Wilhelm, P., Beishuizen, J.J.: The relation between intellectual and metacognitive skills from a developmental perspective. Learn. Instr. 14(1), 89–109 (2004)
Veenman, M.V., Prins, F.J., Verheij, J.: Learning styles: self-reports versus thinking-aloud measures. Br. J. Educ. Psychol. 73(3), 357–372 (2003)
Azevedo, R., Taub, M., Mudrick, N.V., Martin, S., Grafsgaard, J.: Using multi-channel trace data to infer and foster self-regulated learning between humans and advanced learning technologies. In: Handbook of self-regulation of learning and performance, vol. 2. Routledge New York (2018)
Dever, D.A., Wortha, F., Wiedbusch, M.D., Azevedo, R.: Effectiveness of system-facilitated monitoring strategies on learning in an intelligent tutoring system. In: Zaphiris, P., Ioannou, A. (eds.) HCII 2021. LNCS, vol. 12784, pp. 250–263. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-77889-7_17
Taub, M., Azevedo, R.: How does prior knowledge influence eye fixations and sequences of cognitive and metacognitive SRL processes during learning with an intelligent tutoring system? Int. J. Artif. Intell. Educ. 29(1), 1–28 (2019)
Wiedbusch, M., Dever, D., Wortha, F., Cloude, E.B., Azevedo, R.: Revealing data feature differences between system- and learner-initiated self-regulated learning processes within hypermedia. In: Sottilare, R.A., Schwarz, J. (eds.) HCII 2021. LNCS, vol. 12792, pp. 481–495. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-77857-6_34
Pekrun, R., Goetz, T., Frenzel, A.C., Barchfeld, P., Perry, R.P.: Measuring emotions in students’ learning and performance: the achievement emotions questionnaire (AEQ). Contemp. Educ. Psychol. 36(1), 36–48 (2011)
Gross, J.J., John, O.P.: Individual differences in two emotion regulation processes: implications for affect, relationships, and well-being. J. Pers. Soc. Psychol. 85(2), 348 (2003)
Chow, P.I., Berenbaum, H.: Perceived utility of emotion: the structure and construct validity of the perceived affect utility scale in a cross-ethnic sample. Cultur. Divers. Ethnic Minor. Psychol. 18(1), 55 (2012)
Wiedbusch, M. D., Azevedo, R.: Modeling metacomprehension monitoring accuracy with eye gaze on informational content in a multimedia learning environment. In: ACM Symposium on Eye Tracking Research and Applications, pp. 1–9 (2020)
Schraw, G.: A conceptual analysis of five measures of metacognitive monitoring. Metacogn. Learn. 4(1), 33–45 (2009)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Wiedbusch, M.D., Sonnenfeld, N., Dever, D., Azevedo, R. (2022). Clustering Learner’s Metacognitive Judgment Accuracy and Bias to Explore Learning with AIEd Systems. In: Rodrigo, M.M., Matsuda, N., Cristea, A.I., Dimitrova, V. (eds) Artificial Intelligence in Education. AIED 2022. Lecture Notes in Computer Science, vol 13355. Springer, Cham. https://doi.org/10.1007/978-3-031-11644-5_33
Download citation
DOI: https://doi.org/10.1007/978-3-031-11644-5_33
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-11643-8
Online ISBN: 978-3-031-11644-5
eBook Packages: Computer ScienceComputer Science (R0)