The Effectiveness of a Pedagogical Agent’s Immediate Feedback on Learners’ Metacognitive Judgments during Learning with MetaTutor

Abstract

Using pedagogical agents (PAs) in hypermedia learning environments have been found to be an effective way to scaffold students and provide tailored feedback to enhance learning outcomes. In this study, we investigated the effectiveness of immediate feedback provided by PA embedded in MetaTutor [1] (a multi-agent, adaptive hypermedia learning system) on learners’ metacognitive calibration and bias of Feelings of Knowing (FOK) and Judgments of Learning (JOL), and accuracy of Content Evaluations (CE) made during a 2-hour learning session with the system. Seventy (N = 70) undergraduate students were randomly assigned to one of two instructional conditions: Immediate Feedback (IF) or the Control Group, where they were asked to learn about the circulatory system with the environment. Overall, pretest-posttest learning outcome data revealed that participants in the IF condition significantly outperformed those in the Control condition. Additionally, participants who received immediate feedback from the PA were more accurate and calibrated in their metacognitive judgments than those in the Control condition. An overall bias was found toward overconfidence in JOLs and FOKs for participants in both conditions. These findings have significant relevance for the understanding of metacognitive monitoring and regulation during complex learning with multi-agent systems and for designing metacognitive-responsive PAs capable of co-adapting to learners’ cognitive and metacognitive regualtory processes.