The effects of animated pedagogical agents in an English-as-a-foreign-language learning environment

https://doi.org/10.1016/j.ijhcs.2016.06.001Get rights and content

Highlights

  • We investigated degrees of APA embodiment in CALL systems.

  • We implemented a CALL system and an Animated Pedagogical Agent for Brazilian students.

  • Gain scores from using different APA versions were compared to analyze agent effects.

  • Modality and Embodied Effects were found, Image and Expressiveness Effects were not.

Abstract

Animated pedagogical agents (APAs) have frequently been used as a powerful addition to learning environments, since APAs have been known to facilitate learning. APAs can present various features, such as voice, movements, gestures and pointing, and researchers have sought to verify specifically which features of agents effectively contribute to learning. Previous studies have studied these features by comparing different degrees of agent embodiment in the evaluation of the image effect (i.e., students learn more when learning systems have visual APAs), the embodied agent effect (i.e., fully embodied agents that deliver instruction aurally and use gestures to improve learning outcomes in text-only learning systems), the modality effect (i.e., oral instruction contributes to the learning process), and the expressiveness effect (i.e., fully embodied agents promote more effective learning than static ones). Some of these studies have investigated the image, embodied agent and modality effects in the same learning environment, but they were not the same studies that investigated the expressiveness effect. The expressiveness effect allows us to separate the movements of the agent from its other features, such as the agent's image, so investigating this effect is as important as investigating the other effects. We are not aware of any studies that investigated all of these four effects within the same learning system, nor that evaluated any of these effects in language learning environments. Accordingly, this paper describes the design, implementation, and analysis of an APA designed to evaluate the abovementioned effects. The APA was integrated into a computer-assisted language learning (CALL) system to teach English as a foreign language to Brazilian students. A total of 72 Brazilian undergraduate students were divided into four groups, each of which used a different version of the APA in the same CALL system: no agent, a voice-only agent, a static agent, or a fully embodied agent. We compared students’ gain scores (i.e., difference between pre- and posttest scores) across groups to evaluate each of the four effects. Though the outcomes of our study supported the presence of the embodied agent and modality effects, we were not able to demonstrate the image or expressiveness effects in the experiment. Our results indicate that the voice of the agent might contribute more positively to learning than movements, gestures and pointing.

Introduction

Animated pedagogical agents (APAs) are animated on-screen characters that assist learners in multimedia learning environments (Johnson et al., 1998, Lester and Stone, 1997). APAs can exhibit various types of lifelike behaviors, including speech, emotions, gestures, and movements of the eyes, head, and even body (Dehn and van Mulken, 2000). Such characters can flexibly represent numerous pedagogical roles, including those of instructors, coaches, tutors, and learning companions (Chou et al., 2003, Baylor and Kim, 2005, Haake and Gulz, 2009).

Since APAs’ behaviors in some way mimic social exchange present in human interactions, they are considered to be a powerful addition to multimedia learning environments, because they allow these environments to exploit both verbal and nonverbal forms of communication (Johnson and Lester, 2015). Furthermore, as some research (Jeung et al., 1997, Mousavi et al., 1995) has indicated, APAs can be used during the learning process to partially reduce cognitive load.

However, using APAs is not invariably beneficial. One negative effect of their use can be the so-called split-attention effect, which occurs when the agent and its presence are not cognitively integrated enough for the content to be taught. In such cases, the image of the agent represents an additional, non-desirable cognitive load, likely because students are distracted by the agent and pay more attention to it than to the visual content that should be integrated with the spoken text (Dunsworth and Atkinson, 2007, Craig et al., 2002). A possible alternative to avoid the split-attention effect is to explicitly direct learners’ visual attention to the content (Alibali and DiRusso, 1999).

In research on APAs in multimedia learning environments conducted over the years (Johnson et al., 1998, Lester and Stone, 1997, Finkelstein et al., 2013, Jaques et al., 2009), a recurring issue has been how to evaluate agents toward determining which degree of agent embodiment is most effective. For these evaluations, Atkinson (2002) has proposed three agent effects to take into consideration: the image effect, which represents whether the agent's on-screen presence helps in learning; the embodied agent effect, which concerns simulating the student–instructor relationship; and the modality effect, which holds that orally delivered instruction aids learning more effectively than textually delivered instruction. Moreover, some studies, including that of Lusk and Atkinson (2007), have investigated the so-called expressiveness effect, which maintains the effectiveness of fully embodied agents using gestures, locomotion, and gazes over static APAs.

Although several studies in different learning domains have sought to verify how these different effects impact students’ learning, we are not aware of any studies that have investigated these four effects together in the same learning system. Some studies have tried to verify the image effect, the embodied agent and the modality effect in the same learning system (Atkinson, 2002, Dunsworth and Atkinson, 2007). However, these effects do not consider static versions of the agent, which limits conclusions about the impact of the agent's level of animation on learning. Adding the evaluation of the expressiveness effect in the same experiment allows us to verify and compare the effects of the agent's level of animation independently from other features of the agent.

This paper reports a study of the four aforementioned effects in an experiment involving a multimedia computer-assisted language learning (CALL) system. We hypothesize that our system will be able to demonstrate all the four abovementioned effects, i.e., that the agent's visual presence (image effect), ability to simulate the student–instructor relationship (embodied agent effect), voice (modality effect), and gestures locomotion and gazes (expressiveness effect) facilitate learning.

This paper is organized as follows. Section 2 discusses APAs and the effects proposed by Atkinson (2002) and by Lusk and Atkinson (2007). The system created for the project is described in Section 3, while the experiment is detailed in Section 4.1. Results from the data appear in Section 5, a discussion of the results in Section 6 and some final remarks in Section 7.

Section snippets

The agent effects

Although findings of previous studies suggest that the use of APAs facilitates learning (Johnson et al., 1998, Lester and Stone, 1997, Finkelstein et al., 2013, Jaques et al., 2009, Schroeder et al., 2013), Lusk and Atkinson (2007) posit that the effectiveness of APAs may in part be due to their degree of embodiment. Thus, we need to study features of agents in different APAs’ degrees of embodiment to build most effective agents.

Moreno and Mayer (2000) have proposed a way to analyze the several

The multimedia learning system

To evaluate the previously described effects in an English learning system for Brazilian students, we developed a multimedia CALL system specifically for this study. In this section, we describe the design of the CALL system (Section 3.1) and of the APA (Section 3.2).

Evaluation study

The goal of the present study is to evaluate how the agent effects proposed by Atkinson and his colleagues impact learning. Specifically, we aim at finding evidence on whether the presence of the agent on the screen (i.e., the image effect), the student–instructor relationship simulated by the agent (i.e., the embodied agent effect), the instruction delivered orally by the agent instead of textually (i.e., the modality effect), and the use of gestures, locomotion, and gazes by the agent (i.e.,

Results

This section presents the results obtained in the experiment. Section 5.1 describes the results from the pre- and posttests, while results from the effects appear in Section 5.2, and outcomes of the questionnaire in Section 5.3.

Discussions

This section analyzes the results presented in the previous section. In particular, we aim to discuss whether the results from the experiment confirm the existence of the four abovementioned APA effects — namely, the image effect, embodied agent effect, modality effect, and expressiveness effect — in CALL systems. In addition, we also discuss what can be concluded from the evaluation of the four effects in the same learning system.

Conclusion

The goal of this study was to investigate the image effect, embodied agent effect, modality effect, and expressiveness effect of APAs in the learning of English as a foreign language. To verify how these effects impact learning, we integrated three versions of an APA into a CALL system developed specifically for this research, resulting in four different versions of the CALL system.

The effects were analyzed by comparisons across the four versions of the system: image effect (voice-only versus

References (42)

  • A.L. Baylor et al.

    Simulating instructional roles through pedagogical agents

    Int. J. Artif. Intell. Educ.

    (2005)
  • S. Choi et al.

    Cognitive and affective benefits of animated pedagogical agents for learning a second language

    J. Educ. Comput. Res.

    (2006)
  • J. Cohen

    Statistical Power Analysis for the Behavioral Statistical Power Analysis for the Behavioral Sciences

    (1988)
  • S.W. Cook et al.

    Consolidation and transfer of learning after observing hand gesture

    Child Dev.

    (2013)
  • S.D. Craig et al.

    Animated pedagogical agents in multimedia educational environments

    J. Educ. Psychol.

    (2002)
  • S. D'mello et al.

    Autotutor and affective autotutor

    ACM Trans. Interact. Intell. Syst.

    (2012)
  • Finkelstein, S.L., Yarzebinski, E., Vaughn, C., Ogan, A., Cassell, J., 2013. The effects of culturally congruent...
  • N.D. Fleming et al.

    Not another inventory, rather a catalyst for reflection

    Improve Acad.

    (1992)
  • R. Galluccio

    Animated Pedagogical Agents as Spanish Language Instructors: Effect of Accent, Appearance, and Type of Activity on Student Performance, Motivation, and Perception of Agent

    (2008)
  • M. Haake et al.

    A look at the roles of look & roles in embodied pedagogical agents — a user preference perspective

    Int. J. Artif. Intell. Educ.

    (2009)
  • Jaques, P.A., Lehmann, M., Pesty, S., 2009. Evaluating the affective tactics of an emotional pedagogical agent. In:...
  • Cited by (38)

    • Technology-Assisted Language Learning Adaptive Systems: A Comprehensive Review

      2023, International Journal of Cognitive Computing in Engineering
    • A systematic review of pedagogical agent research: Similarities, differences and unexplored aspects

      2022, Computers and Education
      Citation Excerpt :

      Early studies evaluated subjective measurements such as perceived credibility (Lester et al., 1997) and usefulness (Cassell & Thórisson, 1999). More recent studies also inspected perceived cognitive load (Lin, Atkinson, Christopherson, Joseph, & Harrison, 2013; Noh, Song, Nam, & Park, 2014) and variables indirectly benefiting learning, such as motivation and engagement (Carlotto & Jaques, 2016; Schmidt, Bruder, & Steinicke, 2019). Objective learning outcome measures typically included retention, transfer, and recall (Craig & Schroeder, 2017; Domagk, 2010; Wang, Li, Mayer, & Liu, 2018).

    • Foreign Language Teaching and Learning Behaviour with a Big Data Corpus

      2024, Applied Mathematics and Nonlinear Sciences
    View all citing articles on Scopus
    View full text