The influence of learners' perceptions of virtual humans on learning transfer

Highlights

• The revised Agent Persona Instrument fit well with the four-factor structure.

• K-means clustering identified two clusters of response patterns on the API.

• Cluster membership had a small influence on learning transfer scores.

Abstract

Virtual humans are often integrated into novel multimedia learning environments. However, little is known about learner's perceptions of the agents or the relationship between the perceptions and learning. In this study, the authors revise the Agent Persona Instrument, a measurement tool designed to examine how learners perceive pedagogical agents. The factor structure of the revised instrument was evaluated with confirmatory factor analysis. Next, k-means clustering was used to examine how participants' ratings on the instrument cluster into groups based on their perceptions of the virtual human. After describing the qualities of the clusters within the data, linear regression was used to examine to what extent the cluster membership influenced participant's scores on a transfer test of learning. The results indicated that cluster membership only explained a small amount of variance in transfer test scores. This study provides a revised instrument for measuring pedagogical agent persona. It implemented a novel method for investigating perceptions of pedagogical agents using k-means clustering which was able to identify two unique groups of participants based on their perceptions of the agent. Finally, it presents empirical results indicating that learner's perceptions of the agent had a small influence on their learning outcome scores.

Introduction

How we learn with computers is constantly changing, and each year novel learning technologies are developed as researchers continue to push the boundaries of what is technically possible. Yet, no matter how new or novel a learning environment may be, it will still require instructional design decisions to be made. For example, an instructional designer can choose to incorporate virtual humans within the learning environment to facilitate the learning process.

In this paper, we focus on virtual humans, how they are perceived by learners, and the influence of those perceptions on learning. By definition, virtual humans are on-screen, humanlike representations of the software that often facilitate either direct instruction (Johnson, Ozogul, Moreno, & Reisslein, 2013; Yung & Paas, 2015) or other pedagogical approaches such as vicarious learning (Twyford & Craig, 2017). The term ‘virtual human’ embraces many more specific software entities such as pedagogical agents (Heidig & Clarebout, 2011; Schroeder, Adesope, & Gilbert, 2013), conversational agents (Louwerse, Graesser, Lu, & Mitchell, 2005), and motivational agents (Van der Meij, 2013). For the purposes of this paper, we use the terms ‘virtual human’, ‘pedagogical agent’, and ‘agent’ interchangeably.

Researchers have investigated agent-enhanced learning systems across a number of different content domains, such as language learning (Choi & Clark, 2006), mathematics (Atkinson, 2002), earth science (Frechette & Moreno, 2010), physics (Craig, Gholson, Brittingham, Williams, & Shubeck, 2012), and physiology (Dunsworth & Atkinson, 2007). Researchers have also used virtual humans for facilitating learning with many different participant populations, including museum visitors (Lane et al., 2013), primary school students (Holmes, 2007; Hong, Chen, & Lan, 2014), post-secondary students (Kim, Baylor, & Shen, 2007; Veletsianos, 2010), and Amazon Mechanical Turk users (Twyford & Craig, 2017). Despite their broad implementation, research has shown that virtual humans can often aid the learning process (Schroeder et al., 2013), or at least not significantly impede learning (in some cases they have made no significant difference on learning outcomes, see Heidig & Clarebout, 2011). As such, they continue to garner the attention of instructional designers looking to add pedagogical supports to their learning environments.

A wide body of research investigates how people learn with or from agents, and the emphasis of much of this work focuses on how agents should be designed. Designing agents is a complex process, one which has been broken down into detailed levels by Domagk (2010) in her Pedagogical Agents – Levels of Design (PALD) model. The PALD model denotes that designers must consider three different levels of agent design, the detail level, the medium level, and the global level. This framework embraces design decisions ranging from the choice of an agent's gender through more encompassing questions such as should the agent appear as a human or not (Domagk, 2010). Heidig and Clarebout (2011) added to this framework with the Pedagogical Agents – Conditions of Use (PACU) model, which describes how designers should consider the agent's design, the qualities of the learners using the system, the role that the agent plays in the environment, the environment itself, and content to be learned. Using these frameworks one could characterize existing agent studies in order to better understand how to design an effective agent for their specific use case.

Researchers continue to explore agent design, often using social agency theory as their theoretical framework. Social agency theory suggests that agents can aid learning due to creating the feeling of a social interaction between the learner and the agent (Mayer, Sobko, & Mautone, 2003); a theoretical perspective which implies that the learner's perception of the interaction is of the utmost importance during the learning process. While researchers are beginning to understand how agents can be designed to facilitate learning, Schroeder, Romine, and Craig (2017) noted there is significantly less work investigating how agents are perceived and the impact of those perceptions on learning outcomes. Hence, key questions remaining in the literature relate to how people perceive virtual humans and to what extent these perceptions influence learning outcomes. Within these issues, one must also consider the measurement; how are we measuring how pedagogical agents are perceived, and do these measures have validity evidence? This work is essential for deepening our understanding of social agency theory.

In this study, we aim to expand the understanding of how agents are perceived by learners. To do this we examined the factor structure of a newly revised perception instrument designed specifically for use with agents. Next, we analyzed learners' perceptions of agents using cluster analysis, and finally examined the extent to which learners' perceptions of the agent influenced their learning outcomes.