A personality-based emotional model for embodied conversational agents: Effects on perceived social presence and game experience of users

Highlights

• Agent’s nonverbal behaviors can be governed by a personality-driven emotional model.

• These behaviors affect the perceived social presence of users.

• Pronounced, assertive, and noticeable behaviors evoke a higher effect.

• Metrics of game experience are significant predictors of social presence.

• Flow is the strongest predictor of social presence.

Abstract

This paper reports on an experiment that investigates the effect of interacting with a personality-driven embodied conversational agent (ECA) on the perceived social presence and game experience of people in a VR social simulator. Furthermore, the dynamics between the different metrics of game experience and social presence of people are explored to determine which game experience metrics are the strongest predictors of perceived social presence in this context. A personality-based emotional model is used for personifying the employed ECA, which governs the manifestation of its non-verbal behaviors. Three experimental conditions manipulating the existence and intensity of non-verbal behaviors exhibited by the ECA were used to investigate the effect of this proposed approach. The results of the experiment with 41 participants indicate that people who were exposed to an extrovert ECA experienced significantly higher levels of behavioral involvement as part of their social presence compared to the other conditions. These results suggest that incorporating personality by means of non-verbal behavior in the emotional model of an ECA influences users perceived feeling of social presence. Furthermore, our results reveal that there is a bidirectional relationship between game experience metrics and perceived social presence of people as each predict the other.

Introduction

Over the past years we have witnessed an increasing interest in the use of interactive and immersive applications such as games [1] and simulations for serious purposes (e.g. social skill training [2], physical rehabilitation [3], logical problem solving [4]). In contexts where social interactions are considered as an important essence of learning, social entities generally referred to as embodied conversational agents (ECA) [5] capable of imitating the behavior of real people are crafted and utilized by these systems. ECAs facilitate social interactions to take place with users through verbal, non- and para-verbal behavioral cues (e.g. [6]). As learning in these systems occur through socialization with an ECA, perceiving the social presence (a feeling of sensing another entity being present) of the interlocutor (i.e. the ECA) is of key importance. Research has shown that there is a correlation between the perceived social presence and learning where higher levels of perceived social presence in online education lead to higher levels of perceived learning and satisfaction with the instructor [7]. Social presence can be studied in light of constructs such as immediacy (psychological distance) and intimacy (interpersonal closeness) [8]. These experiences are highly dependent on the degree and quality of verbal, non-verbal, and para-verbal behaviors demonstrated by the ECA, as well as the game/simulation experiences of users (e.g. immersion, flow, positive and negative affects). These ECA behaviors provide unobtrusive, and yet extremely expressive feedback to the users about the effect of their actions [9]. Such feedback could evoke a sense of empathy towards the ECA, elicit a sense of behavioral dependency, and drive the learners to adjust their actions. In educational contexts, with an objective of behavior adjustment or improvement, this is highly desired and at the core of the learning strategy.

Extensive argumentation can be found in the literature highlighting the importance of good game experience, particularly in light of metrics such as immersion and flow for effective learning (e.g. [10], [11], [12]). Furthermore, it has been suggested that the immersiveness of an experience actually provides the boundaries within which presence (the feeling of being there) can occur [13]. As such, in serious games and social simulations where learners have to actively engage in social interaction with ECAs, it is highly desirable to evoke both: good game experience and high levels of perceived social presence to facilitate effective learning. In light of this, we have opted to investigate whether the perceived social presence and game experience of people can be positively affected in a social simulation environment perceived through virtual reality (VR), by employing an ECA that expresses its emotional states through non-verbal behavioral cues governed by its personality. Similar research works have investigated the effect of personified ECAs with different characteristics on the perceived social presence of people. However, investigating the effect of manipulating the behavioral model of an ECA based on personality, and its consecutive influence on social presence and game experience has not been sufficiently addressed. Furthermore, although the literature suggests that immersion is an influential metric that informs the elicitation of presence, it is not clear whether the same principle applies for social presence. As such, it is important to understand what game experience metrics are the strongest contributing factors to the feeling of perceived social presence. Such insight can inform and improve the design of future virtual social experiences.

On these grounds, this paper introduces a multi-modal prototype that utilizes a personality-driven ECA equipped with verbal and non-verbal behaviors. Moreover, it presents the results of a pilot study investigating its effects on the perceived social presence and game experience of users, as well as the dynamics between them. In a pilot study with 41 participants, the existence and intensity of the non-verbal behaviors of the used ECA was manipulated in three conditions to assess its effect on the reported game experience and perceived social presence of users: (1) no apparent non-verbal behaviors exhibited by the ECA (2) non-verbal behaviors governed by an extrovert-based emotional model, in which the exhibitions by the ECA are more assertive and pronounced (3) non-verbal behaviors governed by an introvert-based emotional model, in which the exhibitions are more submissive and minimal. Our results suggest that the projection of non-verbal behaviors as a result of incorporating personality as part of the emotional model of an ECA, influence the elicitation of the feeling of social presence from the users. More precisely, the results suggest that more assertive and pronounced non-verbal behaviors produced by an ECA are more noticeable by users and seem to have higher impacts on their feeling of social presence, compared to more submissive and minimal non-verbal behaviors. Furthermore, our results show that among the measured game experience metrics, the experienced levels of flow, immersion, and tension are the strongest predictors for the perceived feeling of social presence with flow as the strongest predictor. These results suggest that the aesthetics and richness of the experience, in addition to minimal experienced levels of tension (although significantly influential), are surpassed by a feeling of losing track of time and being occupied and engaged with activities involving interaction with a human-like entity in relation to experiencing social presence.

The rest of this paper is organized as follows: Section 2 discusses related works on the effect of computer-mediated artifacts (mostly with a focus on ECAs) on perceived social presence, as well as the role of game experience in effective learning. Section 3 discusses the underlying model of the used ECA, and Section 4 elaborates on the used VR simulation environment. Section 5 discusses the experiment with the prototype, and Section 6 concludes the paper while addressing limitations and future work.