Welcome to the proceedings of the 21st ACM International Conference on Intelligent Virtual Agents (IVA 2021). IVA is an annual interdisciplinary conference in which leading scientists worldwide present and discuss their latest work on socially intelligent virtual agents (IVAs). These agents are intelligent virtual characters or robots which humans can interact with. They often have anthropomorphic elements to evoke responses that humans would exhibit when interacting with other humans. For this, agents use natural human communication modalities such as speech, gestures, and facial expressions. Besides these interaction modalities, the agents are capable of real-time perception, artificial cognition, and actions in the social environment they operate in. Constructing and studying these IVAs requires knowledge, theories, methods, and tools from a wide range of fields such as computer science, psychology, cognitive sciences, communication, linguistics, interactive media, human-computer interaction, and artificial intelligence. All these disciplines are represented in these proceedings.
EEG Model: Emotional Episode Generation for Social Sharing of Emotions
Social sharing of emotions (SSE) occurs when one communicates their feelings and reactions to a certain event in the course of a social interaction. The phenomenon is part of our social fabric and plays an important role in creating empathetic responses ...
Social Signals of Cohesion in Multi-party Interactions
Group conversation is a frequently used form of communication for exchanging ideas and making decisions. Cohesion is an emergent phenomenon that describes the members' attraction towards the group and towards working together. In this paper, we present ...
Agents United: An Open Platform for Multi-Agent Conversational Systems
- Tessa Beinema,
- Daniel Davison,
- Dennis Reidsma,
- Oresti Banos,
- Merijn Bruijnes,
- Brice Donval,
- Álvaro Fides Valero,
- Dirk Heylen,
- Dennis Hofs,
- Gerwin Huizing,
- Reshamashree B. Kantharaju,
- Randy Klaassen,
- Jan Kolkmeier,
- Kostas Konsolakis,
- Alison Pease,
- Catherine Pelachaud,
- Donatella Simonetti,
- Mark Snaith,
- Vicente Traver,
- Jorien van Loon,
- Jacky Visser,
- Marcel Weusthof,
- Fajrian Yunus,
- Hermie Hermens,
- Harm op den Akker
The development of applications with intelligent virtual agents (IVA) often comes with integration of multiple complex components. In this article we present the Agents United Platform: an open source platform that researchers and developers can use as ...
Diversity Informatics: Reducing Racial and Gender Bias with Virtual Agents
Job advertisements in white male-dominated organizations are often biased in ways that discourage female and minority candidates from applying. We explored the use of a female African American virtual agent who provides a first-person reaction to a ...
Being Guided or Having Exploratory Freedom: User Preferences of a Virtual Agent's Behavior in a Museum
A virtual guide in an immersive virtual environment allows users a structured experience without missing critical information. However, although being in an interactive medium, the user is only a passive listener, while the embodied conversational agent ...
Enhancing Conversational Agents with Empathic Abilities
Conversational agents are getting increasingly popular and find applications in health and customer services. Conversations in these fields are often emotionally charged. It is, therefore, necessary to handle the conversation with some degree of empathy ...
Factors associated with acceptance of a virtual companion providing screening and advices for sleep problems during COVID-19 crisis
The COVID-19 crisis has generated an increase of sleep problems in the general population. Digital technologies can help dealing with mental health repercussions of COVID-19 but their acceptance by the population need to be better understood. KANOPEE is ...
Design and Evaluation of Virtual Human Mediated Tasks for Assessment of Depression and Anxiety
- Joy O. Egede,
- Dominic Price,
- Deepa B. Krishnan,
- Shashank Jaiswal,
- Natasha Elliott,
- Richard Morriss,
- Maria J. Galvez Trigo,
- Neil Nixon,
- Peter Liddle,
- Christopher Greenhalgh,
- Michel Valstar
Virtual human technologies are now being widely explored as therapy tools for mental health disorders including depression and anxiety. These technologies leverage the ability of the virtual agents to engage in naturalistic social interactions with a ...
Once Upon a Story: Can a Creative Storyteller Robot Stimulate Creativity in Children?
Creativity is a vital inherent human trait. In an attempt to stimulate children's creativity, we present the design and evaluation of an interaction between a child and a social robot in a storytelling context. Using a software interface, children were ...
Effects of Virtual Humans' Gender and Spoken Accent on Users' Perceptions of Expertise in Mental Wellness Conversations
In the context of mental wellness support, trust and intimacy between a counselor and a patient are necessary to converge healing processes positively. However, convincing students to trust a virtual human for topics regarding mental wellness is a ...

Human or Robot?: Investigating voice, appearance and gesture motion realism of conversational social agents
Research on creation of virtual humans enables increasing automatization of their behavior, including synthesis of verbal and nonverbal behavior. As the achievable realism of different aspects of agent design evolves asynchronously, it is important to ...
Questionnaire Items for Evaluating Artificial Social Agents - Expert Generated, Content Validated and Reliability Analysed
In this paper, we report on the multi-year Intelligent Virtual Agents (IVA) community effort, involving more than 90 researchers worldwide, researching the IVA community interests and practice in evaluating human interaction with an artificial social ...
Virtual Reality in Sexual Harassment Prevention: Proof-of-Concept Study
- Shiri Sadeh-Sharvit,
- Jonathan Giron,
- Shir Fridman,
- Maxine Hanrieder,
- Shany Goldstein,
- Doron Friedman,
- Shir Brokman
Sexual harassment (SH) training is a major public health priority, and available programs show limited efficacy. We present a new application for virtual agents in virtual reality (VR), which has the potential to deliver SH training that imitates real ...
Using Intelligent Agents to Examine Gender in Negotiations
Women earn less than men in technical fields. Competing theories have been offered to explain this disparity. Some argue that women underperform in negotiating their salary, in-part due to language in job descriptions, called gender triggers, which ...

Learning Speech-driven 3D Conversational Gestures from Video
- Ikhsanul Habibie,
- Weipeng Xu,
- Dushyant Mehta,
- Lingjie Liu,
- Hans-Peter Seidel,
- Gerard Pons-Moll,
- Mohamed Elgharib,
- Christian Theobalt
We propose the first approach to synthesize the synchronous 3D conversational body and hand gestures, as well as 3D face and head animations, of a virtual character from speech input. Our algorithm uses a CNN architecture that leverages the inherent ...
Introducing VHMason: A Visual, Integrated, Multimodal Virtual Human Authoring Tool
A major impediment to the success of virtual agents is the inability of non-technical experts to easily author content. To address this barrier we present VHMason, a multimodal authoring tool designed to help creative authors build embodied ...
Designing an Adaptive Embodied Conversational Agent for Health Literacy: a User Study
- Joy Egede,
- Maria J. Galvez Trigo,
- Adrian Hazzard,
- Martin Porcheron,
- Edgar Bodiaj,
- Joel E. Fischer,
- Chris Greenhalgh,
- Michel Valstar
Access to healthcare advice is crucial to promote healthy societies. Many factors shape how access might be constrained, such as economic status, education or, as the COVID-19 pandemic has shown, remote consultations with health practitioners. Our work ...
Linking Theory of Mind in Human-Agent Interactions to Validated Evaluations: Can Explicit Questionnaires Measure Implicit Behaviour?
There is a new crisis emerging in human-agent interaction research: Instead of using validated questionnaires, individual studies are creating new questionnaires that claim to measure identical constructs. This makes replication studies and comparisons ...
Designing Personality Shifting Agent for Speech Recognition Failure
This paper proposes a method to shift an agent's personality during speech interaction to reduce users' negative impressions of speech recognition systems when speech recognition fails. Speech recognition failure makes users uncomfortable, and the ...
Multimodal and Multitask Approach to Listener's Backchannel Prediction: Can Prediction of Turn-changing and Turn-management Willingness Improve Backchannel Modeling?
The listener's backchannel has the important function of encouraging a current speaker to hold their turn and continue to speak, which enables smooth conversation. The listener monitors the speaker's turn-management (a.k.a. speaking and listening) ...
Comparing The Accuracy of Frequentist and Bayesian Models in Human-Agent Negotiation
Understanding an opponent's wants is crucial for maximizing the outcomes of a multi-issue negotiation. To do this, automated systems must build an "opponent model" from information conveyed during a negotiation. Bayesian and frequentist models are the ...

Speech2Properties2Gestures: Gesture-Property Prediction as a Tool for Generating Representational Gestures from Speech
We propose a new framework for gesture generation, aiming to allow data-driven approaches to produce more semantically rich gestures. Our approach first predicts whether to gesture, followed by a prediction of the gesture properties. Those properties ...
Pandemic Panic: The Effect of Disaster-Related Stress on Negotiation Outcomes
Prior research often finds increased altruism following natural disasters. One explanation is the social heuristic hypothesis: humans are prosocial by nature but become self-interested when they have the opportunity to deliberate. As the stress of a ...
A Friendly Face in the Crowd: Reducing Public Speaking Anxiety with an Emotional Support Agent in the Audience
We present Friendly Face - a virtual agent designed to reduce public speaking anxiety by standing within an audience. The agent senses the speaker's behavior during an oral presentation and provides emotional and instrumental support. The system ...
Reactive Virtual Agents: A Viewpoint-Driven Approach for Bodily Nonverbal Communication
- Pierre Raimbaud,
- Alberto Jovane,
- Katja Zibrek,
- Claudio Pacchierotti,
- Marc Christie,
- Ludovic Hoyet,
- Julien Pettré,
- Anne-Hélène Olivier
Non-verbal communication body cues are paramount to interact. In this preliminary work, we explore ways to let Intelligent Virtual Agents (IVAs) simulating nonverbal communication capabilities. We propose an approach to control IVAs' reactive behaviour ...
Do you mind if I ask?: Addressing the cold start problem in personalised relational agent conversation
To personalise dialogue to different users, relational agents need to learn about the users' preferences for relational cues used by the agent. In the context of a virtual advisor to reduce students' study stress, we designed a between subjects study ...
Task Allocation in Multi-Agent Systems with Grammar-Based Evolution
This paper presents a grammar-based evolutionary model to facilitate autonomous emergence of task allocation for intelligent multi-agent systems. The approach adopts a context-free grammar to determine the behaviour rule syntax. This allows for ...
INSHA: Intelligent Nudging System for Hand Hygiene Awareness
Maintaining hand hygiene is the one of the most effective way to prevent the spread of germs during a pandemic. This paper focuses on encouraging people to use a hand sanitizer more frequently by applying the nudge theory to improve hand hygiene ...
Someone or Something to Play With?: An Empirical Study on how Parents Evaluate the Social Appropriateness of Interactions Between Children and Differently Embodied Artificial Interaction Partners
Children are raised with technologies that are able to respond to them in natural language. This not only makes it easy to communicate but also to connect with them socially. While communication abilities might have benefits (e.g., for learning), it ...
Cited By
- Proceedings of the 21st ACM International Conference on Intelligent Virtual Agents