How Much Do We Perceive Geometric Features, Personalities and Emotions in Avatars?

Abstract

The goal of this paper is to evaluate the human perception regarding geometric features, personalities and emotions in avatars. To achieve this, we used a dataset that contains pedestrian tracking files captured in spontaneous videos which are visualized as identical virtual human beings. The main objective is to focus on individuals motion, not having the distraction of other features. In addition to tracking files containing pedestrian positions, the dataset also contains emotion and personality data for each pedestrian, detected through computer vision and pattern recognition techniques. We are interested in evaluating whether participants can perceive geometric features such as density levels, distances, angular variations and speeds, as well as cultural features (emotions and personality traits) in short video sequences (scenes), when pedestrians are represented by avatars. With this aim in mind, we propose two questions to be answered through this analysis: i) “Can people perceive geometric features in avatars?"; and ii) “Can people perceive differences regarding personalities and emotions in virtual humans without body and facial expressions?". Regarding the participants, 73 people volunteered for the experiment in order to answer the two mentioned questions. Results indicate that, even without explaining to the participants the concepts of cultural features and how they were calculated (considering the geometric features), in most cases the participants perceived the personality and emotion expressed by avatars, even without faces and body expressions.

A Appendix

1.1 A.1 Visualization of Geometric Features

Our viewer was developed using Unity3D³ engine, with $C\#$ programming language. The viewer allows users to rewind, accelerate and stop the scene through a time controller and can focus on something interesting multiple times at any time. Figure 1 shows the main window of the viewer. As identified in Fig. 1, the viewer is divided in five parts, as follows: 1) time controller, where is possible to start, stop and continue simulation playback; 2) ChangeScene and RestartCamPos buttons to respectively load data file of another video and restart the camera position for first person view; 3) a window that shows the top view of the environment; 4) first-person view of a previously selected agent (this agent is highlighted in area 3); and 5) which contains the resource panel, where users can enable the visualization of data related to agents’ emotion, socialization and collectivity.

This viewer has three modes of visualization: (i) first-person camera, (ii) top camera, and (iii) an oblique camera. Figure 2 shows an example of each type of camera viewpoint in a video available in Cultural Crowds dataset. In addition to these different viewpoints, it is possible to observe all pedestrians present at each frame f. Pedestrians are represented by a humanoid or cylinder type avatar. Each pedestrian i present in frame f has a position ($X_i,Y_i$) (already converted from image coordinates to world coordinates). In addition to positions, it is also possible to know if the pedestrian is walking, running or stopped in frame f through the current speed $s_i$. If in a certain frame the current speed is greater than or equal to $\frac{0.08m}{f}$, which is equivalent to $\frac{2m}{s}$, considering $\frac{24f}{s}$, then the avatar is running. It was defined based on the Preferred Transition Speed PTS [2] to change from walk to run, for instance. The values of the transitions can be seen in Eq. 1, considering the current speed of the agent $s_i$.

$$\begin{aligned} Animation = \left\{ \begin{array}{ll} \mathbf{Idle} , &{} \text{ when } s_i == 0; \\ \mathbf{Walk} , &{} \text{ when } 0< s_i < \frac{0.08m}{f}; \\ \mathbf{Run} , &{} \text{ when } s_i \ge \frac{0.08m}{f}. \end{array} \right. \end{aligned}$$

(1)

Also, for the humanoid avatar type, each speed transition is accompanied by an animation transition, for example, if the current speed $s_i = 0$, then it does not change animation (remaining stationary), but if its speed is $0< s_i < \frac{0.08m}{f}$, then the animation changes for walking as well as if $s_i \ge \frac{0.08m}{f}$, the animation of avatar changes to running.

Fig. 1.

Main window of the viewer.

Fig. 2.

Types of visualization: (a) top view, (b) oblique and (c) first-person view.

1.2 A.2 Table of Scenes

The Table 1 shows all scenes used in all questions in survey.

1.3 A.3 Questions

The Questionnaire. We formulated a questionnaire to evaluate human perceptions, divided into two stages: i) The first aims to assess whether participants can perceive geometric features in avatars and from different points of view; and ii) the second aims to assess whether participants can perceive emotions and personality traits geometrically, extracted in real videos and represented in avatars. Before the questionnaire, we notified the participants that they could withdraw at any time and for any reason. We do not collect personal data. Before each question, participants saw two to three of the scenes (shown in Table 1) referring to geometric or cultural feature presented in the question. No explanation of the content of the survey was provided to try to avoid bias. In addition, we used Google Forms to apply the questionnaire on social networks, so all participants were volunteers. Next sections described the evaluated aspects.

Table 1. Walls were added in scenes 4 and 9 to evaluate density perception with and without walls. In addition, this table also shows the camera and avatar type of each scene, the video identification, and the density level.

Geometric Features. The first part of the questionnaire contains twenty-two questions, as shown in Table 2, six related to density, six to speed, five to angular variation and five to distance. Table 2 also presents the possible answers, and the right ones highlighted in bold. In all density questions (Table 2) we asked in which of the short sequences the participant observed the highest density level. The first question (D1) represented a control question, i.e., we wanted to assess whether participants could perceive density variation: low, medium and high density video scenes of pedestrians in crowds. Questions D2 and D3 aimed to assess whether different points of view (camera types) would influence participants’ density perception. Our goal was to assess whether density perception would change due to the camera’s point of view and the way agents were displayed. As shown in Table 2, both questions D2 and D3 scenes were applied after same scene, but displayed with different points of view, and different avatars. Before question D4, two scenes are presented with same density and same point of view, but changing avatar type. Those evaluations aimed to assess whether different types of avatars and point of view would influence density perception. In D5 and D6, walls were added around scenes (4 and 9). D5 had the same objective as D1, i.e., control question. D6 aimed to assess whether walls would influence density perception using first person camera.

Table 2. All questions related to geometric features and possible answers. Right answers highlighted in bold.

Regarding speed perception, all questions had low density. The purpose of these questions was to assess whether participants were able to perceive different speed levels (presented in Eq. 1) in different points of view and types of avatars. The first-person camera was not evaluated for speed questions, as avatars’ first-person views had a lot of variation. The questions $S1-S4$ aimed to evaluate speed perception over points of view (top and oblique camera). S1 presented two scenes with "run" speed on oblique and top cameras. Same process for S2 but with the "walk" speed. Both S3 and S4 presented two scenes with both speed types. Finally, both S5 and S6 (which aimed to assess the influence of avatar on speed perception) presented two scenes containing two types of avatars using walk speed, one with top camera and one with oblique.

Fig. 3.

Initial (a) and final (b) frames from Scene15.

Regarding angular variation and distance, all questions used $BR-34$ (high density) video scenes. A1, A2, E1 and E2 aimed to assess whether points of view could influence perceptions of angular variation and distance. A1 and E1 presented three scenes with humanoids in three types of cameras. Similar process for A2 and E2, with cylinder type avatars. Finally, A3, A4, A5, E3, E4 and E5 presented, both for angular variation and for distance, two scenes containing two different avatars with top, oblique and first person cameras. These questions aimed to assess whether the types of avatars would influence perceptions.

Table 3. Cultural questions and answers. Correct answers are highlighted in bold, according to Favaretto et al. [13].

Table 4. In addition, the table also presents general averages for each question category: density (with and without control questions), speed, angular variation, distance, all geometric features (with and without density control questions), and all cultural features.

Cultural Features. The second stage of the questionnaire contains seven questions related to emotions and personality traits. In all scenes presented, two avatars of different colors (red and yellow) were highlighted to be perceptual focus questions, as illustrated in Fig. 3. All questions related to cultural features with their answers (right answers are highlighted in bold) are presented in Table 3. We used as ground truth the results obtained by the approach proposed by Favaretto et al. [13]. In the example of Fig. 3, the initial and final frames of scene15 (shown in Table 1) are shown, where there is a group of pedestrians (represented by avatars) on the right part of the scene. The avatar highlighted in yellow is part of this group and the avatar highlighted in red walk trough the group with a higher speed.

This scene is related to Q1 and Q2, questions that questioned which avatars (yellow or red) were, respectively, neurotic and angry. Q3 and Q4, related to scene10, aimed to assess, respectively, whether the participants would perceive which highlighted avatar was open to new experiences and which one was afraid. Scene10 shows a yellow highlighted avatar interacting with a group of avatars and a red highlighted avatar standing alone with no interaction. Finally, Q5, Q6 and Q7 aimed to assess happiness, extraversion and sociability. Q7 was proposed after analyzing the results of question Q6, so it is explained in Sect. 4.2. These questions are related to scene16, which contains a yellow highlighted avatar walking with a group of avatars and a red highlighted avatar walking alone, in the opposite direction to all other avatars.

1.4 A.4 Table of Results

The Table 4 presents the averages of right answers for each question of our work.