Keywords

1 Introduction

Opportunities to telecommunicate are increasing. In telecommunication, such as consultations, conferences, and presentations, the information needs to be transmitted in the form of face-to-face (F2F) interaction. However, this system loses some nonverbal information. Various researchers are actively developing telepresence technology to convey more information that is nonverbal. Telepresence is a set of technologies that enable communication and interaction as if in a F2F interaction.

Previous telepresence researchers aimed at two goals. The first one conveys rich non-verbal information of an environment to a real perception operator, giving the feeling of being in a remote environment. For instance, TELESAR V (Telexistence Surrogate Anthropomorphic Robot) can allow operators to feel visual, audio, kinesthetic, and fingertip tactile sensation [1].

The second one is making an avatar with the presence of the operator at a remote location so that the opponents can have an illusion of talking in F2F environment. In order to convey the body information of the operator to a remote place, several researchers have proposed methods to imitate real people like Geminoid [2] or talking heads [3]. Previous studies attempted to imitate the F2F communication by conveying rich information.

2 Transcendent Telepresence

Several researchers in the telepresence field have focused on the method to convey rich nonverbal information, via telepresence system. However, the requirement in telepresence is not just presence but also to build good relationships and convince others. These demands in a one-to-many communication can be achieved by the efficient use of nonverbal language [4]. Some nonverbal information can be displayed consciously depending on the situation. However, some nonverbal information cannot be displayed by the human ability. In addition, it is difficult to use different kinds of nonverbal information as per ones preference. In order to use nonverbal information more effectively, it is necessary to improve the usage of nonverbal information beyond human ability.

Thus, we propose transcendent telepresence, which enhances nonverbal information that realizes telepresence beyond the F2F interaction. Since telepresence intervenes between the people, it should be able to extend the factors that are involved in the communication and achieve greater convenience than F2F interactions. Transcendent telepresence enhances nonverbal information, realizing remote communication with a psychological effect that cannot be achieved in F2F interaction (Fig. 1).

Fig. 1.
figure 1

Transcendent telepresence

Full size image

2.1 Comparison with Other Studies

Certain previous studies suggested that the telepresence system could allow humans to extend their telecommunication. Hollan et al. questioned this current trend and suggested that telepresence has the advantages in terms of anonymity and semi-synchronousness compared to F2F interaction [5]. As an example, they proposed asynchronous communication including chat system, anonymous communication, and automatic archive communication.

Transcendent telepresence also aims at the telepresence tools that people prefer to use even when they have the option of interacting in a physical proximity. However, transcendent telepresence concentrates on conveying nonverbal information as non-verbal, thus maintaining neutrality and presence. The importance of the method to convey nonverbal information in communication is yet to be mentioned. Human recognition becomes easier by enhancing nonverbal information as nonverbal cues, thus, causing a psychological and physiological effect while the opponents do not notice that nonverbal information is modified by the telepresence system.

2.2 Variations in Transcendent Telepresence

There are various approaches in the transcendent telepresence. Vargus had mentioned that a nonverbal medium could be classified into body, body movements, eye gaze, paralanguage, bodily contact, proxemics, time, and color [4]. Each of nonverbal mediums includes some factors. For example, the body movements include movements’ type, speed, and timing. In addition, nonverbal information is interconnected. For example, eye contact affects the perceived emotion expressed by the face [6]. This suggests there are many variations in transcendent telepresence.

There are other methods to classify transcendent telepresence. One of them is to complement the individual differences and the other is to add functions that humans do not possess. In order to complement the individual differences, we researched the enhancement impression in social touch by changing the gender [7]. In addition, Smart Face improves creativity performance by changing the facial appearance of people during video conferences [8], and PoliTel [9] automatic adjusts the interpersonal distance.

In the next chapter, as an example for the additional function, we introduce the eye avatar, which allows speakers to make eye contact with various listeners at the same time (Table 1).

Table 1. Variation of transcendent telepresence
Full size table

3 Eye Avatars: The Expansion of the Gaze Cone

3.1 Previous Research About Gaze

We propose the eye avatar, which allows the speakers to change the directivity of the eye gaze in a one-to-many communication. One-to-many communication contains highly asymmetrical dialogues such as speech, storytelling, and presentation, in which one person speaks virtually while the others are just listeners during the whole communication. Hence, the speakers and the listeners get information of each other that is different from the audio information by using nonverbal information like gaze, face expression, gestures [10]. Among these, gazes are important and useful for impression formation, adjustment of the conversation and so on.

We will consider the eye gaze effect especially in one-to-many communication. The eye gaze can be classified into direct gaze and averted gaze. Direct gaze, also called as eye gaze or face gaze, is a gaze where the direction of gaze is at another’s face or eyes. The studies on gaze suggest that long direct gazes give a good influence on the evaluations of liking and attraction, attentiveness, competence, social skills and mental health, credibility, and dominance [11]. In class, a teacher’s longer direct gaze gives a good impact on the students’ memory [12]. Long mutual gaze also induces the listeners’ frequent and fast responses such as nodding [13].

On the other hand, an averted gaze is a gaze where the direction is not at anyone’s face or eye. Averted gaze has a gaze cuing function that can convey the direction of attention and the special cue [14]. It produces the observers’ reflexive attention shifts in response to the observed eye gaze direction [15], as well as gaze-following behavior such as joint attention [16].

The duration of the direct gaze influences these effects. In order to use an effective gaze in a one-to-many communication, you have to send long direct gaze along with a mutual gaze to every listener. However, as the number of listeners increases, the time in the mutual gaze with each listener reduces and the effect of gaze decreases. The more listeners, the more difficult it becomes to send a suitable amount of direct gaze to every listener. Thus, we apply the concept of transcendent telepresence to conquest this problem, and propose a system that can have a longer eye contact, having a wide gaze cone. Gaze cone [17] is the range of gaze directions within which a person feels that he/she is being watched. To send direct gaze and averted gaze effectively in one-to-many communication, we propose an eye avatar, which can send a direct gaze to many people at the same time. We made a broad gaze cone eye by using the depth inversion.

3.1.1 Implementation

We made an eye avatar that has a broad gaze cone to utilize gaze effect and to facilitate one-to-many communication. The eye avatar controls the gaze cone by switching the eye shape between convex and hollow. The hollow eyes are illusion eyes that have a broad gaze cone, applied with depth inversion [18].

Figure 2 shows the appearance of the eye avatar. The eyeball’s pupil and iris were painted black and sclera white for high contrast that is easy to see even from a distant place. The eyeball’s diameter was 40 mm. The rotation eye had the mechanism that permitted two degrees of freedom: pan and tilt. The tilt rotation for 180 degrees is a change of convex and hollow. In order to make the observer feel the eye changing the gaze direction, it is necessary to make the switching eye shape, which is invisible to the observer.

Fig. 2.
figure 2

Eye avatar’s hollow eye and convex eye

Full size image

When a brief blank is shown during the alternating displays, a striking failure of the perception is induced. Identification of the changes becomes extremely difficult even when changes are large and made repeatedly. In perceptual psychology, it is known as “change blindness” [19]. We applied the change blindness while switching the eye shape in order to make it inconspicuous. The eyelid hides the eye when the eye avatar is changing from convex and hollow eye, thus imitating the blink. Additionally, we used matte spray and removed the gloss for the hollow eye to make it indistinguishable from the convex eye.

3.2 Evaluation Experiment

Two evaluation experiments were conducted to check whether the eye avatar could use the effect of direct gaze and averted gaze effectively. First, to evaluate convex and hollow eye’s direct gaze effect, the area of the gaze cone of each eye was measured. Second, in order to evaluate each eye’s averted gaze effect we measured how correctly they can convey the gaze direction.

3.2.1 Measurement of the Gaze Cone of the Convex Eye and Hollow Eye

Experimental Procedure.

The research participants watched the eye avatar from a point that was 1.5 m from the side of the eye avatar, for every 10 degrees and were asked if they felt that they were being watched. This was conducted for the both convex eye and hollow eye.

  1. 1.

    A research participant was seated in a chair in front of the eye avatar, attached to a tripod (Fig. 3).

    Fig. 3.
    figure 3

    Top view of experiment1 environment

    Full size image
  2. 2.

    The height of avatar was adjusted to participant’s eye level. The eyeball’s height and angle were fixed after the adjustment.

  3. 3.

    The participant looked at the avatar and answered whether they felt like being watched in three-step evaluation; watched at by right and left eyes, only one eye and none.

  4. 4.

    After the answer, the participant looked away from the avatar once. He/she then shifted to the next position in 10 degrees counter-clockwise and answered in the same way.

  5. 5.

    The participant repeated steps 3 and 4 until he/she answers that they do not feel watched by both eyes. After that, the participant returns to the point in front of the eyeball avatar.

  6. 6.

    Similarly, the participant repeats steps 3 and 4 to a clockwise direction.

Result.

The average of the gaze cone of the convex eye was 32.3 degrees and the hollow eye was 3.4 degrees. Welch’s paired t-test showed that the gaze cone of the convex eye and the hollow eye were significantly different (p = 0.012) (Fig. 4).

Fig. 4.
figure 4

Gaze cone angle of convex and hollow eyes

Full size image

In addition, the number of participants that had the feeling of being watched decreased from the point of 50 degrees. According to the participants, it was because the servomotor hid the eye avatar’s iris and the edge of the hollow eye from the position over 50 degrees. None of the participants answered the eye avatar watching him/her at 60 degrees or more.

3.2.2 Accuracy of Conveyance Gaze Direction

Experimental Procedure.

We measured how accurately the convex and hollow eye can convey direction of gaze and how iszt received to examine whether their averted gaze is effective.

Research participants were male in the age group of 21 to 25 years. We projected the alphabet target behind the participant (Figs. 5 and 6) using a projector. The intervals of the targets are at 500 mm. The experimenter sent the gaze direction to the alphabet target point projected on the screen using the eye avatar. The front of the eye avatar was the center of the projection, which is the target between H and M.

Fig. 5.
figure 5

Experiment photography

Full size image
Fig. 6.
figure 6

Top and side view of the experiment environment

Full size image

First, an experimenter sent a gaze to every target using convex eyes; then, gaze using the hollow eye was sent in the same way. The pointed alphabet targets’ order was sorted randomly for each participant. Next, the research participants answered the point that they felt the eye avatar watched. They were also allowed to answer the middle point such as between A and B or themselves. After the experiment, the participants answered a questionnaire about what they felt about experiment and eye avatar. If the participants answered as looking at, Let the point at which a line from eye to participant’s eye intersects with the screen to be position coordinate of the answer (Fig. 6).

Results.

The sum of the error was calculated as the distance between the gaze direction of the eye avatar and the participants who felt they were being watched by the eye avatar. Next, we calculated the average of every target’s error for convex eye and hollow eye. The sum of the error for the convex eye was 6995 mm and the hollow eye was 18048 mm.

Welch’s t-test showed that the average of error distance of the convex eye was shorter than the hollow eye (p < 0.01) (Fig. 7).

Fig. 7.
figure 7

Average of every target’s error for convex eye and hollow eye

Full size image

4 Discussion

The result of the measurement of the gaze cone shows that hollow eyes have a broader gaze cone than the convex eye. Thus, it is possible to widen the gaze cone by using hollow eyes (Fig. 8). In this experiment, we measured gaze cone area only horizontally. It is presumed that the hollow eyes have similar effects vertically because eyes are hemispherical. In addition, using the results obtained from the accuracy check of conveyance gaze direction, the convex eye’s gaze can be perceived relatively correctly, while for the hollow eyes it cannot be perceived. Since the hollow eye has a broader gaze cone but the conveyance accuracy is lower than the convex eye, it is expected that we can use the direct gaze and averted gaze effect by using them selectively.

Fig. 8.
figure 8

Eye avatar’s gaze cone angle

Full size image

5 Conclusion

We proposed a novel concept, “transcendent telepresence,” which enhances non-verbal information. It complements the individual difference to human or adds new functions to human. Transcendent telepresence can be applied to many kinds of non-verbal information. As an example of the concept of transcendent telepresence, we introduced “eye avatar,” which can expand the gaze cone.

We evaluated the range of the gaze cones of convex and hollow eyes and the accuracy of conveying gaze direction. The result showed that the hollow eye has a broader gaze cone but the accuracy of conveying gaze direction is lower than the convex eye. By switching the eye avatar’s hollow eye and convex eye, we can control the range of gaze cone and the accuracy of conveyance. It is expected that the eye avatar can enhance the impression of the speaker or improve students’ memory in certain important contents.

Future topic in terms of the eye avatar is to implement and evaluate the system to realize the proposed method introduced here. For example, we will evaluate the psychological effect of the hollow eyes such as inducement to listeners’ responses. We will research about it by using eye avatar in one-to-many communications such as actual lectures and presentations.

As a whole, we can further approach the research on transcendent telepresence in various ways. For example, we can develop a new transcendent telepresence with other nonverbal information such as facial expressions and body language. Furthermore, we can use transcendent telepresence systems in combination rather than individually. By combining the effects of the various types of nonverbal information, a human would be able to use their nonverbal communication ability as intended. We believe that transcendent telepresence will enable users to exceed human’s limit and realize that telepresence systems are ideal for users rather than F2F interaction.