Keywords

1 Introduction

Current generation virtual reality (VR) head-mounted displays (HMDs) use high-quality visual displays in order to stimulate the user’s visual sense for a variety of applications. Along the visuals, common VR applications also use spatial sound to increase the level of presence in certain situations such as in horror games. However, the tactile channel is largely neglected due to difficulties in finding a suitable solution that is acceptable in terms of aesthetics, price, and implementation complexity.

In a previous work we presented HapticHead [20, 22], a vibrotactile display around the head consisting of a bathing cap with a chin strap and a total of 20 vibrotactile actuators (see Fig. 1). We showed that our prototype can be used in 3D guidance and localization scenarios in both VR and augmented reality (AR) with relatively high precision and low task completion time. It has already been shown that different kinds of tactile feedback increase the level of presence in a VR scene or game [4, 5, 9]. The purpose of this work is to investigate the effect of a vibrotactile around-the-head HMD on the level of presence that users experience in a VR environment. In order to do this, we implemented two VR scenarios enhanced by vibrotactile feedback and compared them to no vibrotactile feedback in a between-groups comparison study.

2 Related Work

2.1 Tactile Displays

In the research area of tactile displays Paneels et al. [16] investigate tactile patterns on a bracelet for indicating directions and find that static patterns are not well recognized due to the actuators being too close and being recognized as one impulse instead of multiple impulses (funneling illusion) while dynamic patterns are recognized with higher accuracy.

ActiveBelt [8] is the first vibrotactile belt for directional navigation. Vibrotactile belts have also been used to increase the situational awareness of gamers [13] and for guiding visually impaired people [17].

Haptic Radar [10] is a ring around the head, consisting of multiple infrared sensors and vibrotactile actuators in order to give users a “spider sense” of approaching objects. A similar concept is Proximity Hat [19], which uses pressure instead of vibrotactile actuators and stimulates other receptors (Merkel disks). Kerdegari et al. [21] developed a firefighter helmet with seven vibrotactile actuators on the forehead. Their experiment shows lower route deviation times in a navigation task for vibrotactile compared to auditory feedback.

Israr et al. [15] present Tactile Brush, an interpolation concept for multiple tactile actuators arranged in a grid in order to purposefully generate a moving tactile funneling illusion, which simulates the feeling of a continuous motion with a single localization point even though multiple actuators are active at a time. Further work on the funneling illusion [18] shows that the spacing of actuators should be 2.5 cm or less on the forehead for the funneling illusion to occur for most users. HapticHead uses a spacing of at least 8 cm, so we do not expect the funneling illusion to arise. Myles et al. [12] investigate the vibrotactile sensitivity of different head regions and use a headband with 4 actuators to provide navigational cues to soldiers. They found that soldiers preferred a tactile to a visual or auditory display for directional cueing and that the forehead, frontal, parietal, and temple regions were most sensitive to tactile stimuli.

2.2 Immersion and Presence in Virtual Reality

In this paper we use the terms immersion and presence following the definition by Slater [6]. Immersion refers to the objective level of sensor fidelity a VR system provides, while presence refers to a user’s subjective psychological response to a VR system. Of course, the level of presence is somewhat correlated to the level of immersion a system can provide but also highly dependent on the concrete VR scene.

Pausch et al. [2] investigated the performance difference of a system with and without increased visual immersion in a search task scenario and found that users were faster in the immersive condition because they had a better spatial understanding of their virtual surroundings. Dinh et al. [4] evaluated the effect of haptic feedback (wind and heat) on user presence in VR. The effect of a vibrotactile vest and pants on user presence and collision detection in VR was explored by Ryu et al. in [7]. They found that their prototype did enhance the user’s sense of presence, especially when combined with spatial sound. Just like the system by Ryu et al., our system aims to increase the level of presence in VR. However, we decided to give vibrotactile feedback around the head as such feedback could easily be integrated into existing VR HMDs instead of requiring additional garments.

3 Prototype

Fig. 1.
figure 1

HapticHead prototype, to be combined with an HMD.

Full size image

Myles and Kalb [12] recommend actuators on the head to operate at frequencies between 32 and 150 Hz because of discomfort above that threshold. We decided to use actuators operating at 150 Hz at maximum because actuator size increases for equally strong impulses at lower frequencies.

Our prototype consists of a bathing cap with 17 vibration motors (Parallax, 12 mm coin type, 3.3 V, 90 mA, 9000 rpm) attached on the inside and distributed on the whole surface (Fig. 1). The non-stretchable chinstrap hosts another three vibration motors and can be adjusted to different head sizes using a Velcro fastener. The vibration motors are controlled by PWM signals of four Arduino Nanos on a switchboard, which are connected to a stationary PC through USB and are updated at 90 Hz. This prototype was first presented in [20], in which its performance for 3D guidance was evaluated.

On the software side, vibration motors are modeled at their corresponding position in a Unity scene [23]. This allows for easy spatial activation of selected motors, depending on the VR scene and task. The user’s head is tracked by the internal sensors of the HMD that is used in conjunction with HapticHead. In the experiment reported below we use an Oculus Rift CV1.

4 Experiment

We designed an experiment to measure possible effects of vibrotactile feedback in VR scenes on presence. Our initial hypothesis was that appropriate tactile feedback increases the level of presence in VR scenes. We chose a between-subjects study design where the experiment group experienced the VR scenes with tactile feedback and sound, while the control group experienced the same VR scenes with sound but without tactile feedback.

A total of 20 participants (6 female, mean age 24.1, SD 2.5 years) were invited and split up in equally sized experiment and control groups. 11 participants had previous experience with VR HMDs and applications. All participants filled out a mandatory informed consent form and optionally a photographic release form.

4.1 Measuring Presence

In a measurement of psycho-physiological parameters for presence (e.g., electromyography or galvanic skin response) Nacke and Lindley showed significant differences for different experiment conditions [11]. However, these significant differences could not be clearly correlated to participants’ responses in accompanying questionnaires, which means that the real causes and effects of those differences still need to be researched.

Instead of trying to measure presence in a quantitative psycho-physiological way, we decided to use a widely used qualitative method presented by Witmer et al. [3]. Their “Immersion Tendency Questionnaire” (ITQ) measures how likely a person is immersed by a movie or game in general and the “Presence Questionnaire” (PQ) measures the level of presence a person was experiencing in a previous experimental condition. This method requires a between-subjects study design for comparable results as all other effects need to be canceled out.

Witmer et al. evaluated their original ITQ and PQ questionnaires, identified and removed irrelevant questions which did not correlate with the overall result and the participants’ own assessment of the level of presence [3]. We used these reduced questionnaires and translated the remaining questions to German (the native language of the participants). We further rephrased all of them as statements in order to measure agreement with 7-point Likert scales instead of the proposed 7-point semantic differential scales. We used Likert scales because they are easier to answer (always strongly disagree to strongly agree as options) and often less vague depending on the adjectives used in semantic differential scales which can cause reliability issues [1]. We also added one statement to the PQ, letting participants state how well they were able to survey their environment due to haptic clues. Witmer et al. did not use any haptic stimulations in their experiments.

4.2 Virtual Reality Environments

Two VR environments were implemented as exemplary cases. These were augmented by tactile feedback.

Header Simulation. In our soccer header simulation (HS) four catapults and occasionally a plane above the user throw balls in the direction of the user’s head. The user’s position is right in the center between four goals.

Fig. 2.
figure 2

Soccer header simulation in VR.

Full size image

In the beginning of the 20-minute header simulation scene, only a few balls are thrown towards the user one by one. The task of the user is to head the ball into any of the goals. As time progresses the frequency of balls being thrown increases and it may also happen that multiple balls arrive at the same time. In the last third of the simulation planes appear occasionally and drop balls on top of the user in order to increase the challenge. Users can see a goal score indicator for further motivation (Fig. 2).

Participants can locate the balls visually and through spatial sound from the catapults when a new ball is being launched. In the vibrotactile feedback group participants also experience vibrotactile stimuli interpolated between the three closest motors in the direction of the ball. One stimulus is played when the ball is launched (100 ms) and another stimulus is played when the ball impacts on the participant’s head (200 ms).

Viking Village. In the viking village simulation (VV) we took one of Unity’s standard environments called “Viking Village” from the Unity asset store and enhanced it with visual, auditory, and vibrotactile rain and snow effects (Fig. 3).

Fig. 3.
figure 3

Enhanced viking village VR scene with different weather effects.

Full size image

The user experiences a 15-minute camera tour on a wooden wagon, hears effects such as rain (5 min), snow (5 min), and cracking flames. Participants of the experiment group also feel rain or snow as single droplets hit their head causing small vibrotactile stimuli at the nearest actuator position. A slightly less intense effect for snowflakes (100 ms, 75% of the maximum intensity) than for rain droplets (200 ms, 100% of the maximum intensity) is played. It is possible to play “catch the particle” by moving the head towards nearby rain droplets or snowflakes and then feel their impact at the corresponding position. Obviously, feeling impacts on facial areas is not correctly modeled as there are no actuators between forehead and chin.

4.3 Procedure

After arriving in our lab, participants were introduced to the Oculus Rift CV1. If they were part of the experiment group, they were also introduced to the HapticHead prototype. In addition, they were informed about the experimental procedure and filled out consent forms as mentioned above. Furthermore, they filled out the modified “Immersion Tendency Questionnaire” before starting with the first experimental condition.

Depending on whether the participants were in the experiment or control group, they experienced the soccer header VR scene, as explained above, with or without tactile feedback generated by the HapticHead prototype. After finishing the first VR scene there was a pause and participants were asked to fill out the modified PQ.

In the second part of the experiment, participants experienced the viking village VR scene, again with or without tactile feedback depending on whether they were in the experiment or control group and filled out another modified PQ. We further asked the participants to estimate how much time had passed after each VR scene because an increased level of presence can alter time perception [14]. Participants of the experiment group finally filled out another questionnaire on the usefulness and appropriateness of the tactile feedback.

5 Results

Results from the Likert scales in both, the ITQ and PQs were summed up per participant, resulting in the total ITQ and PQ scores per participant as intended by Witmer et al. [3]. Even though Likert scales are usually ordinal scales we chose to also report the mean and standard deviation due to large variances between participants and for comparability (Witmer et al. also chose to report mean and standard deviation in their work [3]).

The modified ITQ yielded a median total score of 79.5 (mean 78.6, SD = 12.6) for the control group and 91.5 (mean 87.7, SD = 12.9) for the experiment group. A Mann-Whitney U test reveals a slight tendency, but no statistically significant differences between the groups (p = 0.08 > 0.05).

After the first VR scenario (soccer header simulation), the modified PQ yielded a median total score of 71.5 (mean 69.7, SD = 14.1) for the control group and 91.0 (mean 87.7, SD = 14.3) for the experiment group. A Mann-Whitney U test reveals a statistically significant difference between the two groups (p < 0.01).

When asked about the total time they thought the 20-minute soccer header simulation took, the participants of the control group (mean 26.5 min) estimated almost the same time as the participants of the experiment group (mean 26.0 min).

The modified PQ after the second VR scenario (viking village simulation) yielded median total scores of 60.0 (mean 57.2, SD = 17.5) for the control group and 77.5 (mean 74.7, SD = 22.3) for the experiment group. A Mann-Whitney U test reveals a statistically significant difference between the two groups (p = 0.027 < 0.05).

When asked about the total time they thought the 15-minute viking village simulation took, again the participants of the control group (mean 18.8 min) estimated almost the same time as the participants of the experiment group (mean 19.0 min).

Fig. 4.
figure 4

Appropriateness of vibrotactile feedback

Full size image

The participants of the experiment group filled out a questionnaire on vibrotactile appropriateness and usefulness at the end of the experiment. Results are shown in Fig. 4. Interestingly, there are some statements with very mixed opinions between participants. As a common pattern, some participants rated the vibrotactile feedback negatively in most of the statements while others rated it positively in most statements which comes down to whether they liked it overall or not. Some participants who disliked the feedback also stated that they disliked the humming sounds of the vibration motors which were close to the ears. Eight of 10 participants of the experiment group agreed that such vibrotactile technology should be incorporated into other applications.

6 Discussion

The difference between the immersion tendency scores of the experiment and control groups is unfortunately close to being statistically significant. This indicates that the two groups were not perfectly balanced. Ideally, the ITQ-scores of both groups should be equal and the p-value should be larger. We suspect that the number of 10 participants per group was too small to cancel out outliers effectively. However, as the difference in ITQ-scores between the groups is not statistically significant, we can compare the results of the presence questionnaires directly.

Both presence questionnaires yielded a statistically significant difference between the two groups. This means that the vibrotactile feedback definitely had an effect on the level of presence users were experiencing in the VR scenes. The participants’ duration estimates of the scenes were not significantly different between groups, but both longer than the actual time the scenes took. We suppose that this is due to the scenes being rather long, which may have resulted in boredom and exhaustion.

Estimated duration values for the viking village scene were generally lower in both groups than for the soccer header scene. We suppose that this is caused by the much lower degree of interactivity of the viking village scene compared to the soccer header scene. The former was essentially just a camera tour through the scene and the participant could only move sideways a little and turn the head on the camera wagon. The scene was also quite long at 15 min, which led some participants to complain about being bored as to the low degree of interactivity. Furthermore, the vibrotactile augmentation did not make the weather effects feel more natural to the majority of the members of the experiment group. This indicates that the simulation itself could be improved.

7 Limitations

The relatively low number of 10 participants per group led to a tendency of the groups being almost statistically different in the ITQ results. More balanced groups would have been preferable.

The vibrotactile augmentation of the weather effects in the viking village scene did not lead to more a natural experience for the majority of the participants. Our concrete implementation lacked and could probably be improved using an iterative design approach for the weather effects.

8 Conclusion

The presented study shows that adding vibrotactile feedback around the head can significantly increase the level of presence a user experiences in certain VR scenes. This can be applied in immersive games as well as certain VR training simulations where the level of presence or spatial awareness (including collision prevention) is important such as in complex maintenance jobs, anxiety therapy, or flight training.

HapticHead is a relatively complex prototype with questionable aesthetics, but it could be integrated with future VR HMDs to enable 3D tactile feedback in VR environments. A greater vision of this vibrotactile prototype is to combine it with other tactile garments and produce a fully tactile and very immersive VR experience. This paper is also a first step in finding implications for the design of vibrotactile feedback in VR scenes such as what kind of vibrotactile feedback works best for the simulation of virtual rain. We will investigate these implications in future work.