Keywords

1 Introduction

Virtual reality has long been pursued to impart a “sense of reality” by generating various stimulations artificially. Recently it has been argued that the incorporation of emotional feeling is also effective for enhancing an immersive virtual experience. An emotional state is considered to be ambiguous; however, in the cognitive field, it is known that emotion can be evoked as a result of physiological changes in the body induced by the autonomic nervous system [1]. This theory endorses that emotions may be amplified or enhanced by a system that controls the physiological state externally. In the past, various technologies to generate artificial emotions have been proposed [2,3,4]. These previous studies added a sensory modality to deceive a user’s emotions through an external control. Specifically, Fukushima and Kajimoto argued that haptic displays are effective in altering emotions via somatosensory areas that represent emotional body images [5]. The EH-VR system is an interactive system that hacks one’s heartbeat and controls it to accelerate a scary VR experience. The system provides vibrotactile stimulus, which resembles a heartbeat, from the floor. It determines a false heartbeat frequency by detecting the user’s heart rate in real time and calculates a false heart rate, which is faster than the one observed. By providing the vibrotactile biofeedback of a false heartbeat while measuring the user’s heart rate in real time and adjusting the acceleration rate accordingly, we observed that the real heart rate synchronized with the false one more significantly than the step-wise acceleration rate in our previous experiment [6]. Based on this research outcome, we developed an EH-VR walk-through system that provides a virtual experience using vibrotactile biofeedback. We demonstrated the EH-VR system at emerging technologies at Siggraph Asia 2016 held in Macau, China [7]. Approximately 100 people experienced the system and we observed that for all participants, the heart rate was more elevated than in the beginning. Additional experiments endorsed that this effect was possibly caused by the presentation of a false heartbeat by the EH-VR.

2 Related Works

2.1 Effect of False Biofeedback in Terms of Emotion

According to the study by Valins in 1967, false heart sounds had an effect the emotional user’s impression of slides of seminude females  [8]. Bogus heart sounds, which were faster or slower than a subject’s real heat rate, were played while showing slides of seminude females. Subjects believed the heart sound played was their real heart sound, and the faster the false heart rate became, the higher the preference the subject indicated for the slide. Though this early work was a tentative analysis, it suggests that external information may affect a person’s emotional state. Nishimura et al. developed a tactile feedback device to provide a false heartbeat vibration on the chest as well as a tactile feedback device implemented in a cushion to provide a false heart rate unobtrusively [9]. They carried on Valins’s experiment to evaluate whether a false tactile heartbeat influenced the evaluation of attractiveness of female/male photographs. They reported that false heartbeats could alter physiological or emotional states. Costa et al. evaluated the effect of false vibrotactile biofeedback of the heart rate on the wrist to regulate anxiety [10]. The research clarified the fact that believing that the vibration represented their heart rate, the interoceptive awareness generated by the vibration would affect the regulation of their emotion. These related studies endorse our research to amplify fright by false biofeedback generated using a vibrotactile sensation. However, the purpose and system design are completely different from the one we propose. In terms of software design, their system provided a simple false heartbeat, either increased heart rate or decreased heart rate, and did not calibrate to a subject’s heart rate when generating false heartbeats. Our research purpose is to discover an effective way of deriving a false vibrotactile heart rate from a real heart rate, which is not yet certain. Therefore, we develop a vibrotactile feedback system to accelerate a user’s real heart rate by referring to a subject’s heart rate in real time and employing an acceleration equation model to calculate a false one. Finally, the goal is to discern an effective algorithm for deriving a false vibrotactile heartbeat from a real heartbeat to artificially add the sense of subjective reality to VR content.

2.2 Relationship Between Physiological State and Subjective Fright

Brosschot and Thayer indicated in [11] that the responses of heart rate associated with negative emotions prolonged the recovery of heart rate compared to responses associate with positive emotions. This implies that physiological response and emotional valence are interrelated. In our previous research [12], we concluded that an enclosed environment changed the physiological state as well as the emotional impression of a movie. We found that watching a horror movie in an enclosed space increased the heart rate more significantly rather than watching it in an open space. We analyzed the data collected in a previous experiment with respect to the relationship between physiological change and emotion. The result is summarized in Fig. 1. The graph shows the difference in the mean heart rate for each movie scene on the basis of a pre-test mean heart rate. The line plot shows the mean ratings of subjective assessment of fear for each scene (4 categories on the Likert scale: 1 (not scary) to 4 (extremely scary)) reported by 9 subjects. This suggests that the acceleration of heart rate and the increased ratings of subjective assessment of fear are related. In order to analyze the relationship between the two variables, we utilized single regression analysis. The heart rate value x is an explanatory variable of rating of subjective assessment of fear y; the equation is described as Eq. 1 \((\text {p} < .01)\). Although we require further investigation to bring this to a conclusion, this result is a positive endorsement of the fact that the acceleration of heart rate drives the increase in fright. Previous related research suggests that emotion and physiological changes are well-related and thus biofeedback will be effective in changing the emotional state based on external stimulus.

$$\begin{aligned} y = 0:04437x + 0:47759 \end{aligned}$$
(1)
Fig. 1.
figure 1

Mean heart rate and subjective assessment of scare

Full size image

3 EH-VR System

3.1 EH-VR Hardware

Figure 2 shows the system. We implemented a cylinder type wireless speaker (JBL Charge2) on the wooden plate set for the footrest. By putting the user’s feet on the footrest while wearing an Oculus Rift head-mounted display, a false vibrotactile heart beat is produced through the footrest at sole of the feet while gradually accelerating the heartbeat frequency. In order to measure a user’s heart rate to calculate a false heartbeat frequency, we used an ear worn photodiode pulse sensor which measures heart rate variability (HRV). An interactive walk-through in EH-VR is controlled by a wheel chair interface which detects a wheel’s speed and direction by the switch of a game controller. As a virtual body can be an important part of a presence-generating experience [13], an avatar that was gender matched and calibrated to the height of the user increased immersion  citelin2013. We intentionally used a wheel chair as an interface because sitting on a wheel chair cancels the height difference of each participant, thereby unifying the eye-level position. In addition, we draw a part of a user’s legs and a part of the wheel chair from the first-person perspective as Fig. 3 shows.

Fig. 2.
figure 2

Emotion hacking VR system

Full size image
Fig. 3.
figure 3

Sitting posture from the first-person point of view

Full size image

3.2 EH-VR Software

We recorded a heartbeat sound using a stethoscope and edited it as a sampling sound of a cycle of a heartbeat in order to provide vibrotactile feedback. We developed software that measures the interval time records of the heartbeat of a participant, calculates a false heart rate, and outputs a vibration pattern of the heartbeat through a spealer. In terms of determining the frequency of the false heart rate, we created an acceleration model of a heart rate, which is approximated by a quadratic equation. We derived this assumption from the analysis of data collected in the previous experiment [12]. The line plot in the Fig. 4 shows the state of the mean heart rate aligned as an increase over the baseline heart rate. The blue line illustrates the resulting analysis of 9 subjects (excluded one subject’s data from the result of the Cochran’s C test) who watched a 4-min horror short movie in a closed locker in an upright position. It shows that the biological boundary of fluctuation of heart rate in the upright position is about 28 bpm faster than the baseline and the outline of the increase obtains an approximate curve of the quadratic equation \((\text {R}^2 = 0.9)\). Based on this result, we conducted an experiment and determined that calibrating the heart rate every 10 s to correct the coefficient of the quadratic equation more effectively accelerated the real heart rate than calculating the false heart rate with a fixed coefficient [6]. In our EH-VR system, a participant maintains a seated posture during the experience. It is known that the biological boundary of fluctuation of heart rate in a seated posture is 30 % lower than that of an upright position [15]. Therefore, we revised the quadratic equation based on the acceleration model shown in Fig. 4 to account for the difference in heart rate of the upright position. In this EH-VR walk-through system, as a person controls walk-through speed by its own pace, we predetermined the fixed position where starting to present false heart rate in the CG world. Figure 4 shows an overview of walk-through space and each alphabet describes the position of either change of the scene or heartbeat feedback. From starting point of A, real heart rate feedback was presented and when it arrived the point B, a false heart rate feedback was presented until the point E.

Fig. 4.
figure 4

Heart rate increase of two postures (Color figure online)

Full size image
Fig. 5.
figure 5

Overall image of the walk-through CG space

Full size image

3.3 EH-VR Contents

With our EH-VR system, we provide a walk-through of the VR experience through artwork called the “Pressure of the unknown” demonstrating a fear metaphor. Figure 6 shows an image of the artwork. The concept of the artwork is as follows:

The “unknown” is a well-known cause of fear but sources of the ‘unknown’ vary. People are one of these sources, and the larger the number of people staring at a person the more’unknowns’ are produced in that person’s brain. This is similar to what happens during a public speech, where the audience analyzes everything about the presenter and therefore, become a source of fear (Fig. 5).

In this artwork, the viewers go into a closed unfamiliar organic space, constructed with human body parts. A large number of eyes stare at the viewer, trying to imitate the pressure induced by an audience when a presenter is on stage. In addition, attempting to amplify the feeling of fear, viewers sit in a wheelchair to provide a sense of vulnerability and constrained movement.

Fig. 6.
figure 6

A work image of EHVR

Full size image

4 Demonstration at Siggraph Asia 2016

EH-VR was demonstrated at Siggraph Asia 2016 held in Macau, China from Dec 6th to 8th in 2016. The duration of a demonstration for one person was about five to seven minutes. During three days of demonstration, 125 people experienced the contents. Comments from the participants were as follows: “It was a terrifying space”, “The vibration of tactile sensation felt like a natural heartbeat”, “I was surprised that my heart rate was rising before I noticed”. Therefore, it is thought that the EH-VR system was able to induce fearful emotions. Figure 7 shows a person experiencing the EH-VR. The contents of EH-VR was not intended to surprise people with ghosts and zombies but it induces the imagination about the inside of the human body so the tactile stimulation emulating the heartbeat sound was effective in enhancing the feeling of walking through the inside of a human body. In addition, we calculated the mean rate of increase in heart rate among 121 participants (4 participants were excluded because of a failure of the recording data). Based on the average heart rate of 74.7 bpm of 121 people before the experience, it increased by 21% on average after the experience. From the perspective of physiological response, as everyone had a rise in heart rate, it was shown that EH-VR amplified fearful emotions. Figure 8 shows one of the examples of a participant who shows an increase in heart rate close to the mean value. It has been shown that the heart rate rises gradually from the beginning of the experience and participants maintain a relatively high heart rate.

Fig. 7.
figure 7

Experience scene at Siggraph Asia 2016

Full size image
Fig. 8.
figure 8

Change in heart rate of a person close to average increase rate

Full size image

5 Verification of Heart Rate Rise Effect of EH-VR

In the walkthrough system of EH-VR, the rotation of the wheel of the wheelchair was detected and used as a space movement input such that the user could move through the space arbitrarily. In the system developed in the previous study used for the generation of a false heartbeat [6], it was observed that the subject was in a stationary state and the heart rate increased due to the presentation of false vibrotactile feedback. In the current EH-VR system, since the exercise of rowing a wheelchair was included as a part of the interaction, there is a possibility that the heart rate rose as a result of the exercise. It may also be possible that there was no influence caused by the visual and haptic feedback presented by the system at all. Therefore, a verification experiment was carried out to confirm whether the rise in heart rate occurred in the seated position as well according to the sensory stimulation presentation. Eight subjects performed an EH-VR walk-through experience using the fame controller with all other conditions the same as the conventional EH-VR as Fig. 9 shows. Based on the average heart rate at the beginning, all subjects increased their heart rate by 8% on average (the mean at the beginning was 82.3 BPM, the mean at the end was 89.1 BPM). Figure 10 indicates the heart rate of one subject during the experiment. It shows that the heart rate remained high and was unchanged during the presentation of the false heartbeat as observed in previous studies. Although the number of subjects in this experiment is different from the number of participants at the exhibition and it is difficult to compare, an increase in heart rate was observed without the effect of exercise. Therefore, it can be assumed that the sensory stimulus presentation generated by EH-VR had some influence on the physiological response.

Fig. 9.
figure 9

Subject experiencing EH-VR walk-through system using a game controller

Full size image
Fig. 10.
figure 10

Change in heart rate of a subject

Full size image

6 Conclusion and Future Works

In this research, we created VR experience content called Emotion Hacking VR (EH-VR) for the purpose of amplifying a scary VR experience. The system was showcased at Siggraph Asia 2016. In a previous study, EH-VR produced a false heartbeat on the sole of a user as a tactile sensation while the user watched 3D horror content in a standing position. However, in this study we applied a method to raise and maintain the heart rate of a user in a seated position. At the exhibition, all users experienced an increased heart rate within a certain range. It appeared that the system allowed users to enjoy the VR-based content. Based on the additional experiment, it was observed that there was a change in the physiological response via the presentation of the tactile stimulus. As a part of a future study, a method that can generate the experience along with artificial emotions will be developed using a combination of multiple senses.