1 Introduction

In this work, we are interested in how to design a training for professionals to learn how to act in crisis situations. These professionals could be emergency responders and crisis managers. In real-life crisis situations, people experience fear or anxiety. Therefore, it would be effective if these professionals would experience such anxiety in training, otherwise the transfer of training would be less. For example, Nieuwenhuys and Oudejans [8] determined that training with anxiety has more short- and long-term transfer of training than training without anxiety. In existing paper-based training for emergency situations, trainees do not experience much anxiety. Therefore, it is useful to investigate the possibilities and effectiveness of a virtual training for crisis situations. Another reason for investigating virtual training is the cost-effectiveness. Real-life practice drills of emergency situations, such as an evacuation due to a fire, have the goal to save lives, but cost a lot of time and money to arrange. Therefore, virtual training can also be a cost-effective method to teach these professionals how to act in emergency situations while experiencing anxiety and fear.

Two possibilities for a virtual training will be tested here, cardboard Virtual Reality (VR) versus 2D video. Cardboard VR was chosen because it was easily accessible to the researchers and the differences between inducing fear with cardboard VR or a traditional head mounted display VR seem to be small [1]. 2D video was chosen as the comparison condition, because this could be a good alternative option for a VR based training. It would be less expensive to develop a video based training then a VR based training.

The main research question is: which video screen (cardboard VR or 2D video) induces fear the strongest? An experiment was set up in which participants view a short horror movie. To measure the induction of fear, ratings of fear were made with: (1) subjective ratings: PANAS questionnaire; (2) objective ratings: heart rate and blood pressure. The PANAS was used, because it is a more reliable and valid mood scale, compared to other mood scales [10]. Although more recent research determined that other mood scales are comparable in measuring anxiety [9]. Heart rate and blood pressure were chosen because it is assumed that watching a horror movie will increase fear or anxiety and therefore the heart rate and blood pressure [2]. For heart rate, it is expected that watching the video will increase heart rate significantly. For blood pressure, it is expected that fear increases systolic blood pressure more than diastolic blood pressure. This is based on earlier findings where fear seems to be mainly a sympathetic nervous system reaction, compared to an emotion such as anger which seems to be both a parasympathetic as well as a sympathetic nervous system reaction [2]. Although in previous research not always an effect of anxiety induction by video on heart rate is found [4]. Personality characteristics and gender were measured to answer a secondary research question: does gender or personality have influence on the mood induction? It could be that males or females are more easily induced with fear or that the personality characteristics have an influence on the mood induction. For example, in a review from McLean and Anderson [7], it was determined that females seem to report greater fear and are more likely to develop anxiety disorders than men.

The rest of the article is organized as follows. Next, the method of the experiment is explained in Sect. 2, followed by the results in Sect. 3. The article is summarized and discussed in Sect. 4.

2 Method

Participants.

Sixty participants (30 males, mean age 25,4 years) were randomly assigned with a random number generator to one of two conditions: VR or 2D video screen. Participants were recruited on campus and through social networks. They received 5 euro for participation.

Procedure.

The participant gave their informed consent, followed by the PANAS questionnaire [10] and a short version of the Big Five questionnaire [5]. Then, the heart rate and blood pressure were measured, followed by inducing the participant’s mood by showing a horror video of 3 min. Participants watched either on a 2D smartphone screen (in 360° mode) or through cardboard VR glasses in which you can place a smartphone. During and after watching this movie, the heart rate and blood pressure were measured again. Finally, the participant filled in the PANAS again and a last form with 4 personal questions (age, gender, nationality and education level) and two questions concerning the movie and the induced mood (1. Which emotion did the watching the movie provoke?; 2. How scary did you find the movie?). To make sure the participant left the experiment with a positive mood, he/she would play a video or song that induces a positive mood.

Material.

The video was shown on a smartphone with a 5-inch screen size. The horror video that was used can be found here (reference). Cardboard VR glasses were used with a strap around the head, in which the smartphone was placed. In the 2D condition, the participant held the smartphone in his/her hands. In both conditions, the participant sat in a rotatable desk chair, which allowed the participants to move freely. Turning around with the smartphone (2D) or VR glasses made the participant explore the environment in the video (left and right, up and down). Heart rate and blood pressure were measured with a digital OMRON M6 blood pressure meter.

3 Results

3.1 Homogeneity and Manipulation Check

One participant was excluded from the data analysis, because of using antidepressants which influences the mood induction. Fifty-nine participants were included in all data-analyses (29 males, mean age: 25,4 years, range: 19–56 years). See Table 1. Homogeneity tests were performed to test if the variances in both groups were equally distributed and therefore there were no differences in the convenience sample with random assignment. Variances in gender were equal in both groups, \( \chi^{2} \)(1) = 0.18, n.s. Age was not equally distributed in both video screen conditions, F(1, 57) = 9,445, p < 0.05. This can be expected in a convenience sample, therefore the medians and the 25th and 75th percentiles are shown as well in Table 1. Education level was equally distributed in both groups, \( \chi^{2} \)(4) = 2.552, n.s.

Table 1. Group characteristics of participants included in the data-analyses
Full size table

Next, it was tested if the horror video induced fear in the participants. First, normality was tested with the Kolmogorov-Smirnov test and visual inspection of the Q-Q plots. Both samples for PANAS items ‘scared’ or ‘afraid’ were not normally distributed. See Table 2 for the average scores ± standard deviation and the medians. To test whether the mood induction was successful, the two PANAS items were tested with the Wilcoxon sum-rank test. The scores on both items increased significantly: scared, W = 764,5, z = 4,47, p < .001, r = 0,58; afraid, W = 397,5, z = 4,54, p < .001, r = 0,59. Moreover, the answers to the two questions at the end of the experiment were analyzed. See Table 3. All answers to question 1 were categorized by two independent reviewers in two categories: fearful/scary emotions versus non-fearful/scary emotions. Examples of answers are: ‘fear’, ‘anxiety’, ‘sensation’, ‘neutral’, ‘tension’, ‘anticipating’. The agreement between the two raters was 80%. After resolving disagreement, the outcome was 46 fearful/scary emotions versus 14 non-fearful/scary emotions. Therefore 78% of the participants rated the video as provoking a fearful/scary emotion. The average answer to question 2 was 2,75 and the median was: 3. The Likert-scale for the answer was 1 = not scary at all to 5 = very scary. Overall, the participants did find the movie scary and the mood induction was successful.

Table 2. Mean ± standard deviations and medians on PANAS items ‘scared’ and ‘afraid’.
Full size table
Table 3. Answers to questions about the provoked emotion
Full size table

3.2 Fixed and Random Effects of Video Screen, Gender and Personality

Normality and homogeneity of variances.

First, the data was checked on normality by combining the Kolmogorov-Smirnov test with inspecting the data visually with histograms, P-P and Q-Q plots and also calculating the skewness and kurtosis. The results from the Kolmogorov-Smirnov test corresponded with the visual inspection: most data was not normally distributed. Positive Affect (PA) was not normally distributed at time point 2 (after the video) for both video screen conditions, D(30) = .169, p < 0.05, D(28) = .192, p < .05. Negative affect (NA) was not normally distributed before the video for the VR condition, D(30) = .162, p < 0–.05, and after the video for both conditions, D(30) = .168, p < 0.05, D(28) = .183, p < 0.05. The data for systolic blood pressure and heart rate were normally distributed, but not the diastolic blood pressure data. At time point 1 (before the video) it was normally distributed, but during the video it was not normally distributed for both video screen conditions, D(30) = 0.33, p < 0.001, D(28) = 2.57, p < 0.01. And after the video it was not normally distributed in the 2D condition, D(28) = .195, p < 0.01. The data was checked for homogeneity of variances with Levene’s test. For all dependent variables and for both video screen conditions, the variances were equal. When transforming all data in log10 or with square root, the same results showed. Based on these tests, heart rate and systolic blood pressure were analyzed with parametric tests, diastolic blood pressure was rank transformed to allow a parametric test and PA and NA with non-parametric tests.

Fixed Effects of Video Screen and Time on Affect.

Table 4 gives an overview of all mean ± standard deviation scores on all dependent variables, sorted by condition (VR or 2D video screen). Before watching the video or after watching the video, PA did not significantly differ between video screen conditions, U(438), z = .046, n.s., r = .006, U(457), z = .334, n.s., r = .043. For both VR and 2D conditions, there were no significant differences between PA before and after the video, W = 235, z = .379, n.s., r = .049, W = 204,5, z = .373, n.s., r = .049. NA did not significantly differ between video screen conditions at time point 1 (before watching the horror video), but did significantly differ at time point 2 (after watching the horror video), showing a higher score for the VR condition, U = 349,5, z = −1,3, n.s., r = −0.17, U = 301, z = −2,034, p < 0.05, r = −0,26. Looking within each condition, NA did not significantly increase in the VR condition, W = 248,5, z = 1,039, n.s., r = 0,14, but did significantly increase in the 2D condition, W = 210,5, z = 2,208, p < 0.05, r = 0.29. For both video screen conditions ‘scared’ and ‘afraid’ increased significantly. Scared x VR: W = 276,5, z = 3,213, p < .01, r = 0,42, Scared x 2D: W = 127, z = 3,096, p < .01, r = 0,40, Afraid x VR: W = 146,5, z = 3,458, p < .01, r = 0,45, afraidx2D, W = 66, z = 2,965, p < .01, r = 0,39. Even though the NA increases in the VR condition, it is not significant, like in the 2D condition. The large variances make it non-significant. It could be explained by sample size or coincidence. More importantly, ‘afraid’ and ‘scared’ did significantly increase, which were the targeted emotions. To generalize this to the full range of NA was not significant in the VR condition, but that was not necessary in this experiment.

Table 4. Mean scores ± standard deviations for all dependent variables
Full size table

Fixed Effects of Video Screen and Time on Heart Rate and Blood Pressure.

The effect of Video Screen and Time on heart rate and systolic blood pressure was tested with 2 × 3 two-way ANOVA’s with Video Screen (VR, 2D) and Time (before, during, after) as between Factors. Heart rate differed significantly over time, F(2) = 10,513, p < .001. The interaction Time x Video Screen was not significant F(2) = 1,241, n.s., indicating that for both video screen conditions, the exact pattern was found. With post hoc tests, including Bonferroni correction, it was found that all means for the heart rate at time 1, 2 or 3 (before, during, after) differed significantly, before-during: p < .001, before-after: p < .05, during – after: p < .05. Systolic blood pressure differed significantly over time as well, F(2) = 3.571, p < .05. The interaction between Time x Video Screen was not significant, F(2) = .235, n.s., indicating participants in both video screen conditions showed the same pattern in their increase and decrease in systolic blood pressure. With post hoc tests including Bonferroni correction, it was found that the systolic blood pressure differed significantly from during to after the video and there was a trend found for before to during, no significant contrast was found before-after, before-during: p = 0.11, before-after: n.s., during – after: p < .05. The effect of Video Screen and Time on diastolic blood pressure was tested as well with 2 × 3 two-way ANOVA’s as before, but then on rank transformed data, because of the violation of the normality assumption. Diastolic blood pressure did not differ significantly over time, F(2) = .024, n.s. The interaction between Time x Video Screen was not significant as well, F(2) = .095, n.s., indicating participants in both video screen conditions did not significantly increase and decrease in systolic blood pressure and did not differ from each other in their patterns over time. These results show an interesting pattern for heart rate, which for both conditions significantly increases during watching the video and then significantly decreases afterwards but not yet to the beginning rest heart rate. The systolic blood pressure was significantly higher during watching the video than afterwards. The trend shown for the increase from before to during the video could be explained by positive ‘pressure’ to begin the experiment which already raised the blood pressure a bit, making the increase from before to during the movie not significant. Although this was not the case for the heart rate. In summary, the horror video did significantly increase the heart rate and systolic blood pressure, but not the diastolic blood pressure (Figs. 1, 2 and 3).

Fig. 1.
figure 1

Heart rate before, during and after watching a horror movie

Full size image
Fig. 2.
figure 2

Systolic blood pressure before, during and after watching a horror movie

Full size image
Fig. 3.
figure 3

Diastolic blood pressure before, during and after watching a horror movie

Full size image

Random Effects of Gender.

The random effects of Gender on the dependent variables heart rate and systolic blood pressure were analyzed with a mixed model ANCOVA analysis with Time as Within-Factor, Video Screen as Between-Factor and Gender as covariate. The Main effect of Time showed a trend, F(2) = 2.996, p = .054 and the Time x Gender and Time x Video Screen interactions were not significant, F(2) = .436, n.s., F(2) = 1.674, n.s. The same mixed model ANCOVA was performed for systolic blood pressure and none of the main effect or interactions of Time x Gender and Time x Video Screen were significant, F(2) = .687, n.s., F(2) = .184, n.s., F(2) = .267, n.s. Indicating that there is no difference between females and males in the heart rates and systolic blood pressures in both conditions and over time. Mixed Model ANCOVA’s were also performed on rank transformed ‘afraid’ and ‘scared’. For both ‘afraid’ and ‘scared’ the main effects of Time and interactions Time x Gender and Time x Video Screen were not significant, F(1) = .271, n.s., F(1) = .303, n.s., F(1) = .099, n.s., F(1) = .354, n.s., F(1) = .395, n.s., F(1) = .03, n.s. Indicating there is also no difference between females and males in the negative affects ‘afraid’ and ‘scared’ in both conditions and over time. Therefore, gender does not explain part of the variance in the dependent variables.

Random Effects of Personality.

First it was tested if the independent variable (VR or 2D screen) has a relationship with the Personality variables (openness, conscientiousness, extraversion, agreeableness, neurotic) to decide if they can be taken as covariates. Mann Whitney tests were performed for each Personality Variable x Video Screen. Only extraversion levels differed significantly between type of video screen, U = 307,5, z = −1,962, p = .05, r = −0,26. All other personality variables had the same distribution and can therefore not be taken as a covariate, because they explain some of the variance in the independent variable as well as a covariate, U = 455,5, z = 0.321, n.s., U = 363,5, z = −1.104, n.s., U = 386,5, z = 0.761, n.s., U = 393, z = −0.652, n.s. Next, the random effects of Extraversion on the dependent variables ‘afraid, ‘scared’, heart rate and systolic blood pressure were analyzed with a mixed model ANCOVA analysis with Time as a Within-Factor, Video Screen as Between-Factor and Extraversion as covariate. The effects on diastolic blood pressure were not analyzed, because the previous analysis showed there was no effect of mood induction for diastolic blood pressure. For ‘afraid’ and ‘scared’, the ANCOVA was performed on rank transformed data, because of the violation of the normality assumption. The main effect of Time and the interactions of Time x Extraversion and Time x Video Screen were not significant for any of the dependent variables. For heart rate and ‘scared’, trends were found. For ‘scared’ the main effect Time and Time x Extraversion showed trends, F(1) = 2,937, p = .092, F(1) = 3,136, p = .082. For heart rate a trend was found for the main effect of Time and Extraversion as the covariate, F(2) = 2.117, p = .13. This indicates that when extraversion is taken as a covariate, there is a trend visible in that it could possible explain some of the variance in the dependent variables heart rate and ‘scared’ affect.

4 Conclusion and Discussion

This study was designed to determine whether watching a horror video with cardboard VR can induce more fear than watching the movie at a 2D smartphone screen. Results showed that both video screen conditions induced fear significantly. For both video screen conditions, the subjective measurements (PANAS-afraid, PANAS-scared) increased significantly from before to after watching the movie. For both video screen conditions, the objective measurements of fear (hear rate and systolic blood pressure) increased and/or decreased significantly. For heart rate, it increased significantly from before to during the movie and then decreased significantly, while systolic blood pressure decreased significantly from during the movie to after the movie. There was no effect of mood induction on diastolic blood pressure. The successful mood induction on the subjective ratings and the heart rate and systolic blood pressure were in line with the expectations. The finding that both video screen conditions were equally good in fear induction and showed the exact same patterns in the data was unexpected. When taking gender or extraversion as covariates, no significant part of the variance was explained by the covariates. This indicates that gender and personality do not seem to have a significant influence on the fear induction.

Strong points of this experimental design are the random assignment, the strong fear induction found in both VR and 2D video and the measurement of fear with both objective as well as subjective measurements. Improvements would be to add a third condition of inducing fear by a ‘paper-based’ scenario. This would allow a second comparison that would be interesting to decide how much stronger 2D and VR could induce fear than a paper-based scenario.

For the future of virtual trainings, these results show that both VR as well as 2D video can increase fear and anxiety in the user. Both in subjective as well as objective measurements. This would make virtual trainings both realistic and hopefully more effective. At the same time, there were no significant differences found between VR and 2D video screen, implying that training developers do not have to invest in (expensive) VR technology but could explore video based training. Of course, this is only based on this experiment, more research would be needed to fully investigate this perspective. A recent study for example found that 2D video can provoke anxiety in users with ‘threatening stimuli’ in the video, but provoke less anxiety than in real-life situations [3]. Also, besides mood induction other elements of crisis situations, such as having an overview or taking different views/perspectives in reality could also be important points for training. Perhaps that on other elements of training there is a significant difference between VR and 2D. We hope this study is a valuable contribution to current and future research in this area.