Keywords

1 Introduction

The VR market is entering a new phase from the sales of headsets to the spread of platforms, for example, Sony released PlayStation VR and Microsoft made Windows Mixed Reality available on Windows 10. Also, various kinds of completely new content are being developed. Specifically, a lot of content is constantly released in which users can enjoy special experiences by utilizing the high reality feeling of VR headsets. For example, content to experience live-action 3D videos such as Exploring the SHINKAI VR [1] and Skydiving [2], and interactive experiences using CG such as BIOHAZARD 7 [3] and SUMMER LESSON [4] have been developed.

We have developed a VR ski jump system [5]. This system has already been experienced by more than 600 people at the Sapporo Television Broadcasting (STV) Cup’s Virtual Ski Jumping Competition in Chi Ka Ho held in the underground walking space in front of Sapporo station (Fig. 1). Participants could jump and enjoy a long-distance flight regardless of age as shown in Fig. 2.

Fig. 1.
figure 1

A scene from the virtual ski jump tournament.

Full size image
Fig. 2.
figure 2

Age of the participants and their scores.

Full size image

In this study, we focus on improvement of experiences with HMD, and propose how to design experiences in video content. The common way was to develop a game content inspired by the video content like Super Robot Wars [6]. Conversely, we focused on the fact that we can reproduce the world of the virtual video content and perform various actions there by using HMD and VR technology. We regard such technology as a new style of make-believe play and examined how the experience of video content was improved by introducing the fun of make-believe play. For our study, we selected Ski Jumping Pairs [7] known as a media art work, and we examined how to experience the content.

2 Related Studies

2.1 Experience Ski Jumping with VR

There are many VR ski jumping systems using live-action video, and they enable users to vicariously enjoy player’s experience. Various VR ski jumping systems using CG have also been developed, such as VR SKI JUMP [8] developed as the first app in Oculus. Some of these systems have been used to launch events, for example, Planica 2015 Virtual Ski Flying by Zavarovalnica Triglav [9]. On the other hand, systems with large-scale equipment developed for training are also reported such as the Staurset’s system [10].

2.2 Experience Video Content

Focusing on the experience of the video content, various systems are being developed. For example, BANDAI NAMCO ENTERTAINMENT released DRAGONBALL XENOVERSE in 2015 [11], and Rare Ltd. released GoldenEye 007 in 2011 [12]. In these cases, video content is made into games. Flow theory is well-known as a method for immersing in games. However, this theory is not applicable when there is a significant difference between players’ skill levels because flow theory is proposed to explain one player’s balance of skill level and the difficulty of the game.

As for multi-player VR content, Mikulus Kinect Online was developed by Needle in 2013 [13]. This is a technological prototype to realize that two players can play the role of CG character Hatsune Miku in the same VR space, so to speak, it is a “cosplay” using VR. In commercial software, Sony Interactive Entertainment Inc. has released RIGS Machine Combat League for PS VR in 2016 [14]. This is a game content in which multiple players manipulate robots and fight with each other from a first-person perspective.

On the other hand, in this study, we focused on studies of make-believe play from the standpoint of improving the experience. Specifically, we focused on the elements of fun in make-believe play proposed by Yagi [15]. He classified the fun of make-believe play as five elements: “role- fun to become someone”, “object- fun to create or use something”, “action- fun to do something”, “space- fun to consider as somewhere”, and “interaction- fun to play with someone”. He said that the weights of these elements varied depending on the types of make-believe play.

3 Developing Pair Ski Jump System

3.1 Targeted Video Content

Ski Jumping Pairs is video content of an imaginary sport in which two players jump using a pair of skis. The highlight of the content is incredible aerial style of players. According to Wikipedia, it became very popular by the internet streaming service and it was screened in various film festivals around the world. In addition, Ski Jumping Pairs - Reloaded - for PS2 was been released by KAMUI in 2006 [16]. This game has interesting live comments on various aerial styles and actions selected by the user, so it attracts fans of video content.

3.2 Method

We propose that we can enhance the experience of video content by deciding the details of the five elements by Yagi that players will experience, and by assigning them an appropriate weight according to the video content. In the Ski Jumping Pairs experience, we first decided the content of five elements as shown in Fig. 3. We assumed that it had two features, jump and pose as “action”, and jump by two players as an “interaction”. Then we changed the weight of the “action” and “interaction” elements as shown in Fig. 4. There could be many designs of the balance of weights, for example, an action-oriented mode in which each player concentrated on “action” by improving aerial styles completion, and an interaction-oriented mode in which each player concentrated on “interaction” by requiring cooperation.

Fig. 3.
figure 3

Elements and details of fun.

Full size image
Fig. 4.
figure 4

Weights of elements.

Full size image

4 System

4.1 System Configuration

We developed a prototype of the system targeting two-player, side-by-side at game centers etc. Therefore, we configured the system with an HMD (Oculus VR, Oculus Rift DK2), an HMD head tracking camera, a motion sensing input device (Microsoft, Kinect v2), and a PC for each player (Fig. 5).

Fig. 5.
figure 5

System configuration.

Full size image

Two PCs shared data such as jump timing and character pose by UDP with P2P. HMD was used to present the virtual world. Kinect v2 was used to sense the pose of the players and to reflect it on the character IN the virtual world. As the development environment, we used Windows 8.1 for PCs OS, and we used Unity 5.3.4 and C# for development language. The SDK we used for HMD was Oculus SDK 0.8, and for Kinect v2, Kinect SDK 2.0. We used LightWave 11.5.1 for modeling CG objects.

4.2 CG Model and Jump

We made a CG model with the motif of Okurayama Ski Jump Stadium (Fig. 6). A jump stand for skiing has an altitude of 133.6 m, a total length of 368.1 m, and a slope length of 403.8 m. The altitude of the starting point is 307 m. When creating CG models, we created a jump stand for skiing to be this size. Also, in order to make it easy to understand the timing of jumping, we colored the take-off point pale pink and dark pink.

Fig. 6.
figure 6

Scenery in CG. (Color figure online)

Full size image

We also adjusted the virtual jump. First, the actual duration of jumping was about three seconds, but we changed it to about five seconds because there was a feedback that it was too short. Also, in order to give a feeling of flight, as shown in Fig. 7, we set the height of the jumping orbital from the slope to be high and made a direction in landing towards the ground to make it easy for players to recognize that they have landed. The range of distances was set at five levels. The longest distance was set about 4.35 s when the CG character just passed through the take-off point. Other levels were set around 4.35 s. The distance was set randomly. All distances were set over 100 m so that even small children or elderly people can enjoy the jumping experience.

Fig. 7.
figure 7

Timing and orbit in jump.

Full size image

4.3 Developed Mode

In Ski Jumping Pairs, the score is mainly calculated by the distance and aerial styles similar to real life ski jumping competitions. In our virtual ski jumping, we scored the distance according to the timing of the jump and scored the “performance (waza)” instead of the aerial styles, and combined them as the score of the jump. We calculated “performance” based on the difference between the player’s pose and an ideal pose. As shown on the left side of Fig. 8, after displaying the ideal “performance” of the video content on the display beforehand, and players jumped while taking a pose after remembering it (Fig. 8 right).

Fig. 8.
figure 8

Example pose and posing.

Full size image

Figure 9 is an example of “performance” we developed in this system. These are reproduced representative “performance” of video content. In detail, players take different poses in sliding, in jumping, and after landing, so that they can enjoy various “performance” during one jump.

Fig. 9.
figure 9

Type of poses.

Full size image

After landing, the score is displayed on the scoreboard as shown in Fig. 10. We created an action-oriented mode and an interaction-oriented mode shown in Sect. 3.2 by changing the display method of this score. In an action-oriented mode, we displayed players’ individual scores on the timing of the jumps and the degree of completion of “performance”. By displaying independent scores, players can concentrate on improving timing and “performance.” They can raise the elements of “action”, and improve the experience. In an interaction-oriented mode, we calculated the difference of the timing when two players jumped and displayed the same value to both players. The degree of completion of “performance” was the average of two players. In this mode, players enjoy communication. They can raise the elements of “interaction”, and improve the experience.

Fig. 10.
figure 10

Scoreboard.

Full size image

5 Evaluation

5.1 Purpose and Method

We performed an experiment to evaluate in which mode the experience of this video content is more enjoyable for users.

We showed participants the video of Ski Jumping Pairs to introduce it. Then we told participants that the prototype system is designed to experience the world of Ski Jumping Pairs. We instructed them how to play and explained that there are two scoring methods. When we discussed the scoring methods, we avoided using the names of the modes. In the experiment, we asked participants to wear HMDs. Then, we asked participants to experience each mode five times. In the action-oriented mode only, we asked them not to tell their own score to each other. This was because we wanted them concentrate on their play and prevent competition. When they experienced each mode, we asked players not to remove the HMD unless complications arose, such as VR sickness or fatigue. The order of the modes was randomly assigned. After the experience, we asked participants to answer the questionnaire of five items shown in Fig. 11 on a seven-point scale. Item (1) is a question about the experience of the video content. Item (2) is a question about the evaluation of the content. Items (3) to (5) are questions about elements of fun. After that, changing modes, we asked participants to experience another mode in the same procedure. During the experience, we recorded scores. Participants are twenty people aged 19 to 22 years (eighteen males and two females).

Fig. 11.
figure 11

Experimental scene.

Full size image

5.2 Results

Figure 12 shows the results of the questionnaire. The results show that we could develop a fun and enjoyable platform regardless of the mode because the average value of item (2) was very high, 6.4, in each mode. A Wilcoxon signed-rank test was conducted in order to reveal differences between the action-oriented mode and the interaction-oriented mode, and these results are also shown in Fig. 12. Item (5) shows significant differences at a significance level of 1%. Items (1) and (4) are even more different, at a significance level of 5%. The interaction-oriented mode was evaluated higher for each of these items.

Fig. 12.
figure 12

Result of evaluation.

Full size image

In addition, the difference between the jumping timing of the two participants is shown in Fig. 13. A T-test was conducted in order to reveal differences between the action-oriented mode and the interaction-oriented mode, and these results are also shown in Fig. 13. The second jump and the fourth jump of experience show significant differences at a significance level of 1%. The average of all trials shows differences at a significance level of 5%. It shows that the difference between the jumping timing of the two players was smaller in the action-oriented mode than in the interaction-oriented mode.

Fig. 13.
figure 13

Transition in score.

Full size image

5.3 Discussion

There was a significant difference at item (1). This means that the experiences of the content changed according to the weight of each element. As shown in the results of items (4) and (5), participants felt that they communicated in the interaction-oriented mode, that is, they increased the element of “interaction”. Also, as a result of the score, the timing of the jumps in the action-oriented mode was better than that in the interaction-oriented mode. This means that participants could concentrate on their jumps during the action-oriented mode, that is, they increased the element of “action”.

From these results, we can conclude that the element of “interaction” is important for the content experience of Ski Jumping Pairs.

In the future work, we plan to apply the weight of each element to various video content.

6 Summary

In this study, we developed a system in which users can experience Ski Jumping Pairs. We used this system to evaluate how to experience the world of video content better. According to the results, the experience of the world of video content which two players cooperate was improved by increasing the rate of the element of “interaction”.