Keywords

1 Introduction

Advancement in IT technologies creates disturbances to the business and scientific world but also create many opportunities. One of such technologies is Virtual reality (henceforth VR). VR can create and simulate a three-dimensional (3D) interactive environments. Such technology can create immersive learning experiences in a new and unique way (Sun, Wu and Cai 2018). The aim of this article is to show the conclusions from the design, production and testing the serious VR game for teaching soft-skills in public teaching through a simulated scenario of a board meeting. We would also like to discuss the various learning contexts, in which such games can be used and implemented.

The capabilities of the VR technology for learning and education is potentially very broad (Hockey et al. 2010), from creating simulators and showcasing functioning of almost any environment possible to imagine, through supporting 3D driven design and development, up to the full function interactive virtual worlds and supporting telepresence. The VR environments, in todays form, present both intrinsic and extrinsic value to the learners (Shukla and Conrad 2011). Although, VR technologies for learning show their true learning capabilities when are linked in interactions with the physical world (Christopoulos et al. 2018; Bower et al. 2010; Hoshi et al. 2009). Looking for synergies between serious VR game design and learner choices in the interactive environment can create a very high level of engagement and increase the learning effectiveness with the educational content that is present in the virtual and physical environment.

When considering using VR tools in classroom environment one needs to challenge the problem of group participation. As the VR headsets are rather expensive to have more than one at once in use, the VR experience designers are trying to overcome that with engaging the whole audience in the experience. It would be extremely difficult to maintain everyone’s attention while doing the same exercise with only one participant at once. The idea of including course participants who don’t wear the HMD at the moment into whole learning experience was a design goal throughout the project.

VR technology also faces a range of limitations. One of the major limitations is simulator sickness (Maraj et al. 2017) or as many researchers relate it to the VR implantation through HMD as cybersickness. Rebenitsch and Owen (2016:102) define cybersickness as “the onset of nausea, oculomotor and/disorientation while experiencing virtual environments in head-mounted displays, large screens, and curved screen systems.” The growing number of research is concentrated around the topic of sources of cybersickness, and the results are mixed. Some of the research results reported that HMD usage in VR environments could cause cybersickness (Yörük et al. 2018; Treleaven et al. 2015) and it has a negative effect on the presentence in the VR environments. However, other earlier studies indicated that walking and movements, which are in line with the natural body movement can prevent simulator sickness (Chance et al. 1998) or that presence in the VR environment is not related to the cybersickness but other external effects are causing it (Nichols et al. 2000). The topic of cybersickness is still underdeveloped and more research on this topic is still needed for a better understanding of this phenomena.

The following paper describes design concept of positive social VR and presents results of a pilot study of VR user experience perspective. That data was a valuable input for the next iterations of the simulation development and can be treated as a benchmarking measure for similar projects.

2 Game Design

One of the most challenging tasks in the VR application for learning is designing the environment that is both natural and immersive. The VR games are good at recreating existing or fictional environments with a high grade of physical fidelity. However, in simulation gaming, we use abstraction for showing high-level theoretical concepts or when dealing with complex systems. VR games present on the market so far for educational purposes are struggling in using abstraction without being perceived as silly or unprecise. Thus designing a serious VR game for management filed in the soft skills area was a very challenging and uneasy task.

The serious VR game “CICERO” has been designed with learners of beginner or entry level of knowledge and skills in public speaking for business. The basic aim of the game is to create a realistic environment and game scenarios that will incorporate the most typical business environments and situations that require public speaking skills.

The whole idea for creating a VR presentation training tool was ignited during a brainstorming session of the core team. As a starting point of generating ideas the team got the concept proper use of VR medium with a focus on building an immersive simulation of a situation that can be helpful for business school students. The final choice was made after an internal discussion about the most valuable scenario to simulate. What is interesting is that forms of interactions were present in most of the ideas, which were using voice and gesture recognition.

Design goals set at first were as follows:

  1. 1.

    Training of public speaking in a realistic environment and context: the user will have the possibility to try her skills in a situation that can occur in the future.

  2. 2.

    Focus on creating a feeling of presence inside VR: real-time reactions of virtual models, background sounds, questions from the audience.

  3. 3.

    Measurement of users performance: development of the scoring system that can be used throughout different scenarios and will enhance engagement.

Since the simulation was aimed to be used in a classroom setting there was an additional design goal of extending the experience beyond just one user wearing a VR headset. Including a broader audience was definitely an interesting challenge that could result in the increased attention span of the whole group and not only the person that is in the training process at the moment. One of the main objectives of the game system design was to fully use the potential of VR hardware and create an immersing virtual reality environment, which will help with public speaking training. Cicero can be used as a solo experience or in a classroom environment, with the live audience. In making the experience more attractive for all people present in the room we have designed asynchronous and asymmetric participation experience for the audience. We provide a simple voting mobile tool, which they can collectively choose questions based on the majority rule. The chosen questions will be asked by virtual models inside the simulation. That corresponds to Jagnow (2017) talk during VRDC Fall 2017 about building positive social VR. He mentions three attributes of VR and AR experiences that are not getting much attention yet during design: asymmetric, asynchronous and abuse. Asymmetric is about joining the same experience from different devices. Asynchronous enables participation of a group of users at different time intervals. Abuse aspect treats negative interactions that can be present within the virtual group experience between its participants. In Cicero we designed an asymmetric and asynchronous experience, if the training takes part in the classroom. The first version of the voting tool aimed to research on how that positive social experience of VR will work in a learning context. By design the voting will happen after the user finishes her presentation and the real audience will vote on one out of four questions randomly chosen from the whole set of twenty (Fig. 1).

Fig. 1.
figure 1

The starting screen of Cicero VR

Full size image

The game has been designed and build in the Unity3D engine with HTC Vive head-mounted displays in mind. The type of the HMD is important as they feature very different screen resolution and thus have implication for the depth of the field of vision presented to the player. The field of vision depth is important for the design of the CICERO VR game as we present periodically the information on the effectiveness of the player on the edge of the field of vision in the form of a HUD (heads-up display).

One of the other concerns was cybersickness. Although, it is impossible to fully get rid of the cybersickness in VR, we can decrease the probability of the cybersickness occurrence with the game design (Yörük et al. 2018; Treleaven et al. 2015; Chance et al. 1998). In the case of CICERO, the player is positioned in the natural way of a presenter standing in the brightly lighted room. The movement in the game is natural (walking) and limited to the 2–3 steps in every direction and this is in-line with the role of the presenter in the game.

3 Game Scenario

In the process of the scenario design for the first version of the game, we have decided to use the two most common scenarios with different game modes in mind.

Scenario no 1. The first scenario is a typical boardroom setup. The story follows the situation when you are the project manager of the growing carsharing company (CarMa), and you present two variants of the project execution options to the board of directors. In this case, you will receive a prepared presentation from your “team”, and your task is to present it in front of the board in a professional way.

In this scenario, the game works in the mastery mode. In this mode, the task of the learner is to be as close as possible to the perfect presentation pattern, i.e. master. The mastery, in this case, is established by two public speaking professionals – male and female, who present the same presentation in an instructional video. Also, the learner has a virtual card in one of the hands with useful hints and tips, which can work as a form of a virtual teleprompter. In the master mode, the internal scoring system counts the deportations of the learner from the “perfect” master presentation and displays the positive and negative scores in the form of the visual aids every time the learner switches the slide in the game. The presentation has 18 slides (with starting and the closing one) the whole scenario has been designed to be played around 15 min to minimalize cybersickness occurrence probability (Yörük et al. 2018; Kolasinski 1995). In this scenario, there are also around 20 questions with a different level of complexity. In the classroom setting live audience can vote with a specially designed companion web app on the up to 4 questions that can be assigned to be asked by one of the avatars anytime in the game.

Scenario no 2. The second scenario describes a situation of the business idea pitch to potential investors. In this case, the learner can provide their presentation in PDF form, and it will be implemented into the game by the instructor. Thus the learner can see and interact with own presentation. In this, scenario game works in the challenge mode. In the challenge mode, the game does not show any scores on the player’s performance but records the presentation and all designated parameters.

Both scenarios feature some elements and measures for gameplay and learning effectiveness. The first and one of the mechanics of the most important game is the four dimension measure system. The game measures and in the mastery mode scores following four elements:

  • Sight direction – where the players look during the gameplay, measured in seconds by object or avatar;

  • Speaking speed – how fast is the player speaking, measured by the number of words per minute;

  • Speaking sound volume - how loud is the player speaking and if she modulates the voice, measured by decibels per millisecond – approx. Ten times per second.

  • Gestures – how often a player is gesticulating, measured by the distance between HMD and haptic virtual controllers in the player’s hands.

The virtual room setup and interactive avatars are the common features of both scenarios. The room setup, table placements, and avatar positioning pattern are the key factors in this design area (Fig. 2).

Fig. 2.
figure 2

Control panel view with real time preview of users actions in the background

Full size image

The key features of this room setup are the professional office setting for better immersion, the boardroom is in the high floor of downtown’s skyscraper, and from the windows, you can see other buildings. In the room, you find v-shaped table of the furniture and decoration, the position of the table and other elements in the room is designed with gaze positioning in mind, so no other object comes in the way. In this way, we achieve the ability to monitor the position of the players gaze without disturbing the game flow and immersion.

Second common design elements are interactive avatars. In total, up to 5 avatars spawn at the beginning of every each around the V-shaped table. The basic rule to the spawning point is an only non-obstructive view to each avatar so it can be seen clearly and the gaze mechanic measurement system can work seamlessly. The avatar models have both male and female versions (Fig. 3).

Fig. 3.
figure 3

Avatar models sitting at the conference table.

Full size image

The avatars can have different skin tones, hairstyles, eye color, and ethnic resemblance. The algorithm spawning the avatars randomizes the gender and ethnic origin with diversity in mind. It creates up to 5 avatars and places them at the table. Their clothes are always business casual, but cloth colors and details can vary. The avatars have assigned a fixed set of animations that can be applied both indecently (player independent reactions) and as visual feedback to the presenter (player dependent reactions) (Slater, Pertaub, Steed 1999). Each one of them also can ask the question, female and male versions of the questions audio has been recorded for this purpose.

The last common design mechanic in the game for both scenarios are distraction mechanic. Distractions are a series of audio-visual incidents that can be administered to the virtual environment by the instructor at random. There are two types of incidents. The first group is incidents based only on sound, e.g. a sudden fire alarm or police sirens. The second group is audio-visual incidents, e.g. a flock of loud birds is flying outside the window or someone opens the door and looks inside the room quickly etc. The role of the incidents is twofold. The first role is to create a distraction. The second is to create stronger immersion by stepping up the environment fidelity and prepare a player for such distractions in the real public speaking environment.

4 Learning Modules and Strategy

The VR technology has significant potential for learning, but it is critical how the game is positioned within the learning experience and how it mixes with physical reality for better learning effectiveness (Christopoulos et al. 2018; Bower et al. 2010; Hoshi et al. 2009).

Four different combinations of technology and learning can be pieced together: learning about technology, learning from technology, learning with technology and learning in technology (Schrader 2008). This approach is very similar to the traditional placement of simulation gaming placement within the learning space represented by Klabbers (2006) or Duke and Geurts (2004). In case of the VR game in this particular scope is critical how player communicates with the game. Players can move and move within the virtual space (Herbet et al. 2012; Hockey et al. 2010) they can speak, gaze and use hand gestures (Carter 2012; Hockey et al. 2010).

The optimal setting for the course has been designed with people who are beginners in public speaking or have little experience. The basic aim of the game-based learning program is to give the students basic knowledge of the public speaking and the necessary skills. The whole course is organized in the experiential learning methodology (Kolb and Kolb 2005). Before the beginning of the course, students receive two pages of instructions with information about the course and the game. In the instruction, they also find a link to the closed YouTube channel with video material. The video material contains seven video materials. First, two contain the presentation from the scenario no 1 performed by two professional public speaking experts (male and female). In the third and fourth video, they explain the presentation techniques they have used in the presentation. The fifth and sixth video material is devoted to the most important aspect of public speaking and their view on the presenter’s style and charisma. The last material is a short tutorial on how to play the CICERO VR game. Altogether, it is around 1 h of video material divided in the short videos of 7 to 15 min long. The students are asked to review this material before the game with special care to the presentation itself and the VR game tutorial. The instruction also contains the link to the copy of the presentation so they can train it before the game session.

In scenario no 1, players learn by example and by repeating the masters. In a scenario no 2 the players learn from feedback analysis of their performance. The game measures four player actions and gives visual feedback in the game and also presents all data in the performance report produced in the PDF file on the end of the gaming session. The sessions should be separated in time so the simulator fatigue will not increase the chance of cybersickness.

The icons presented in Table 1 appear in a HUD style with color code in the bottom of the HMD field of vision depth every time the player changes to the next slide, as the players control the pace of the presentation. The color-coded icons are the primary feedback system, which informs the player how he or she performed in the previous slide; they glow red or green for a few seconds. This secondary feedback system is avatar behavior. As mention in the game design section, each avatar comes with two sets of behavior manifested by the animations. First avatar behavior set is player independent and represents the basic set of human behavior. The purpose of this mechanic is to increase immersion and bring the avatars more to live, e.g. following the player with eyesight, looking at the phone or out the window, etc. The second set of avatar behavior is player performance dependent. Basically, the avatars will pay less attention and will be distracted more often if the performance is poor and more focused and engaged if the performance is high.

Table 1. Visual feedback icons in CICERO VR game.
Full size table

After the game session finishes, the system automatically creates a PDF file with generated data from the measurement and scoring system, and it can be sent to the player’s email (see Appendix 1).

Each playthrough should be accompanied by a debriefing session with the instructor. In the classroom setup player, group and instructor can analyze the data and talk about the performance of the players. After the gameplay analysis instructor assigns the players with reading, other exercises activities and lectures but in this setup can individualize the learning program based on the gathered data and gameplay video.

5 Research Methodology

Research of the created VR simulation was an important step not only because of improving user experience of our tool but also to generate data that could be later used as a benchmark for other simulations. Three research questions were stated:

  1. 1.

    How students perceive the current usability of the VR presentation training tool?

  2. 2.

    What are the design elements that can be improved?

  3. 3.

    What are the design elements that are missing?

Participants were chosen among university students, aged 19–24. We involved 36 people (15 female, 21 male), most of them (20 people) had experienced some form of VR before with PlayStation VR as a leading brand.

Each of the students had to prepare before the test by watching instructional videos and get familiar with presentation slides. Although there were some students that were just observers as the participation in the research was voluntary. After finishing the presentation in VR users were asked to complete a questionnaire. We have used modified version Game Experience Questionnaire (GEQ) which was created to examine UX of games and similar software (IJsselsteijn et al. 2013), which our VR presentation training seemed to fit better than into other tools (i.e. UEQ). GEQ is based on 3 modules: core questionnaire, social presence module, and post-game module. We have given up the social presence module due to no vivid interactions with in-game models at the moment. That will be needed in our regard in the next iteration of the simulation. Other modules were used in the original form of the questionnaire. In our case there were additional open questions regards better recognition of which elements are working good and which are missing.

The experience of asymmetric and asynchronous experience was not in the scope of this research. To make more reliable we decided to improve the VR experience first.

6 The Results

We used a modified version Game Experience Questionnaire (GEQ) for this research and thus the first analysis is focused on the reliability of the questionnaire and gathered data. Statistical analysis was based on Cronbach’s alpha test to assure the reliability of answers for each component (Table 2).

Table 2. The questionnaire reliability analysis.
Full size table

Reliability of answers for most components resulted as significant, which means that those components can be analyzed in the whole population of the study. The results show that we were able to reliably measure the Competence, Tension, Tiredness, Positive affect and Returning to reality also Negative affect with borderline significance (Fig. 4).

Fig. 4.
figure 4

GEQ components and average means

Full size image

Majority of the students felt competent while playing the game, which is a design intent logic. Moreover, from the internal correlation matrix (Pearson’s at p < 0,05) they were statistically significant correlated with Positive affect (Pearson’s 0,4703) and negatively correlated with Tension (Pearson’s −0,3582). Tension was also statistically significant correlated with the Returning to reality (Pearson’s 0,4510). Tiredness is statistically significant correlated to the Returning to reality (Pearson’s 0,3353), which is logical and points at the fact that cybersickness is an important and always present issue. We were not able to reliably measure the perception of the Sensory and Imaginative Immersion, Flow and Challenge. Thus our conclusions about these elements for the design have to be limited. The reasons being lack of reliability can be relatively low number of data or more refined questionnaire data, as we had to adapt the research tool that was not designed with VR serious games in mind.

For those components that were statistical significant, it is possible to conclude about components and answers for its questions among the investigated population. In Competence component, about 50% of users felt successful while doing the task and using their skills. Responses for Tension component questions have shown that 80% didn’t feel frustration and 77% were not irritated during the experience. From the post-game components, about 70% of users were satisfied of the experience (Positive affect) and about 60% felt tired or unsatisfied (Negative affect). Relatively high results in the last significant component – Returning to reality – where about 66% stated that they had disorientation and tiredness problems.

Three open questions stated at the end of the questionnaire has helped to understand better user experience and needs for making it better in future iterations of the tool. During the process of coding four categories emerged from the answers which are described as follows:

  1. 1.

    Visuals (positive/negative): This category contains how users were satisfied or unsatisfied about the visual side of the simulation. On the positive side, they have stated that they like the conference room and models or the view from the window. On the negatives – blurry image, problems with readability of the slides. That was something that may be hard to improve without changing headset technology into higher resolution in HMD.

  2. 2.

    Novelty (positive/negative): This category contains how users perceived novelty of the simulation and how they feel about it. On the positive side, they liked that new experience and possibilities that it has to offer. On the negatives, we found answers about the inconvenience of the device or unfamiliarity of the training process.

  3. 3.

    Immersion (positive/negative): This category contains how users feel a presence inside the simulation. Positive aspects they perceived were connected to audio cut-out from the rest of the room (via headphones and ambient noise) or realistic set and setting of the scene. Negative aspects that influenced immersion were distractions played by the instructor and dissonance between VR scene and real room – some were afraid that they will hit something while moving in virtual space.

  4. 4.

    Stress (positive/negative): This category contains perceived feelings about stress level of users during the simulation. On the positives, the students stated such opinions like the ability to challenge own presentation skills, less embarrassment when presenting in VR, less stress in the virtual room. Negative stress factors of the experience were on the contrary – some of the users were stressed because of the awareness that there are other people in the real room watching them during the simulation (Fig. 5).

    Fig. 5.
    figure 5

    Open questions coding and frequency distribution

    Full size image

Additional question about missing features resulted in a couple of interesting remarks about the quality of current models of the audience in VR. It was hard to read their reactions or body language and treat it as quick feedback for the presentation. Some students stated that adding more ways for interaction with between real audience and simulation user would enhance both experiences. That is another lead to explore during the next iterations that will be taken into consideration.

7 Limitations

Authors are aware of the limitations of described research. It was already stated that it is a pilot study of a rather novel use of VR in educational purposes and the knowledge about the proper way of conducting such study is still not grounded. The GEQ that we have used was created with entertainment games in mind, although it rather well worked with this case then some tweaks of this tools can be made to consider other factors of experience that are VR specific.

8 Conclusions

The VR game design process and implantation into learning experience produced a large number of interesting insights into the specific aspects of the VR technology, as well as, questions about effective implantation of the VR gaming technology in the curriculum (Shen et al. 2018).

VR technology allows game designers to create an environment they desire or need for the particular setup. In serious game design, we want to use this feature with the player and purpose in mind (Duke 1972). In this particular setup, the author chose to use VR technology in the way of mimicking the reality with additional gaming layer and by creating a safe space environment for users to play with (Kriz 2003).

Every technology comes with some limitations. The VR technology comes with limitations to hardware nature in the form of cybersickness when using HMD, while designing the serious game we have to keep in mind safety of the players and incorporate safety design and procedures while testing and playing. Instructors have to undergo safety training and follow the procedure of decreasing the potential for cybersickness. The soft limitation is the ability of the game to immerse the players and deliver a meaningful experience. This limitation works with the notion, that so far the VR technology is rather weak in transferring highly theoretical or complex knowledge and systems. However, we expect this to change in the coming years, as much more good quality games and solutions will appear on the market and in the research.

In the end, authors support the statement that serious VR games work in a most effective way in connection with the physical world (Christopoulos et al. 2018; Bower et al. 2010; Hoshi et al. 2009) through interactions and meaning. Giving the ability to include the player’s own presentation into the virtual environment was designed with that statement in mind.