Keywords

1 Introduction: Virtual Reality as Training Tool

The use of Virtual Reality technology makes it possible to acquire and practice the workers’ correct actions even in emergency situations. This is done in a controlled and safe circumstances. The visuals used in the virtual environment keep the trainees interested and facilitates the memorization of information as well as consolidates their skills. The training system will allow to simulate different conditions in hazardous situations. Using the VR (Virtual Reality) technologies will enable observing the trainees’ behavior in stressful situations.

Efficacy of new technologies can be further enhanced thanks to the experience in the field of gamification, research processes and training. Gamification – this term refers to the mechanics of computer games which are used to modify people’s behavior in tasks which aren’t related to entertainment or games. Worldwide research shows that the use of gaming elements in a training environment helps improve the usability experience and participants’ commitment. For some time, the gaming technology are used for a much wider range of applications rather than just entertainment. Evidence of such trend shows a growing market for so-called serous games, which are applications-like games, but used for research or training purposes. The newest psychological research analysis shows that playing computer games and using game-like training applications improves the cognitive functioning of individuals (e.g. increasing the individuals attention). These results are in agreement with reports on the effect of games on the cognitive functioning (it is related with improvement of visual attention performance, especially for first-person perspective games). It has been proven that the effectiveness of such applications supports gaining knowledge and skills. Gamification became a method for creating more engagement in work environment, whose main feature is natural and facilitated cooperation with other users [1, 2].

One of the most frequent causes of accidents in the industry (e.g. in mining) is the lack of workers’ attention. The complexity of the environment in which people (especially miners) work, has an extreme burden on cognitive processes. Limited visibility, high temperature and humidity, the necessity of wearing protective clothing and equipment are conditions in which miners are pushed to the limit. In such a situation, some tasks can be easily overlooked, which may result in an accident. For the past 10 years, many studies show that people with gaming experience are more successful with achieving cognitive tasks [3]. This applies especially to the visual attention and the ability to switch to different tasks. It is particularly interesting what the experimental studies show. Trainees can improve their attention just after relatively short trainings. The improvements last for several weeks or even months after the training [4, 5].

Positive effects of gamification are also visible in the area of improving the ability to perform many tasks simultaneously [6] and cognitive skills, especially in the case of elderly [7]. Therefore, interactive games and virtual environments are increasingly used for cognitive rehabilitation in order to improve cognitive abilities and resources of various groups of people, including children [8] or people after stroke and other brain injuries [9, 10]. Rehabilitation activities based on computer simulations have the potential to provide patients with enriched training environments that motivate to a longer, heavier and regular activity aimed at overcoming subsequent challenges in the field of motor control. Particularly good adaptation to the defined requirements of effective rehabilitation are achieved by systems using virtual reality technologies. The reason for this is the possibility of stimulating many senses of the patient within the simulation: sight, sound, sensory experience. Rehabilitation with the use of immersive virtual environments, based on the experience of conventional rehabilitation techniques, provides tools that intensively stimulate patients and more effectively maintain motivation for exercise. An important aspect of the assessment of the impact of physical training in virtual environments, on changes in behaviors observed in the natural environment, is the determination of the correctness of kinematics of movements performed in the context of 2D interfaces and more immersive 3D solutions. The improvement of the motor performance of the upper limbs is most often measured by the parameters of speed, precision, smoothness of movements and the degree of coordination between changes in the position of individual components of the musculoskeletal system [25]. Movements of upper limbs performed in the context of three-dimensional interfaces are more similar to natural ones than the same movements realized using two-dimensional interfaces (in our work we are using VR-equipment to track upper limb movement in three dimensional space, but the image of the virtual environment can be presented in HMD, as a stereoscopic image, or on the two dimensional screen – see Fig. 1). This is evidenced by the studies in which the kinematics of the movements of indicating various points in the field of the arm’s work in the natural environment were compared to movements performed in the immersive virtual environment presented in the HMD [26, 27]. Virtual environments allow for precise measurement of behavior parameters while performing complex but safe tasks in simulations, the realism of which is similar to real situations [28, 29]. In many solutions based on virtual reality technologies and computer games, feedback is provided by both visual and auditory stimuli. In the case of upper limb rehabilitation, users can interact with virtual objects directly through the hand and body movement, through interfaces that enable sensory feedback, performing activities that give the impression of immersion in simulation, which increases the probability of transfer of learning outcomes to natural situations [30].

Fig. 1.
figure 1

An image of a virtual environment can be presented in many ways, e.g. on the screen or in the HMD. The choice of the display method and image depends on the type of training scenario, e.g. it is not possible to force all trajectories of the upper limb movement by presenting only a two-dimensional image.

Full size image

Virtual reality is often used as a training tool, and also in this study we are using virtual reality-based training methods. It is assumed that immersive simulations in virtual environment enhance trainee ability to modify inefficient and false working procedures [11], mainly because the support acquisition and transfer of tacit knowledge and enhance human abilities and motivation to absorb new knowledge [11]. Results of many studies show that training is effective in reducing accidents at work and this effect is increased by the level of engagement of the training methods [12] and the virtual reality is one of the most engaging and immersive training method. A review of 95 quasi-experimental studies was performed to determine the relative effectiveness of different methods of worker safety and health training aimed at improving safety knowledge and performance and reducing negative outcomes (accidents, illnesses, and injuries) [13]. The main outcome of this study is following: as training methods became more engaging (i.e., requiring trainees’ active participation), workers demonstrated greater knowledge acquisition, and reductions were seen in accidents, illnesses, and injuries [13]. Results of virtual reality-based pilot training of underground coal miners [14] show that in almost all cases high immersive VR, especially combined with wide FoV, is assessed as better solution for training, than less immersive training methods. Moreover, trainees consider used system useful and feel the positive effects of training even after three months [14].

It should be noted that virtual reality is only a tool and in some cases it is not working as it is supposed to be. At the current stage of technology it is impossible to use virtual reality to all possible training scenarios and always achieve expected results. It was even more complicated in the past with big, heavy equipment, especially head mounted displays and motion tracking systems. Some studies show lack of transfer of positive effects of virtual reality training. For instance in [15] real-world training, virtual reality training, and no training in the transfer of learning to the same task performed in real-world conditions were compared. The results show that there was no significant difference between the virtual reality training group and the group that received no training on the task, indicating that improvements to virtual reality for the purpose of enabling the transfer of training are needed. Moreover it is still difficult to train manual tasks due to poor mechanical performance of the simulated haptic feedback [16]. In some cases problems with quality of simulation may lead to learn improper practices and as a result control group achieve better rating scores than group trained with the simulator [16]. However in most cases positive effects of virtual reality training is clearly visible, e.g. the use of virtual reality surgical simulation significantly improved the operating room performance of residents during laparoscopic cholecystectomy [17]. Outcomes of 10 laparoscopic cholecystectomies performed by novices shows that The VR-trained group consistently made significantly fewer errors. Moreover, residents in the control group made, on average, 3 times as many errors and used 58% longer surgical time [18]. It indicates that virtual reality based simulation significantly improves performance and error rate. Results of systematic review of 91 articles, which all reported on training in arthroscopic surgery, shows that VR training leads to an improvement of technical skills in orthopaedic surgery [19]. Interactive virtual reality training was found to be a relatively inexpensive and effective mode for teaching operating room fire prevention and management scenarios [20]. Even if both groups have reached the same level of cognitive knowledge, when tested in the mock operating room fire scenario, 70% of the simulation group subjects were able to perform the correct sequence of steps in extinguishing the simulated fire whereas only 20% subjects in the control group were able to do so. The simulation group was better than control group in correctly identifying the oxidizer and ignition source [20]. Virtual reality is also used in many safety training applications [21, 22] and military applications [23]. In the case of soldiers training the stress during training is an important factor [23] and it is possible to lower the stress level by proper (even virtual) training. It should be noted that virtual reality has potential not only in the case of workers training to increase safety and work efficiency. It is also useful to simulate of different potentially dangerous events. Virtual reality seems to be advantageous for observations of near misses and accidents at work. Use of VR for accident investigations facilitate new insights into cause of events and thereby provide new perspectives for the development of measures of hazard and risk reduction [24].

2 Methods and Results

The implementation of an effective training system requires precise preparation of assumptions regarding the scope of functionality of the products planned to be implemented. In the case of people working in dangerous conditions, the key task is also to describe the specificity of the work environment to be transferred to virtual reality. For this reason, the assumptions for the training scenarios were first developed. In this way, the process of training simulators was defined. The source data necessary for the simulation of the work process of self-propelled mining machinery operators was collected and analyzed. These data included technical, photographic, film documentation and 3D scans.

The principle of operation of the training and examination system is be based on the use of a hybrid motion tracking system allowing to determine the movement of the entire figure of people participating in the simulation, as well as the location and orientation of any items in space (e.g. physical surrogates of virtual object can be used). HMDs (Head Mounted Display) are used to observe the virtual environment, wireless gloves allow full interaction with the virtual environment (e.g. moving virtual objects). The belt with battery for wireless data transmission system is also used. The analysis of data from the motion tracking system (which is based on passive markers) allows to determine the position of the entire body, i.e. the position and orientation of all limbs, torso and head, with particular attention to the head and hands. Special gloves monitoring the degree of flexion of the fingers allow for advanced interaction with the environment, e.g. turning machines on or off, gripping, moving and throwing virtual objects. The use of commanding gestures is also possible. Immersive interaction with virtual environment is possible because the collected data is transferred to a specially adapted physics engine operating in real time.

The use of the wireless system (Fig. 2) allows the trainee to freely explore the virtual environment, the size of which does not have to be limited to the size of the room in which the training or examination takes place. The trainee can go, and even run, in any direction. This approach gives a unique opportunity to build scenarios such as exploring a large area. A redirected walking algorithm will allow free movement even in very large virtual environments. This is possible due to the domination of the sense of sight over the sense of proprioception associated with the sense of touch and the vestibular system, which allows a certain degree of invisible manipulation of the direction of human motion while maintaining the conviction of a person immersed in the simulation of maintaining the chosen direction.

Fig. 2.
figure 2

For the training a wireless HMD and gloves are used. Since the heavy machinery chamber has a very large surface, and mining machines such as a loader have a length of about 10 m, it is advisable to use sufficiently large training rooms. Equipped with a vision based motion capture system, the room shown in the picture is 16 × 11 m.

Full size image

Software related to the implementation of training scenarios is closely related to the functioning of a ICT tool to support the assessment of the course of the training. All important events and decisions made by the trainee will be recorded in the central database. The software prepared for this purpose will enable management of this data, and above all their display and description using a web browser (i.e. a thin-client application paradigm will be used). Authorized persons will have access to the collected data, which can be used, among others to assess changes in the knowledge and skills of trained people. A simplified diagram of the training course is presented in Fig. 3. Using such ICT it is possible to easily change training scenarios and adapt them to the needs and capabilities of the trainees (this is especially important in the case of training games supporting rehabilitation process). Users of the system have easy access to almost all that through a web browser (diagram of data transmission is shown in Fig. 4). The accessible functionalities depend on user role, e.g. the training scenario can be changed only by a trainer.

Fig. 3.
figure 3

A simplified diagram of the training course.

Full size image
Fig. 4.
figure 4

Diagram of data transmission between different users of the supporting the training process system.

Full size image

The described above methods are used to support the rehabilitation and physiotherapy process in order to accelerate the return to work after injuries. Such training is a particularly necessary for people working in dangerous conditions (e.g. in underground mines), where the accident rate is clearly higher than in other branches of the economy. It should be noted that the same VR-technology combined with ICT tools supporting training process is used to teach employees how to do their job safely, and this is also particularly necessary for people working in dangerous conditions. We are preparing training applications for miners working in different types of underground mines (e.g. coal mines and copper mines – see Fig. 5) as well as firefighter (Fig. 6) were training of cooperation in a team is very important.

Fig. 5.
figure 5

Training application for miners working in underground coal mine (left) and a 3D model of one of the mining machine in underground copper mine. A control panel is visible through the glass (right).

Full size image
Fig. 6.
figure 6

Virtual reality with motion capture technology allow to train group of people. Such cooperation training is very important in the case of people working in dangerous conditions, like firefighters. On the right side the use of physical surrogates of virtual objects is visible. Trainee can use real nozzle to change properties of the water stream.

Full size image

3 Summary

Evaluation of the effectiveness of solutions based on virtual reality technologies as an effective tool for acquiring mobility skills requires evidence of effective transfer of acquired skills to behavior in the natural environment. Nowadays, only a few studies control important indicators of learning effectiveness, such as long-term maintenance of increased mobility, transfer of training effects and their generalization to untrained tasks [31]. In one of the studies of this type, the relationship between exercises performed using the interface in the form of a specialist glove has been proven and overall improvement in the efficiency of functioning outside the virtual environment, but undoubtedly more research is needed on the issue of the generalization of training effects. Another example of the transfer is a study [32], in which after training in a virtual environment, patients with side-shedding syndrome coped better with the task of crossing the street in real situations, despite the lack of significant differences in other range tests and the degree of deficit.

Despite the considerable diversity of individual studies in the field of correctness of movements performed in virtual environments (2D vs. 3D interfaces), a suggestion emerges that the movements performed in the simulation may translate into significant changes in everyday functioning [33]. Therefore, it can be assumed that it is not essential that the movement in the simulation reflects the kinematics of movement in the natural environment, nor is the role of the natural rotational behavior of individual joints necessary for the movement. The potential for transfer of skills learned in virtual environments may be related to other advantages possible thanks to virtual reality technologies.

The described challenges and the lack of unambiguous evidence on the effectiveness of the transfer of training effects remain an important area of research on the possibility of using virtual environments in training and rehabilitation. A separate problem is the further integration of these solutions in clinical practice, because rehabilitation tools in the first place must serve the effectiveness of the activities of professionals choosing the training variants of each patient.

In summary, there is a lot of evidence that it is possible to effectively use virtual reality technologies to create enriched training environments in accordance with established principles of physical control and movement learning, to support upper limb rehabilitation processes. The flexibility of computerized solutions gives therapists the opportunity to precisely determine the goals and scope of the exercise, while providing feedback that effectively induces neuroplasticity and motivates patients.

The possibility of real and practical application of the project into the educational process of people working in dangerous conditions will have a significant impact on improving collective and individual work safety among people identified as target groups. It should be emphasized that this does not apply only to people subjected for training. The long-term effect of it would be to increase the safety of all employees. The results of global research on the effectiveness of training show that trained staff have a positive impact on the people in their environment who did not participate in the training. In this way the knowledge is transferred naturally among people who work together. It will also affect the level of their own potential to improve knowledge of the diversity of threats and the ability to react to a specific threat through acquired skills and competences. In real work environment it is not possible to carry out a number of simulation events. It results from the organization and labor productivity, the cost-efficiency requirements for employees, as well as the production cycle that takes into account the natural movement of the rock mass. These restrictions do not apply to VR technology.

It should be noted that all training scenarios are fully integrated with ICT training assessment system. This is a clear step ahead to a fully comprehensive approach to a complex training which is based on knowledge and skills which come directly from experience and not from a theoretical course.