1 Introduction

Virtual Reality (VR) is widely used in engineering education [1, 2], designing new complex mechanical systems [3], medical training applications [4, 5], prototyping of ergonomic workplaces [6] and advanced training and simulation systems [7, 8], among other things. Development of low-cost Virtual Reality interaction devices has been significant in recent years, which makes availability of hardware much higher than several years ago. These new devices (e.g. systems for tracking and gesture recognition or haptic devices [9]) allow users to directly interact with elements of a virtual scene in a more natural way and obtain a deep feeling of presence in an artificial environment. It is known as the immersion [10] and is one of the conditions that should be fulfilled when using Virtual Reality as a training tool.

In recent years, software previously used to create games was adopted to enable preparation of Virtual Environments. Popular game engines such as Unreal Engine or Unity 3D offer sets of plugins enabling integration of low-cost (consumer) VR hardware in any application. It makes it easier to create VR based games and training applications, but it also poses many new challenges.

This paper discusses examples of using low-cost devices to build immersive training systems and compares effectiveness of training, depending on specific peripheral devices. Two different hardware systems were built, both based on low-cost VR projection and interaction devices, one of them using standard components (the non-haptic approach) and the other – a custom made haptic device in form of a Delta robot. The same educational application was tested on both hardware setups, on a group of users and the results of this testing are presented in the final part of the paper.

2 Materials and Methods

2.1 Learning Transfer

Transfer of learning or learning transfer is the amount of knowledge gained during training that can be applied to a new task [11]. Learning transfer can occur only when there is task similarity or domain knowledge and a given person is able to perceive the similarity [12]. Currently available VR systems offer possibility to create a real-like environment and/or situation. Based on the assumption, that skills acquired in a virtual environment can be transferred to a real situation, virtual reality can offer many training benefits:

  • it enables simulation of difficult or dangerous work conditions or failures that cannot be reproduced with real equipment, thus enabling learning skills in safe conditions and avoiding downtime or damage of equipment [13];

  • it allows making mistakes while learning procedural tasks, without any real consequences, what is particularly important in high-risk situations, such as airplane flying [14];

  • it enables avoiding downtime of machines [15].

2.2 Virtual Reality in Industrial Training Applications

In industry, training of operators to use complex technical systems is an important source of expenditure. VR systems can be effective tools used in industrial training, reducing costs significantly. Analyzing the potential of this type of solutions, with particular regard to tactile interaction that occurs between the user and elements of the virtual environment, one can observe two approaches in its implementation. The immersive approach assumes use of stereoscopic visualization systems, which ensure a high level of immersion when used together with tracking systems. The downside of this solution is lack of a haptic feedback. In the immersive approach, devices for Rapid Prototyping (mostly via 3D printing) are increasingly used, thanks to which physical representations of digital models that are necessary during virtual simulation (e.g. models of hand tools, control panels) can be prepared [16, 17]. This increases the degree of immersion, but lack of the tactile stimulus limits the interaction with the user. The expected degree of tactile interaction can be provided by a haptic approach, which assumes the use of a manipulator with force feedback effect. In this approach, RP solutions are also used [18].

As a result of the analysis of existing approaches, it was noticed that the level of effectiveness of the interactive training simulation is determined by immersion (which can be achieved by capability of manipulation of virtual objects interactively) and tactile interaction. It is worth mentioning, that commercial solutions integrate immersive and haptic VR equipment, while allowing the manipulation of physical models (representations of digital data) to a small extent [19,20,21,22,23].

In the context of the above-mentioned disadvantages and limitations of existing solutions, as well as on the basis of the authors’ own experience resulting from the implementation of research work [9, 18], it has become justified to develop similar but alternative to the haptic approach and comparing it with a solution based on non-contact interaction techniques.

2.3 Low-Cost VR Devices

A set of low-cost projection and interaction devices was used to build a demonstration station designed for tactile interaction studies. Proposed solution is based on a consumer HMD - Oculus Rift CV1 (parameters presented in Table 1). The HMD itself was used in both hardware setups. Also, a pair of dedicated controllers (Oculus Touch) were used for the non – haptic approach.

Table 1. Parameters of Oculus Rift CV1
Full size table

The optical PST-55 tracking system was used in the second hardware setup (the haptic approach), for interaction testing. It is a set consisting of a device for emission and detecting reflected infrared light, a camera registering visible light, and markers placed on a tracked object. The most important parameters of device are presented in Table 2.

Table 2. Parameters of PST-55
Full size table

The solution is based on passive markers, which are used determine the position and tracking of objects in the distance of about 0.4–3.2 [m] from the device [24]. For each object to be recognized by the system, an appropriate, unique, constant marker arrangement must be recorded into the tracking system software, using an appropriate procedure. The PST system enables to track linear movements in all three axes and object rotation relative to each axis.

2.4 Low-Cost Haptic Device

The authors have prepared a solution based on the so-called an active touch device, whose role is played by a manipulator with a parallel kinematic structure (Delta type robot). The main task of the robot, as an alternative to the currently used haptics, is to simulate the shape of digital objects, and thus provide user with a tactile stimulus during the interaction [9]. Introducing a new element to a virtual environment - an active touch device – is an approach that has not been used in the creation of VR training systems so far. In comparison to existing solutions, use of the robot also provides considerable freedom of movement for the user (available systems are based on a permanent contact of user’s hand with the working tip of the haptic manipulator, which has a small working range). Design of the device is based on kinematics of a stationary industrial robot of the so-called parallel structure 3 (RPR) [25] is crucial in a proposed system designed for touch interaction research in Virtual Reality. The basic design assumptions (including number of degrees of freedom, ranges of motion, forces, speed and positioning accuracy of the end effector) of the robot were developed as part of the research project VISIONAIR [26]. It was assumed that the working area of the device will be about 600 × 600 × 600 [mm], the maximum speed in individual axes will not be higher than 100 [mm/s], and the maximum generated force will not exceed 10 [N].

On the basis of the mentioned assumptions, the drives, measuring and controlling elements of the manipulator were selected. Synchronous motors with permanent magnets PMSM were used as drivers. They were connected to the gear and coupled to a resolver. The robot is shown in Fig. 1.

Fig. 1.
figure 1

The Delta robot [27]

Full size image

Due to the fact, that the device was to carry out tasks in direct contact with the user, during its design, a special emphasis was put on the security issues (including flexible arms). The factors that decided about choosing mentioned device for the developed concept of the tactile interaction testing system were:

  • costs (low-cost solution);

  • motion range (sufficient to implement the assumed research scenarios);

  • constant orientation of the end effector of the robot (use of three drives in theory ensured maintaining horizontal position of the working tip, which was a requirement of the designed VR application).

The proper configuration [9] of the manipulator, thanks to which it was possible to use it effectively as an active touch device simulating digital objects, was also taken into account.

2.5 Tracked Objects

In addition to the Delta robot, physical models (having their digital representations in VR simulation) were used to deliver the tactile sensation when interacting with elements of the virtual scene. Physical representation of the digital data was crucial from the point of view of the developed VR system for tactile interaction studies. The real, physical models were made on the basis of previously prepared 3D CAD models using additive manufacturing technology of Fused Deposition Modeling (FDM). For the purpose of the experiment, physical models of shafts, a simplified assembly for mounting the shafts and a bracelet enabling tracking of user’s hand were made (Fig. 2), they were then implemented in an integrated test stand (equipped with markers to track their position, marker arrangement recorded in the PST system software).

Fig. 2.
figure 2

Physical models used in simulation

Full size image

2.6 VR Application

The main aim of the research was to check if proposed solutions can be an effective tool for training the activities performed at the production site. The software application was prepared in the Unity 3D programming environment. Bearing in mind the research scenarios (see chapter 3), the following digital models have been placed in the virtual scene (Fig. 3):

Fig. 3.
figure 3

Objects placed in the virtual scene

Full size image

  • two tables (built from geometric primitives inside the Unity 3D engine);

  • shafts and simplified assembly for mounting them (prepared in 3D CAD environment);

  • hand (used in the haptic approach)

Custom scripts have been assigned to both the virtual hand and individual shafts, to allow them to move along their physical counterparts (based on data from the tracking system, sent by the UDP protocol). In each slot of the assembly, a collider object has been assigned. The role of these colliders is to check if shafts were placed in correct slots. The general colliders were placed in the scene, to allow physics-based object interactions with the environment.

3 Test Procedure

3.1 Test Scenario

The experimental research was aimed at checking how much delivery the tactile sensations (with use of various interaction devices) during interaction of a user with elements of the virtual world will affect immersion of the application. In case of the haptic approach, the objects were simulated by the end effector of the touch device and physical models. In case of the non-haptic approach, only the Touch controllers were used. The experiment was a modified “peg-in-a-hole” procedural assembly task (Fig. 4).

Fig. 4.
figure 4

Procedural task, top – virtual simulation, bottom – physical models

Full size image

There are 6 shafts in various colors on the virtual table number 1. In the haptic approach, it has its representation in the form of a physical table, on which real six shafts models are placed. On the virtual table number 2, there is a simplified assembly for mounting the shafts (a box with six holes with chamfered edges). In the haptic approach, its physical model with a simplified construction was placed at the end effector of the touch device. It was a real box model with one opening. Task of a user is to move shafts of particular color from the table No. 1 and put them in a specific hole in the assembly. The task of the robot in the haptic approach is to move to a position that allows placing the shaft in a specific hole of the assembly (Fig. 5). In this case, as it is a prototype system, a human operator is necessary- his task is to pull out the physical shaft model from the unit installed at the top of the end effector.

Fig. 5.
figure 5

Research stand for the virtual training of procedural task

Full size image

In case of the non-haptic approach, based on the Oculus Touch controllers, only the virtual models were used. The user, seeing his virtual hands while holding the controllers is supposed to grab virtual shafts, then press the specific “grasp” button on the controller (a grabbed object is shown in Fig. 6). The next step is to move a virtual shaft to an appropriate position, while avoiding collisions with elements of the virtual environment. After placing a virtual model of the shaft in a correct position, it is necessary to release the button on the controller in order to release the virtual hand and place the shaft in the hole.

Fig. 6.
figure 6

Grabbed object being transported from Table 1 to Table 2 with Oculus Touch (a - virtual simulation, b – real stand)

Full size image

Apart from both VR-based approaches (haptic and non-haptic), a physical work stand was also prepared, for the control group (Fig. 7). It consisted of additively manufactured, plastic parts and cardboard elements.

Fig. 7.
figure 7

The physical stand for training

Full size image

3.2 Test Groups

The tests were aimed at determination of degree of effectiveness of training. Four groups of users were tested (15 persons in each group):

  • Group 1 (control group) - persons who have joined the test at the physical stand after 1 min analysis of assembly instructions (the shafts of a particular color in dedicated holes in the assembly were specified) and 4 min of training at the real stand;

  • Group 2 - persons commencing the test at the physical stand after 5 min of training completed on the Delta robot and physical models research stand (the haptic approach);

  • Group 3 - persons joining the test at the physical stand after 5 min of training using Oculus Touch (the non-haptic approach).

3.3 Collected Data

Each participant in each group carried out five successive attempts to complete the procedural task. The following data was collected for the purpose of evaluating the effectiveness of the training:

  • total time of completing the task (in each trial);

  • total number of errors (in each sample and total in 5 attempts);

  • number of attempts without errors;

  • number of attempts with errors;

  • number of times when participants asked for help or a graphical hint (in each attempt and in total in 5 attempts).

What was recognized as an error:

  • grabbing a part of a particular color in the wrong order;

  • placing part of a specific color in the wrong assembly hole.

If a user made an error while performing the experiment, he was immediately informed of that fact by the operator, who recorded the error. At the same time, the participant was informed (in the form of a voice command) which part should be taken (in accordance with the current sequence) or where the part should be placed in the assembly. Hints were displayed in form of graphic prompts (as shown in Fig. 8) - place of the target assembly of the selected part was displayed, depending on the color of the part. In case of such a situation, the time of the task being performed was not stopped.

Fig. 8.
figure 8

Graphical helper indicating where the shaft should be placed

Full size image

After the test at the real stand, the participants of groups 2 and 3 were asked to perform the procedural task one more time in both virtual environments using appropriate sets of devices.

4 Results

Total 45 users took part in the experiments described above. The results (Fig. 9) show that participants of Group 1 performed the best. They completed the procedural task quickly and made the smallest number of errors. The ability to practice on a dedicated physical stand has proved to be a key factor. Worse results were obtained by representatives of Group 2. They completed the task in 10% more time, making more mistakes. Very similar but slightly worse results were obtained by representatives of Group 3. Here, the decisive factor was contact with physical objects - participants knew the correct sequence but for the first time they had to practically put the part in the assembly.

Fig. 9.
figure 9

Results of the test

Full size image

For comparison, representatives of Group 2 and 3 were also asked to perform a measured attempt to perform a procedural task in a virtual environment. Results (Fig. 10) show that using non-contact interaction techniques, users performed the task faster - this may be related to the fact that there was no need to capture physical objects and put them in the assembly, but only “virtual transport” using controllers - restrictions related to precise placement of parts in the assembly were vastly reduced. Users were able to learn the correct sequence, but they did not acquire manual skills. However, when it comes to number of errors and requests for help, the results are very similar. The learning curve of the procedural task changes similarly for all the test groups - progression is observed with each subsequent try.

Fig. 10.
figure 10

Performance of procedural task in VE

Full size image

5 Conclusions

VR solutions are a viable option for training in the field of performing a procedural task. The purpose of the test was to show how the touch affects the immersion of simulation and effectiveness of the learning process of the procedural task. The results are clear - training using both the proposed solutions significantly improves results (fewer mistakes and requests for help) in relation to the group, which had only a traditional instruction. It is worth noting, however, that the discrepancies between the proposed two VR solutions are relatively insignificant. The authors assume that with the increasing complexity of the procedural task, the difference between traditional and virtual training will deepen. It is also worth paying attention to the fact that the Oculus Touch controllers are equipped with motors that enable them to vibrate at selected moments. To a limited extent, this may imitate the tactile impression or at least give additional feedback to a trainee, e.g. if the device starts to vibrate at the time of collision between virtual elements. In the future, it is planned to examine how increasing the complexity of the task and using the controller’s vibrations affect the difference between the proposed methods of VR training.

It is worth noting that the assembled elements were lightweight. In the event that the assembly task would involve parts of significant mass (e.g. if the shafts were made of steel), a person after virtual training (using the Oculus Touch) could feel uncomfortable after starting the operation on a real object. It would result from the difference between the experience gained during the training and the reality. The authors plan to investigate this issue and its impact on the effectiveness of the proposed methods of training as part of further research.