Abstract
With an ever-increasing need of skilled healthcare workers, efficient learning methods like Virtual Reality (VR) are becoming increasingly important. We developed and tested a VR simulation for endotracheal suctioning.
The aim of this pilot study was to examine the VR simulation’s acceptance and increase of knowledge among participants. Furthermore, the effects of presence on acceptance and increase of knowledge were investigated.
A total of 51 students participated in the pilot study, using a quasi-experimental pre-post-test design. A modified Unified Theory of Acceptance and Use of Technology (UTAUT) and the Igroup Presence Questionnaire (IPQ) were used. Correlation and regression analyses were performed. Pre- and post-tests showed a significant increase of knowledge (p < 0.001). The correlation between presence and behavioural intention was highly positive (r = 0.52, p < 0.001).
Performance and effort expectancy are dominant effects on behavioural intention of using the VR simulation as an educational tool. The results indicate that a simulation which conveys a higher sense of presence is more likely to be accepted by learners. Regarding outcomes of presence on increase of knowledge, we found no significant correlation. Based on our study, we propose a design for a future mixed reality simulation with haptic elements and a plan on how to assess skills improvement.
1 Introduction
Like in many other countries, the number of people in need of in-patient and home care in Germany has been rising steadily for decades [1]. Since the number of personnel is not increasing accordingly there is an emerging shortage of care workers [1], [2]. Inevitably, this leads to a higher demand in efficient education of nurses on a large scale. Apart from theoretical knowledge, the mastering of complex clinical skills is a crucial aspect of training competent practitioners. One example is endotracheal (ET) suctioning, which has been shown to be particularly stressful in a home ventilation environment [3]. If nursing staff can perform suction well relatives and patients feel safe [4]. We therefore chose to develop and test a virtual reality (VR) simulation for learning the ET suctioning procedure.
The main aim of the study was to deliver proof of concept for our VR simulation, by (i) looking at feasibility and (ii) demonstrating that VR can successfully improve a learner’s knowledge of nursing procedures. We conducted a pilot study using a quasi-experimental pre-post-test design with 51 health care physiotherapy students to assess the participants’ acceptance of the VR simulation, and whether it achieved a significant increase in clinical knowledge. Furthermore, we investigated whether the sense of presence – a feeling that users of virtual reality experience while being in the virtual environment – had an effect on the increase of knowledge and on behavioural intention. According to the unified theory of acceptance and use of technology (UTAUT), behavioural intention is one step in the process of acceptance of a new technology and eventually using it [5].
In the following chapters we will describe the current practice of nursing education in Germany with regards to simulation and skills training; discuss definitions of VR, the sense of presence and relevant learning theories; and explain educational outcomes like acceptance, knowledge and skills and how to assess them. Section 3 describes our use case, objectives and hypotheses. After providing details on materials and methods, we list the results and finally discuss them in Section 6.
1.1 Simulation Based Learning in Nursing Education
In Germany, nursing is a hospital-based apprenticeship where most of the learning of clinical skills takes place on the wards. In view of the tight staff situation in health care, the practical instruction of students can easily become a burden. New skills requiring detailed and lengthy instructions are often only practised to some extent, if at all. This results in a lack of readiness and competencies among graduating nurses, which has a negative impact on the quality of care, and can lead to life-critical errors [6].
Simulation-based learning is an established method in nursing education that is backed up by a large number of studies including systematic reviews [7], [8], [9], [10]. This research defines simulation types and attributes, and educational outcomes that can be achieved [8], [9], [10]. Simulation types include low- and high-fidelity mannequins, virtual patient simulations, lifelike virtual environments such as skills-labs, simulation wards and role-playing or simulated patients [10]. Simulations can be categorised according to their fidelity, which describes their degree of realism. A medium to large effect size for various educational outcomes suggests that simulation-based learning in nursing education is overall highly effective [10].
Despite the positive effects of simulation in nurse education, there are some limitations. High-fidelity mannequins and skills-labs are expensive and sometimes used only for presentation purposes [11]. Moreover, learners and practitioners still report a gap between practice and theory [12]. One way to address these issues could be virtual reality.
1.2 VR Skills Training in Nursing Education
Virtual Reality (VR) simulators in medical technology research are widespread and there are even ISO-standards for validation [13]. A survey of 31 studies with overall 2407 participants found evidence suggesting that VR education improves post intervention knowledge and skills outcomes of health professionals when compared to traditional education or other types of online or offline digital education [14].
Compared to medical training, VR is a relatively new research topic in nursing education. Reviews have found that there are not enough studies with high quality design to estimate an evident effect of VR on various educational outcomes [15], [16], [17]. Moreover, these reviews investigated simulations that did not cover immersive VR simulations but included non-immersive 2D-screen simulations [18], leading to even less evidence about the effects of immersive virtual reality in nurse education.
Despite that, there has been a steady increase in the development and evaluation of immersive VR simulations with innovative designs ever since the first commercial head mounted displays were launched in 2014 [17], [19]: it was, for example, used as an effective training modality for empathy. In this simulation the learner experiences the condition of dementia first hand [20]. The user has to accomplish basic household tasks, such as cooking. However, the objects that are required for the tasks, sometimes disappear from their location and appear in unexpected places [20]. Another example for affective training, is a VR simulation, where the user’s assignment was to evacuate a neonate from a ward in an emergency scenario. The developers utilised the potential of VR to create fictional life-like situations that can be highly stressful to prepare learners for real-world worst-case-scenarios [21].
Some simulations used haptic devices to enhance the degree of realism in their VR simulations, and to enable learners to train technical skills [22], [23]. One simulation employed a device that is similar to a stethoscope in its handling [22]. When users moved it near the heart region of the virtual patient they heard respective sounds through their headphones, like they would when using a real stethoscope [22]. Another simulation used a force-feedback device that was attached to the wrist of the user to simulate the resistance one would feel when inserting an endotracheal catheter into a patient [23]. Additionally, the simulation also provided advanced virtual guidance by showing imaginary hands, that the learner could follow and copy their movements in real-time [23].
In order to address some of the existing issues of simulation in nursing education, Virtual Reality (VR) simulation technology might provide an alternative. It could help students learn more efficiently and more practically. Compared to skills-labs, VR is more affordable, requires fewer resources and is not location- or time-bound while still providing the benefits of low risk and low anxiety learning [24]. Furthermore, VR can offer contactless learning without a teacher by giving objective and direct virtual guidance and feedback. Due to its immersive capabilities, VR has the potential to create fictional life-like simulations that could be used to teach empathy [25], self-confidence [26] or train psychomotor skills through special haptic devices [22].
2 Background
2.1 Learning Theories Behind VR Based Education
Virtual Reality can be described as a computer-generated reality, that allows learners to experience various auditory and visual stimuli experienced through specialised ear and eyewear [27]. Across several publications, three learning theories are repeatedly referenced and provide a theoretical basis for VR based education [28], [29], [30].
Firstly, there is situated learning: Many teaching practices implicitly assume that knowledge can be abstracted from the situations in which it is learned and used [28]. However, according to the situated learning theory, learning is placed into a context and an environment. Rather than having learners transfer abstract knowledge from classrooms into practice, learning should be embedded within the activity, context and culture in which it occurs [31].
Secondly, there is the constructivist philosophy which proposes that knowledge is created through an individual’s interaction with the environment, in a learning-by-doing fashion [28]. Learners take an active role in their learning, since they not only absorb information, but also connect it with previous knowledge to construct their new knowledge [32].
Finally, in implicit learning, learners acquire knowledge or skills without being aware of what they are learning or being able to communicate what they have learned [29]. This happens, for example, in childhood when learning language and grammar. Grammatical rules can be applied, but no reasoning can be given as to why a sentence is grammatically correct or incorrect. Implicitly learned skills and abilities such as language or riding a bicycle tend to be too complex to articulate verbally.
All three theories have in common, that active situations are in the foreground and support a learning-by-doing approach rather than a theory-first-then-practice approach.
2.2 Presence and Learning in VR
In order to further understand learning with VR, the concepts of immersion and presence are important [33]. Immersive VR simulation requires the use of a head-mounted display [16]. Immersion is defined as the psychological reaction to a virtual environment [34]. Depending on the types and amounts of sensory stimuli provided by the VR technology participants experience various degrees of immersion. Therefore, the more senses that are accommodated by the simulation the higher the levels of immersion for the participant [34], [35], [36]. Presence or sense of presence means the subjective experience of being in a place or environment despite being physically situated in another. In general, simulations that provide higher degrees of immersion result in higher levels of presence [37].
We found no consensus regarding the effect of presence on educational outcomes in the literature. Based on the assumption that learners most effectively gain knowledge in practical situations, as argued by learning theories, one assumption is that the sense of presence learners experience in a VR simulation plays an important role in the learning process and directly influences educational outcomes positively [33]. Another theory, in contrast, speculates that highly immersive VR environments are more likely to distract and overload users and their memory capacity, resulting in lower levels of learning [38]. Better learning could therefore be achieved by reducing unnecessary strain on learning, i. e. extraneous cognitive load, as much as possible [38]. In empirical research, we found two studies with a positive correlation between presence and learning outcomes [39], [40], one with a negative correlation [41] and two with no significant correlation at all [42], [43]. We conclude that the effects of presence on educational outcomes are inconsistent and must be investigated further.
2.3 Acceptance of VR in Nursing Education
According to the unified theory of acceptance and use of technology (UTAUT), technology acceptance is defined as a process. It reaches from testing a technology, developing behavioural intention to eventually using the technology in the field [5].
In the context of nursing, research about acceptance of VR simulators is sparse. We found only two studies that tested the acceptance of a VR skills training simulator among nursing students or scrub nurses [44], [45]. They found a high rate of acceptance concluding that their VR simulator is ready to be investigated further, such as results of educational outcomes [44], and that it could potentially increase students’ motivation for learning [45]. Due to the paucity of studies, acceptance of VR simulations amongst learners in health care requires further research.
2.4 Evaluation of Knowledge and Skills
In order to analyse the potential of VR in nursing education, educational outcomes and the respective evaluation have to be defined. Generally, the literature distinguishes between three primary outcomes that are: (i) cognitive outcomes, such as increased knowledge, understanding or critical thinking, (ii) psychomotor outcomes such as improved technical skill performance, (iii) affective outcomes such as learner satisfaction or increased self-confidence [8], [10]. In a medical research meta-analysis, increase of knowledge is defined as the learner’s factual or conceptual understanding measured using change between pre- and post-test scores of a knowledge test [14]. Psychomotor outcomes such as improved skills performance are generally assessed through objective structured clinical examinations (OSCE) in the nursing domain [46]. In an OSCE, an expert observes a learner demonstrating a procedure on a mannequin or a simulated patient and rates the performance using a checklist. The VR simulation in our pilot study teaches ET suctioning. However, at this point in time, none of the studies that used VR scenarios for practising or learning the ET suction procedure used an OSCE to assess the skills for this procedure. A Japanese research team tested the adherence to aseptic techniques via a monitoring the hand movements during the procedure [47].
Screenshot collage of the VR simulation.
3 Use Case and Objectives
3.1 Use Case and Simulation
As described in Section 1, the ET suction procedure is highly stressful for everyone involved and must be performed precisely and flawless in order to avoid risks. To address this issue, we developed a VR simulator that helps students to learn and practice the procedure. The VR simulation was developed at the Care and Technology Lab (Institut Mensch, Technik und Teilhabe), Hochschule Furtwangen. The simulation runs on a HTC Vive Head Mounted Display (HMD) with Vive controllers. The aim of the simulation is to execute the steps of an ET suction intervention, as shown in Figure 1, on a virtual home care patient, guided by audio voice records and text messages on a virtual TV screen. The steps followed by the learner went from disinfecting hands at the beginning through to disposing the used equipment after the procedure. The order of steps followed the standard operation procedure of a German hospital.
The patient reacts in a realistic manner, looking at the learner, showing discomfort and coughing with a painful expression when the catheter is inserted in too deeply. Sound effects have been recorded from real medical equipment. After execution, the learner is presented with an evaluation of his performance including time required for the suction procedure as well as number of faulty actions like contamination. The simulation was developed iteratively, continuously checked by health care professionals and corrected accordingly. A pilot study with nursing professionals who have had previous experience in ET suctioning was conducted to validate the usability of the simulation as well as correctness of the procedure.
3.2 Objectives
The study’s approach was based on a semi-explorative design, analysing hypotheses, and revealing further research questions. The large number of hypotheses compared to the relatively low number of participants can lead to errors, however we do not draw final conclusions, but use the results to provide further insight on possible research topics. The first objective was to investigate if the UTAUT constructs in the special context of VR learning in nursing affect behavioural intention the same way as in general technological acceptance domains like office tasks. Besides the four key constructs, anxiety and self-efficacy were included as these could play a role in the domain of acceptance of VR learning. This led to the following hypotheses:
- h1:
[a–f] have an effect on behavioural intention to use a VR simulation for learning.
h1a: Performance expectancy, h1b: Effort expectancy, h1c: Social influence, h1d: Facilitating conditions, h1e: Anxiety, h1f: Self efficacy.
- h2:
[a–d] have a positive effect on behavioural intention to use a VR simulation for learning.
h2a: Sense of presence, h2b: Spatial presence, h2c: Involvement, h2d: Experienced realism.
- h3:
[a–d] in the VR learning simulation have a positive effect on increase of knowledge of the intervention.
h3a: Sense of presence h3b: Spatial presence, h3c: Involvement, h3d: Experienced realism.
4 Materials and Methods
4.1 Study Design
In order to evaluate the hypotheses and gain further insight, 51 physiotherapy students were recruited as study participants. Three VR setups separated by visual covers were installed in a large room. The students waited outside and filled in a knowledge pre-test. They were told that this test would not influence their course grades in any way and were observed by an instructor to prevent any cheating. Participants were asked to enter the room separately by the instructor whenever a VR setup was available. Next, they were given a standardised scripted explanation of how to use the system by one of the three instructors allocated to each VR setup. The students were asked to go through the simulation twice. This gave them an opportunity to familiarise themselves with the controls during the first run and enable them to focus on the task during the second run. Finally, participants had to complete a knowledge post-test, the UTAUT questionnaire and the Igroup Presence Questionnaire (IPQ).
Study sample.
| category | participants | gender | age | experience in | ||||
|
|
|
|
|
|
||||
| subcategory | overall | m | f | 18–25 | 26–30 | >30 | VR | suctioning |
| n | 47 | 7 | 40 | 43 | 2 | 2 | 18 | 5 |
| rate | 100 % | 14.9 % | 85.1 % | 91.5 % | 4.2 % | 4.2 % | 38.3 % | 10.6 % |
4.2 Instruments and Analysis
As an assessment tool for measuring presence, we used the IPQ. This divides presence in three subscales that are (i) spatial presence – the sense of being physically present in the VE; (ii) involvement – measuring the attention devoted to the virtual environment (VE) and the involvement experienced; (iii) experienced realism – measuring the subjective experience of realism in the VE [48]. Due to its multi-dimensional nature and confirmed validity, IPQ was the tool of choice. The IPQ guide on how to calculate subscales was followed [48]. In order to evaluate presence related hypotheses Pearson’s correlation analysis was performed. The standard significance level of 0.05 was used throughout evaluation. Statistical calculations were performed using the statistics software R© (The R Foundation, https://www.r-project.org/).
For assessing acceptance and acceptance related constructs, we used the UTAUT (see Section 2.3). This is a validated instrument that combines eight technology acceptance models. The UTAUT defines constructs, that causally affect behavioural intention and therefore acceptance. There are four key constructs: Performance expectancy, effort expectancy, social influence and facilitating conditions. Additionally, there are constructs that have been identified to not significantly affect behavioural intention: Anxiety and self-efficacy. The constructs consist of multiple items, which are statements about the technology. For example, one item for assessing performance expectancy would be “I would find the system useful in my job” [5]. To apply UTAUT, these items must be adapted to the setting and usage scenario of the technology being evaluated. Items were translated into German and adapted to the context of VR learning for students, trying to stay as closely to the semantic meaning as possible. Internal consistency using Cronbach’s alpha was measured to ensure constructs are reliable. To estimate the effect of the constructs on behavioural intention partial least squared path modelling was performed as suggested by Venkatesh, Morris, and Davis [5]. Calculations were performed with the PLS-PM library in R©.
A knowledge test based on the order of steps in the ET suction procedure according to the standard operating procedure of the Universitätsklinik Freiburg was created to assess increase of knowledge. The test contained 14 items, i. e. the steps of the intervention, that had to be put in the correct order. Assessment was performed using a pre-post-test-design. The same method was used previously in a similar study investigating increase of knowledge through a nursing simulator [49]. Each participant was taught the same knowledge about the intervention prior the test. They learned from a theory lesson and an information sheet. Increase of knowledge was measured by the percentage decrease of error in the post-test compared to the pre-test. Error was calculated using the mean of absolute deviation from each item’s put position by the learner compared to its correct position. Percentage decrease of error in the post-test compared to the pre-test was calculated and used as the score to measure increase of knowledge. To validate increase of knowledge through the simulation was significant, a paired t-test was performed on the pre- and post-test scores.
5 Results
Study sample and increase of knowledge
Out of the 51 participants, 47 completed the IPQ and UTAUT questionnaires, while 45 filled in the knowledge tests.
Table 1 summarises the study sample. The sample was relatively homogeneous consisting of health profession students, mostly aged ≤ 25 (91.5 %) and being female (85.1 %). Overall, 38.3 % had previous experience in VR and 10.6 % had previously performed endotracheal suction on a human patient. There was no significant difference in test performance between the group with and without previous VR experience.
As a requirement for further research questions, increase of knowledge through using the VR simulation had to be confirmed. Mean absolute error in the pre-test over all participants was 2.26 and 0.79 in the post-test. Mean percentage decrease of error was 65 %. A paired t-test confirmed high significance (p < 0.001).
5.1 UTAUT Constructs Effect on Acceptance
While four of the six UTAUT constructs reached at least a value of Cronbach’s Alpha α > 0.8, which is indicates a construct with good internal consistency, social influence (α = 0.64) and facilitating conditions (α = 0.37) did not reach the α ≥ 0.7 border which is considered inacceptable reliability, Thus, the two constructs had to be discarded for further conclusions leaving h1c and h1d unanswered.
UTAUT constructs mean and effect on behavioural intention (*p < 0.05, **p < 0.01).
| construct | mean | effect (f2) | t-value | p-value |
| Performance expectancy | 4.40/5 | 0.34 | 2.33* | 0.025 |
| Effort expectancy | 4.03/5 | 0.25 | 1.80** | 0.008 |
| Anxiety | 1.74/5 | −0.10 | −0.83 | 0.41 |
| Self-efficacy | 3.36/5 | 0.11 | 0.88 | 0.38 |
| Behavioural intention | 3.80/5 | – | – | – |
Table 2 shows the mean score of the UTAUT constructs as well as their effect on behavioural intention and its significance. Overall, behavioural intention of using VR for learning was high (3.8/5 (76 %)). The constructs performance expectancy (f2 = 0.34, p = 0.025) and effort expectancy (f2 = 0.25, p = 0.008) had a significant effect with a medium to large effect size on behavioural intention suggesting that h1a and h1b are likely to be true.
5.2 Effects of Presence
The scores of the presence subscales were as follows. The general sense of presence was relatively high (4.5/6), spatial presence (4.3/6) and involvement (3.9/6) were in the medium range, while the expected realism was rather low (2.5/6).
Pearson’s correlation between presence and behavioural intention (***p < 0.001, *p < 0.05).
| construct | Correlation (r) | t-value | p-value |
| sense of presence | 0.52 | 4.07*** | 0.000 |
| spatial presence | 0.39 | 2.86* | 0.01 |
| involvement | 0.26 | 1.79 | 0.08 |
| experienced realism | 0.34 | 2.43* | 0.02 |
Table 3 shows Pearson’s correlations between the presence subscales and behavioural intention. Correlation between sense of presence and behavioural intention was highly significant (p < 0.001) and according to Cohen also has a large effect (r > 0.5). Correlation between spatial presence and behavioural intention, as well as experienced realism were also significant with a medium effect. Thus, h2a, h2b and h2d were confirmed. The effect of involvement was not significant and had only a low effect size, which lead to h2c being rejected.
Pearson’s correlation between presence and increase of knowledge (*p < 0,05).
| construct | Correlation (r) | t-value | p-value |
| sense of presence | −0.28 | −1.89 | 0.07 |
| spatial presence | −0.33 | −2.31* | 0.03 |
| involvement | −0.15 | −1.03 | 0.31 |
| experienced realism | −0.16 | −1.07 | 0.29 |
Likewise, Table 4 shows correlations between the presence subscales and increase of knowledge. Correlation between spatial presence and increase of knowledge was negative with medium effect size (
6 Discussion
Like the UTAUT model suggests, performance expectancy and effort expectancy had a significant effect on behavioural intention. Although some participants showed fear during execution of the simulation and had to cancel it, anxiety did not have a significant effect and neither did self-efficacy. Due to their insufficient reliability, no statement about social influence and facilitating conditions can be made. Overall, technology acceptance in VR learning does not seem to differ from other domains regarding the UTAUT constructs. However, the number of participants was small and further research needs to be done to verify this. This is especially the case with the anxiety construct since our subjective observations differed from the statistical findings.
Sense of presence had a medium to strong size positive effect on behavioural intention that was highly significant. A higher sense of presence in a simulation could lead to a higher behavioural intention by learners. This seems plausible, since a simulation that absorbs the learner into a virtual training scenario and feels subjectively more realistic, should be more likely to be accepted and used.
Out of the presence metrics, only spatial presence correlated significantly (p < 0.05) with increase of knowledge. It seems likely that a user who feels more spatially present in the virtual training scenario would have a more imprinting experience and could therefore memorise more of it. However, against this presumption the correlation was negative. Our result is in line with previous research about the effects of presence on educational outcomes that come to highly different results which show similar negative correlations and struggle to come up with a clear conclusion or a verdict on the effect of presence (see Section 2.2).
There could be various reasons for this discrepancy. To begin with, it is possible that there are various survey instruments for assessing presence which are based on slightly different definitions and are therefore not measuring the same constructs. Another more complex explanation includes viewing presence from two different perspectives. Firstly, presence can be seen from a user’s perspective, suggesting that each person has a different sense of presence towards the same simulation. Secondly, presence can be seen from simulation perspective suggesting that the same user feels different degrees of presence in different simulations. This would lead to two different study designs. In the first study design, differences in the degree to which users perceive presence in the same simulation are measured. In the second study design, different simulations are compared by having the same users participate in multiple simulations, and the differences in degrees of presence which those simulations can convey are measured. On one hand, users who generally feel more present in VR simulations than others might focus more on the environment than the actual task and receive a cognitive overload [38]. On the other hand, simulations that generally convey a higher sense of presence could lead to more imprinting experiences and therefore positively affect increase of knowledge [33].
Besides increase of knowledge, psychomotor outcomes, such as improved technical skill performance, are an important topic for further research. In order to investigate these in the domain of nursing education and ET suctioning, we are developing an OSCE to quantitatively assess skills outcomes. Based on theoretical insights as well as evidence from medical technology research, some presumptions can be made.
An important feature of VR is the ability to incorporate haptic elements as well as interaction with them. Learning theories (see Sections 2.1 and 2.2) support the use of haptics and interaction, as hands-on experiences ensure that learners remember the content better. The importance of haptic and interaction fidelity of simulations has been described in several papers. Some reviews argue that skills cannot be fully learned through virtual training, and real training on patients is still essential [13], [30]. The skill level that can be achieved through VR would be low-plateau, i. e. less maximum skill compared to training on real patients [13]. Furthermore, it would be important that movements are learned in the most realistic way possible by incorporating haptics and force feedback. Otherwise, there might be a risk that interventions will be applied incorrectly in practice due to inaccurately learned motor sequences in simulations [13]. A summary of best practices for educational VR simulations suggests that movements with controllers should be mapped as realistically as possible [50]: Rotating movement, for example, should be mapped to a circular movement of fingers on the touchpad rather than to pressing a button. Finally, in the non-VR simulation domain of nursing education, one review concluded that medium fidelity patient simulators with focus on haptic and motoric interaction are more effective in producing psychomotor skill outcomes than high-fidelity ones with additional features, such as speech or breathing simulation [10].
Based on these presumptions, the effect of VR skills training with real-world haptic and motoric interaction has yet to be estimated, especially in the domain of nursing education and ET suctioning. A previous approach described a VR simulation with a force-feedback device attached to the wrists of users to simulate catheter drag [23], but has not reported results regarding educational outcomes yet. We plan to use a similar approach in our next study. However, in order to fulfil the presumption that haptic and motor fidelity has to be as maximised, we attempt to incorporate a real catheter into our VR simulation, which in turn will transform the VR simulation into a mixed-reality simulation. Furthermore, we will compare two simulations with varying levels of immersion to learn more about the effects of presence.
6.1 Limitations
There are some flaws in the study design. On one hand, the VR simulation aims to teach nursing skills. The participants, however, were physiotherapy students as there were no larger nursing student groups available to the authors for testing. Additionally, previous experience in suctioning is likely to influence performance. On the other hand, the large number of effects investigated with a relatively small amount of study participants could lead to statistical errors. Another problem is that no split of groups was performed. Participants all ran the same simulation which led to making assumptions based on general effect size classes and correlation of constructs rather than comparing effect sizes.
6.2 Conclusion
Our pilot study analysed various aspects of VR learning in the health care education sector. Performance expectancy and effort expectancy had a significant effect on behavioural intention in the target group while self-efficacy and anxiety did not. This reflects the results from the creators of UTAUT [5]. We could not confirm the effects of social influence and facilitating conditions due to unreliability of our constructs. Moreover, our results suggest that the sense of presence could have a positive effect with large size on behavioural intention of health care VR learning simulations. Future research needs to improve the UTAUT based items in order to gain more reliable insight particularly into the anxiety construct, which seemed to play a role according to our observations. Moreover, our detected effect of presence must be confirmed through further studies.
We found that there was a significant negative correlation between spatial presence and increase of knowledge and no significant correlation between other presence metrics and increase of knowledge. We explained a possible reason for the discrepancy of the measured effects of presence on learning outcomes in different studies. There is still no consensus about the effect of presence on learning and further research needs to investigate the topic in further studies.
Finally, the effects of haptic immersion and motoric fidelity VR simulations in nursing education need to be studied. For this purpose, we are developing an OSCE to assess skills outcomes and a mixed reality simulation that uses a real catheter.
About the authors
Christian Plotzky completed his M.Sc. in computer science at Furtwangen University, Germany. Currently, he is working as a researcher at the Care and Technology Lab (IMTT), Hochschule Furtwangen. He is also working on his dissertation at University of Potsdam. His research areas include the development and assessment of VR simulations as an educational tool in nursing.
Ulrike Lindwedel is a nursing researcher in the Care and Technology Lab at Furtwangen University. Her research interests include nursing in the context of assistive technologies, palliative care, and global health issues in various settings.
Alexander Bejan is a researcher at Furtwangen University located in the Black Forest mountain range. His general research interest includes all kinds of assistive technologies that add value to the lives of people with disabilities and their carers.
Peter König began his professional career as a nurse and nursing manager in the 1980s and 1990s at Freiburg University Hospital in the areas of neurosurgery, oncology and geriatrics. He studied nursing management and nursing science and received his doctorate from the Philosophical-Theological University of Vallendar in 2014. He is full-time professor at Furtwangen University in the field of nursing and rehabilitation management. His work focuses on nursing care concepts and assistive technologies in nursing especially in people with dementia, palliative care, and global health as well as innovative educational concepts.
Christophe Kunze has been Professor of Assistive Technologies in the Applied Health Sciences degree programme at Furtwangen University since 2011. Together with Peter König he is head of the Care & Technology Lab (IMTT) in Furtwangen. Previously, he was the spokesperson for the AAL Living Lab and head of the Embedded Systems & Sensors Engineering research area at the FZI Research Center for Information Technology in Karlsruhe.
Acknowledgment
A special thanks goes to Barbara Loessl who corrected language and style, as well as to our participants and Vanessa Redlof who helped during the study.
References
[1] Statistisches Bundesamt (Destatis), “Pflegestatistik: Pflege im Rahmen der Pflegeversicherung: Deutschlandergebnisse 2017,” 2018.Search in Google Scholar
[2] S. Lindner and N. Fleischer, “Arbeitsmarktsituation im Pflegebereich,” Nürnberg, Berichte: Blickpunkt Arbeitsmarkt, 2019.Search in Google Scholar
[3] U. Lindwedel-Reime and P. König, “Wahrgenommene Belastungen professionell Pflegender in der außerklinischen Beatmungspflege. Welchen Einfluss hat die Technik?” Zukunft der Pflege – Tagungsband der 1. Clusterkonferenz, vol. 2018, no. 1, pp. 142–146, 2018.Search in Google Scholar
[4] Y. Lehmann and M. Ewers, “Sicherheit in der häuslichen Intensivversorgung beatmeter Patienten aus Sicht professioneller Akteure” (in ger), Gesundheitswesen (Bundesverband der Arzte des Offentlichen Gesundheitsdienstes (Germany)), vol. 82, no. 1, pp. 75–81, 2020, doi: 10.1055/a-0667-8198dx.Search in Google Scholar
[5] V. Venkatesh, M. G. Morris, G. B. Davis, and F. D. Davis, “User Acceptance of Information Technology: Toward a Unified View,” MIS Quarterly, vol. 27, no. 3, p. 425, 2003, doi: 10.2307/30036540.Search in Google Scholar
[6] L. H. Aiken et al., “Patient safety, satisfaction, and quality of hospital care: cross sectional surveys of nurses and patients in 12 countries in Europe and the United States,” BMJ (Clinical research ed.), vol. 344, e1717, 2012, doi: 10.1136/bmj.e1717.Search in Google Scholar PubMed PubMed Central
[7] D. A. Cook et al., “Technology-enhanced simulation for health professions education: a systematic review and meta-analysis,” JAMA, vol. 306, no. 9, pp. 978–988, 2011, doi: 10.1001/jama.2011.1234.Search in Google Scholar PubMed
[8] S. Prion, “A Practical Framework for Evaluating the Impact of Clinical Simulation Experiences in Prelicensure Nursing Education,” Clin Simulation Nurs, vol. 4, no. 3, e69–e78, 2008, doi: 10.1016/j.ecns.2008.08.002.Search in Google Scholar
[9] R. P. Cant and S. J. Cooper, “Use of simulation-based learning in undergraduate nurse education: An umbrella systematic review,” Nurse education today, vol. 49, pp. 63–71, 2017, doi: 10.1016/j.nedt.2016.11.015.Search in Google Scholar PubMed
[10] J. Kim, J.-H. Park, and S. Shin, “Effectiveness of simulation-based nursing education depending on fidelity: a meta-analysis,” BMC Medical Education, vol. 16, p. 152, 2016, doi: 10.1186/s12909-016-0672-7.Search in Google Scholar PubMed PubMed Central
[11] S. N. de Oliveira, M. L. d. Prado, and S. S. Kempfer, “Use of simulations in nursing education: an integrative review,” Rev Mineira Enfermagem, vol. 18, no. 2, 2014, doi: 10.5935/1415-2762.20140036.Search in Google Scholar
[12] J. E. Brown, “Graduate Nurses’ Perception of the Effect of Simulation on Reducing the Theory-Practice Gap,” SAGE Open Nursing, vol. 5, 2019, doi: 10.1177/2377960819896963.Search in Google Scholar PubMed PubMed Central
[13] P. Pantelidis et al., “Virtual and Augmented Reality in Medical Education,” in Medical and Surgical Education – Past, Present and Future, G. Tsoulfas, Ed., InTech, 2018.Search in Google Scholar
[14] B. M. Kyaw et al., “Virtual Reality for Health Professions Education: Systematic Review and Meta-Analysis by the Digital Health Education Collaboration,” Journal of Medical Internet Research, vol. 21, no. 1, 12959, 2019, doi: 10.2196/12959.Search in Google Scholar PubMed PubMed Central
[15] F.-Q. Chen et al., “Effectiveness of Virtual Reality in Nursing Education: Meta-Analysis,” Journal of Medical Internet Research, vol. 22, no. 9, e18290, 2020, doi: 10.2196/18290.Search in Google Scholar PubMed PubMed Central
[16] S. Rourke, “How does virtual reality simulation compare to simulated practice in the acquisition of clinical psychomotor skills for pre-registration student nurses? A systematic review,” International journal of nursing studies, vol. 102, 103466, 2020, doi: 10.1016/j.ijnurstu.2019.103466.Search in Google Scholar PubMed
[17] S. Weiss, H. Bongartz, S. C. J. Boll, and W. Heuten, “Applications of Immersive Virtual Reality in Nursing Education – A Review,” in 1. Clusterkonferenz Zukunft der Pflege – Innovative Technologien für die Praxis, 2018.Search in Google Scholar
[18] S. D. Reyes, K. Stillsmoking, and D. Chadwick-Hopkins, “Implementation and Evaluation of a Virtual Simulator System: Teaching Intravenous Skills,” Clin Simulation Nurs, vol. 4, no. 1, e43–e49, 2008, doi: 10.1016/j.ecns.2009.05.055.Search in Google Scholar
[19] C. Plotzky et al., “Virtual reality simulations in nurse education: A systematic mapping review,” Nurse education today, 2020. doi: 10.1016/j.nedt.2021.104868.Search in Google Scholar PubMed
[20] A. Adefila, S. Graham, L. Clouder, P. Bluteau, and S. Ball, “myShoes – the future of experiential dementia training?” The Journal of Mental Health Training, Education and Practice, vol. 11, no. 2, pp. 91–101, 2016, doi: 10.1108/JMHTEP-10-2015-0048.Search in Google Scholar
[21] S. Farra et al., “Storyboard Development for Virtual Reality Simulation,” Clinical Simulation in Nursing Education, vol. 12, no. 9, pp. 392–399, 2016, doi: 10.1016/j.ecns.2016.04.002.Search in Google Scholar
[22] M. Chiba and K. Hamamoto, “Development of Learning Support System on Auscultation for Nurse Using Virtual Reality,” in 2018 2nd International Conference on Biomedical Engineering (IBIOMED), Kuta, 2018, pp. 34–38.Search in Google Scholar
[23] S. Komizunai, N. Colley, and A. Konno, Eds., An immersive nursing education system that provides experience of exemplary procedures from first person viewpoint with haptic feedback on wrist, 2020.Search in Google Scholar
[24] S. Kardong-Edgren, K. Breitkreuz, M. Werb, S. Foreman, and A. Ellertson, “Evaluating the Usability of a Second-Generation Virtual Reality Game for Refreshing Sterile Urinary Catheterization Skills,” Nurse educator, vol. 44, no. 3, pp. 137–141, 2019, doi: 10.1097/NNE.0000000000000570.Search in Google Scholar PubMed
[25] J. Hirt and T. Beer, “Use and impact of virtual reality simulation in dementia care education: A scoping review,” Nurse education today, vol. 84, 104207, 2020, doi: 10.1016/j.nedt.2019.104207.Search in Google Scholar PubMed
[26] S. Shorey and E. D. Ng, “The use of virtual reality simulation among nursing students and registered nurses: A systematic review,” Nurse education today, 104662, 2020, doi: 10.1016/j.nedt.2020.104662.Search in Google Scholar PubMed
[27] INACSL Standards Committee, “INACSL Standards of Best Practice: SimulationSM Simulation Glossary,” Clin Simulation Nurs, vol. 12, S39–S47, 2016, doi: 10.1016/j.ecns.2016.09.012.Search in Google Scholar
[28] T. Leung, F. H. Zulkernine, and H. Isah, “The use of Virtual Reality in Enhancing Interdisciplinary Research and Education,” CoRR, abs/1809.08585, 2018.Search in Google Scholar
[29] M. Slater, “Implicit Learning Through Embodiment in Immersive Virtual Reality,” in Smart Computing and Intelligence, Virtual, Augmented, and Mixed Realities in Education, D. Liu, C. Dede, R. Huang, and J. Richards, Eds., Singapore: Springer Singapore, 2017, pp. 19–33.Search in Google Scholar
[30] S.-K. Loke, “How do virtual world experiences bring about learning? A critical review of theories,” AJET, vol. 31, no. 1, 2015, doi: 10.14742/ajet.2532.Search in Google Scholar
[31] J. Lave and E. Wenger, Situated Learning, Cambridge University Press, 2012.Search in Google Scholar
[32] S.-H. Wu et al., “Effect of virtual reality training to decreases rates of needle stick/sharp injuries in new-coming medical and nursing interns in Taiwan,” Journal of educational evaluation for health professions, vol. 17, p. 1, 2020, doi: 10.3352/jeehp.2020.17.1.Search in Google Scholar PubMed PubMed Central
[33] A. Dengel and J. Mägdefrau, “Presence Is the Key to Understanding Immersive Learning,” in Immersive Learning Research Network, 2019, pp. 185–198.Search in Google Scholar
[34] M. Slater and S. Wilbur, “A Framework for Immersive Virtual Environments (FIVE): Speculations on the Role of Presence in Virtual Environments,” Presence: Teleoperators and Virtual Environments, vol. 6, no. 6, pp. 603–616, 1997, doi: 10.1162/pres.1997.6.6.603.Search in Google Scholar
[35] H. L. Miller and N. L. Bugnariu, “Level of Immersion in Virtual Environments Impacts the Ability to Assess and Teach Social Skills in Autism Spectrum Disorder,” Cyberpsychology, behavior and social networking, vol. 19, no. 4, pp. 246–256, 2016, doi: 10.1089/cyber.2014.0682.Search in Google Scholar PubMed PubMed Central
[36] S. S. Kardong-Edgren, S. L. Farra, G. Alinier, and H. M. Young, “A Call to Unify Definitions of Virtual Reality,” Clin Simulation Nurs, vol. 31, pp. 28–34, 2019, doi: 10.1016/j.ecns.2019.02.006.Search in Google Scholar
[37] B. G. Witmer and M. J. Singer, “Measuring Presence in Virtual Environments: A Presence Questionnaire,” Presence: Teleoperators and Virtual Environments, vol. 7, no. 3, pp. 225–240, 1998, doi: 10.1162/105474698565686.Search in Google Scholar
[38] M. Mulders, J. Buchner, and M. Kerres, “A Framework for the Use of Immersive Virtual Reality in Learning Environments,” Int. J. Emerg. Technol. Learn., vol. 15, no. 24, p. 208, 2020, doi: 10.3991/ijet.v15i24.16615.Search in Google Scholar
[39] Y. Cho, “How Spatial Presence in VR Affects Memory Retention and Motivation on Second Language Learning: A Comparison of Desktop and Immersive VR-Based Learning,” 2018.Search in Google Scholar
[40] M. N. Selzer, N. F. Gazcon, and M. L. Larrea, “Effects of virtual presence and learning outcome using low-end virtual reality systems,” Displays, vol. 59, pp. 9–15, 2019, doi: 10.1016/j.displa.2019.04.002.Search in Google Scholar
[41] J. Bailey, J. N. Bailenson, A. S. Won, J. Flora, and K. C. Armel, “Presence and Memory: Immersive Virtual Reality Effects on Cued Recall,” in Proceedings of the International Society for Presence Research Annual Conference, 2012.Search in Google Scholar
[42] D. Lerner, J. Pranghofer, and A. Franke, “Der Einfluss des Präsenzerlebens auf die Lern- und Trainingseffekte in einer Virtual-Reality Simulationsumgebung,” 2019.Search in Google Scholar
[43] S. Persky, K. A. Kaphingst, C. McCall, C. Lachance, A. C. Beall, and J. Blascovich, “Presence relates to distinct outcomes in two virtual environments employing different learning modalities,” Cyberpsychology & behavior: the impact of the Internet, multimedia and virtual reality on behavior and society, vol. 12, no. 3, pp. 263–268, 2009, doi: 10.1089/cpb.2008.0262.Search in Google Scholar PubMed PubMed Central
[44] M.-S. Bracq et al., “Learning procedural skills with a virtual reality simulator: An acceptability study,” Nurse education today, vol. 79, pp. 153–160, 2019, doi: 10.1016/j.nedt.2019.05.026.Search in Google Scholar PubMed
[45] A.-K. Lange et al., “Learning With Virtual Reality in Nursing Education: Qualitative Interview Study Among Nursing Students Using the Unified Theory of Acceptance and Use of Technology Model,” JMIR Nursing, vol. 3, no. 1, 20249, 2020, doi: 10.2196/20249.Search in Google Scholar PubMed PubMed Central
[46] M. Zayyan, “Objective structured clinical examination: the assessment of choice,” Oman Medical Journal, vol. 26, no. 4, pp. 219–222, 2011, doi: 10.5001/omj.2011.55.Search in Google Scholar PubMed PubMed Central
[47] N. Colley et al., “Hygienic Manipulation Evaluation System during Endotracheal Suctioning using Bilateral Dorsum Manus Total Travel Distance*,” in 2020 IEEE/SICE International Symposium on System Integration (SII), Honolulu, HI, USA, 2020, pp. 500–503.Search in Google Scholar
[48] T. W. Schubert, “The sense of presence in virtual environments,” Zeitschrift für Medienpsychologie, vol. 15, no. 2, pp. 69–71, 2003, doi: 10.1026//1617-6383.15.2.69.Search in Google Scholar
[49] L. Meyer and T. Pfeiffer, Eds., Vergleich von Leap Motion Hand-Interaktion mit den HTC-Vive MotionControllern in einer VR-Trainingssimulation für manuelle Arbeiten, 2017.Search in Google Scholar
[50] M. C. Johnson-Glenberg, “The Necessary Nine: Design Principles for Embodied VR and Active Stem Education,” in Smart Computing and Intelligence, Learning in a Digital World, P. Díaz, A. Ioannou, K. K. Bhagat, and J. M. Spector, Eds., Singapore: Springer Singapore, 2019, pp. 83–112.Search in Google Scholar
© 2021 Walter de Gruyter GmbH, Berlin/Boston
