Keywords

1 Introduction

The ultimate goal of educational applications is to enable users to acquire knowledge or skills more intuitively. This is termed the educational objective [1]. Therefore, creating an immersive experience and imparting knowledge are the two most crucial considerations. Advances in computer hardware have widened the use of digital technologies in education, especially in recent years when connected cellphones have become a necessity in daily life, allowing individuals to download educational applications in order to study math, foreign languages, history and physics among other subjects. In the 1990s, various educational software packages were sold in large bookstores throughout China. The software consisted of 2D animations and aroused users’ interest in learning through vivid images. Consequently, users could seek out knowledge and thus gain pleasure. With recent and rapid development in digital technology, numerous individuals are no longer content with the passive impartment of knowledge, instead wanting virtualized and dramatized knowledge with which they can interact with computers and simultaneously improve their problem solving skills. However, creating a qualified human-computer experience which integrates education remains a significant challenge. Moreover, the best means of converting the enjoyability of digital entertainment into education also merits investigation [2].

The design of educational applications differs from that of ordinary entertainment games. Instead of perfect visual effects, strong tactile feedback and complex scenarios, educational applications must be able to balance entertainment and educational objectives. No unnecessary objects need be preserved, as proper feedback with well created plots centered on key knowledge can help students to better realize their educational goals. Application developers alone cannot complete the whole design process. If they can neither blend in and communicate with service objects (teachers or students) nor offer clear educational objectives, they will be unable to produce qualified educational products. Hence, application developers as well as service objects (teachers or students) should be involved in the design of educational products. The design process extends beyond system development, encompassing communication between individuals with different perspectives and values. The aim is to identify a method for accomplishing educational objectives that can satisfy all parties [3].

In this study, students are required to use several VR educational applications and evaluate their audiovisual experience, usability, exploratory nature, direction and feedback, and enjoyability. These evaluations are quantified in order to determine the relationship with educational objectives and thus construct a participatory design method. This method is used to conduct participatory design evaluations with service objects (teachers or students) who are jointly developing the beta version of the Beijing Subway Maintenance Teaching System VR (BSMTS VR).

2 Student Participatory Design

Participatory design began in Scandinavia during the 1970s–1980s, when trade union movements forced through new laws which gave employees new rights and a say in working environment change [4]. Moreover, manufacturers also noticed the limitation in product development, with design revolution especially important. Bela H. Banathy, a famous system design expert from the US, identified the four generations of design in human activity systems. The first generation, Design by Dictate, is deeply affected by the Systems Engineering Approach and often implemented from top to bottom through legislation. The second generation is known as Designing for, according to which experts and consultants are invited to investigate a specific system problem, conduct demand analysis and provide solutions to decision-makers. The third generation is called Designing with/Designer Guided, according to which design is produced following the discussion between designers and decision-makers. The fourth generation is called Designing Within, which proposes that human activity systems must be jointly designed by those within the system, those using the system and those the system serves [5]. This paper believes that the fourth generation method can serve as the fundamental basis for participatory design. This method differs from the first three generations in that communication, cooperation and sharing are required, with user participation playing a role from the very beginning of the design. Users become an integral component of the entire design team through active participation [6, 7]. Therefore, everyone involved is both a designer and a user, thus exerting direct influence on the ultimate product quality.

According to participatory design, students, as the service objects in educational applications, should participate in app design and development so as to improve app quality and the extent to which students’ demands are satisfied.

3 VR Educational Applications

VR creates a virtual space accessible through the use of computers and sensors. Users can sense and operate virtual objects with the help of various sensors and participate in virtual events. Participants can immerse themselves in the vivid environment created. In short, VR constitutes a virtual world similar to the real one where people can interact through watching, listening, touching and feeling, thereby engendering vivid interactions.

VR educational applications differ from ordinary educational applications in that the latter imparts knowledge in a simple and straightforward manner, while the former advocates student-centered design and guides students in actively acquiring and understanding knowledge while simultaneously formulating a knowledge network. This is part of Constructivism. The role of educational applications is to promote and guide students in developing their own knowledge network, rather than simply imparting knowledge [8]. Therefore, the situation, activity and interaction in constructivism-based learning environments can constantly challenge the students’ experience, so as to promote the development of new knowledge [9].

VR technologies can vividly fill 3D space, thus providing students with the opportunity to directly interact with objects and experience a stronger sense of involvement. Multi-dimensional VR learning environments can offer students direct and efficient means of acquiring knowledge and skills. Therefore, thanks to its many advantages, VR is often utilized in special education, simulation experiments and specialty training. Here, special education refers to dangerous or difficult education for which direct student involvement is rarely required in reality, such as learning to drive. In this scenario, new drivers typically receive extensive theoretical training before actually driving on the road. Furthermore, theoretical training is rather limited compared to actually driving. Therefore, VR can be adopted in order to enable new drivers to experience real on-the-road driving at an earlier stage. Moreover, virtual high-risk driving is made possible through VR. Consequently, data concerning how new drivers handle high-risk incidents can be collected and analyzed in order to improve their ability to anticipate and thus reduce risk [10].

The BSMTS VR case is another example. Given the danger associated with subway maintenance, VR adoption in training could greatly reduce risk in real-life scenarios.

However, applying VR in education up until now has focused more on technology, such as improving simulation vividness and the interactive experience from a hardware perspective. This paper believes that, when designing VR educational applications, users can be better assisted in achieving their objectives by adopting a user experience perspective rather than a technological perspective. Operation should be made as easy as possible, with training offered to teachers and students. Expansibility should be available so that users can themselves edit and adjust the application to some extent, thus allowing them to better adjust to and recognize the virtual environment and enhance their interest in learning.

4 Participatory Design of VR Educational Applications

Participatory design emphasizes user participation. Kuhn & wino-grad suggest that the participation degree can be evaluated from four dimensions: participation mode, time, scope and control [11]. He stresses that designers should fully consider user demands when designing a teaching system. In VR application design, such factors must also be taken into account. In previous teaching, student problems encountered while using VR educational applications included short participation time and a lack of understanding of interaction behavior, which directly impacted their accomplishments. Therefore, it is hoped that students can participate in the design and development of VR educational applications, by researching and analyzing the audiovisual experience, usability, exploratory nature, direction and feedback, and enjoyability. Subsequently, how these factors relate to educational objectives can be determined.

  • Audiovisual experience: How users evaluate the images and sounds encountered when using the application.

  • Usability: The extent to which using the application is perceived as difficult by users.

  • Exploratory nature: How users evaluate the participation process and their time when using the application.

  • Direction and feedback: How users evaluate operation directions and proper feedback when using the application.

  • Enjoyability: To the extent to which users enjoy using the application.

  • Accomplishments: The extent to which users have mastered relevant knowledge after using the application.

Before designing BSMTS VR using the participatory design method, existing educational applications were investigated and analyzed. Two existing VR applications of science popularization were selected: THE BODY VR and Seismic VR. Then, user research was conducted vis-à-vis the five variables (see Fig. 1). THE BODY VR [12] is a VR game where players use VR devices to explore the microscopic world of human bodies. Users can travel through the organs and learn how they work. This application is more than a game, it is a biology teaching software. Seismic VR simulates a seismic event and guides users in properly escaping. In addition to explaining what should be done when seismic activity is detected, the application also emphasizes practice and action. Both applications are distinctive. THE BODY VR focuses on imparting knowledge, while Seismic VR has practical value. Data from 11 THE BODY VR users and 10 Seismic VR users were analyzed. All users are college students without professional knowledge in biology or disaster mitigation. However, they have all studied biology and participated in live seismic exercises at middle school. Hence, they have some basic knowledge. Students evaluated the applications with regard to the five variables after using them, as well as their own accomplishments. Then, other students or experts questioned the students in order to test their real accomplishments.

Fig. 1.
figure 1

THE BODY VR and Seismic VR user research

Full size image

According to the data (see Figs. 2 and 3), the two applications barely differ in terms of the audiovisual experience, usability, exploratory nature, direction and feedback, and enjoyability. Both applications are well rated by students. However, the accomplishments of THE BODY VR are rated lower than those of Seismic VR in both self-evaluation and the teacher’s test. Two potential reasons were identified following discussion with students and teachers. First, students unanimously believe that, in terms of visual effects and enjoyability, THE BODY VR is superior to Seismic VR. However, during use, the interaction is more concerned with creating interesting cells, which do not complement the background voice imparting cell knowledge. In this case, the visual effects may even distract students. In addition, the teachers found that students paid more attention to observing the shape and motion of objects within cells but cared little about the background knowledge.

Fig. 2.
figure 2

THE BODY VR UX research data

Full size image
Fig. 3.
figure 3

Seismic VR UX research data

Full size image

According to the evaluations of both students and teachers, THE BODY VR is a theory-based VR application, which allows users to do something that is not possible in real life - entering cells. However, in THE BODY VR, interaction is not closely related to the knowledge, meanings students are still passively imparted knowledge, but with more attractive images. Seismic VR requires students to practice their theoretical knowledge and thus improve their understanding. Hence, Seismic VR is an operation-based application. All operations are related to the knowledge, thus solidifying students’ understanding and enhancing their accomplishments.

In summary, VR-based education suits operation or practice-based activities, which requires students to get involved in each step in person in order to solidify their foundational knowledge. For traditional activities which mainly impart knowledge, VR can enhance engagement and interest compared to traditional video teaching. However, the impact on students’ accomplishments is weaker, as is the case in the BSMTS VR application.

5 BSMTS VR

Subway maintenance is highly dangerous. Students in this field require extensive practice before they can master the relevant skills. According to teachers in this field, new students are at a greater risk than old ones. Therefore, it is necessary to design a VR teaching application for subway maintenance, so that students can acquire knowledge with minimal risk.

The first stage in this VR application is cleaning locomotive screws, which is jointly developed with the participation of seven students majoring in subway maintenance. This paper details the four versions of the participatory design process, analyzing student usage data in the design process (see Figs. 4 and 5). The four versions are as follows:

Fig. 4.
figure 4

Participatory design process with students

Full size image
Fig. 5.
figure 5

Participatory design process with students

Full size image

  • BSMTS VR version 1 (video explanation): Explains the process and risks associated with cleaning subway locomotive screws through the medium of video.

  • BSMTS VR version 2 (VR): Explains the process and risks associated with cleaning subway locomotive screws using VR.

  • BSMTS VR version 3 (VR direction and hint): Incorporates text directions and feedback based on Version 2.

  • BSMTS VR version 4 (voice hint and a score mechanism): Incorporates voice directions and a score mechanism based on Version 3

5.1 BSMTS VR Version 1

This version explains the process and risks associated with cleaning subway locomotive screws through the traditional medium of video. The students are asked to watch and evaluate the video using the participatory design method. The results are as follows (see Fig. 6):

figure a
Fig. 6.
figure 6

BSMTS VR Version1 UX research data

Full size image

According to the results, the students are not satisfied with the overall performance except for its usability. Moreover, the accomplishments are poor and the method fails to arouse their interest (see Fig. 6).

5.2 BSMTS VR Version 2

Compared to Version 1, in Version 2, students can wear a VR helmet and enter a virtual subway carriage. They are able to watch the cleaning process in the virtual world, freely exploring the carriage and interacting in a simple manner without directions or educational objectives. Students simply observe the carriage. Students experience and evaluate this version in turn using the participatory design method. It is hoped that they can offer suggestions. The results are as follows (see Fig. 7):

figure b
Fig. 7.
figure 7

BSMTS VR Version2 UX research data

Full size image

The results show that students’ evaluations of the audiovisual experience, exploratory nature and enjoyability increase significantly, while their accomplishments improve only mildly. However, no significant improvements are visible in usability and direction and feedback (see Fig. 7). The students believe that the VR technology enables them to personally experience and involve themselves in the cleaning process, arousing their interest in a short time. However, merely watching and interacting superficially do not allow students to understand and remember all operations. It would be better if someone were present in order to guide them through real operations at this point in time.

5.3 BSMTS VR Version 3

In BSMTS VR Version 3, a blackboard with operation procedures is placed next to the coach door. All the necessary objects are labeled with their names above them. Students can follow the procedure in order to learn operations step by step. The students are also required to evaluate this version. The results are as follows (see Fig. 8):

figure c
Fig. 8.
figure 8

BSMTS VR Version3 UX research data

Full size image

The results reveal a significant improvement in usability, direction and feedback, as well as accomplishments (see Fig. 8). In the students’ view, the procedures and added names are of great help because they allow students to follow the procedure for cleaning the screws step by step. Compared to passively watching as in Version 2, the practical operation element in Version 3 can better help students to acquire and retain the relevant knowledge.

5.4 BSMTS VR Version 4

In the discussion on Version 3, students express the belief that the time could be limited to 100 s, with a reward and punishment system added. Therefore, both voice directions and a score mechanism are included in Version 4. After each step, voice directions are given for the next step. The overall operation time is calculated after each student completes the task. Then, their performance speeds are ranked against one another. The students are again required to evaluate this version. The results are as follows (see Fig. 9):

figure d
Fig. 9.
figure 9

BSMTS VR Version4 UX research data

Full size image

The results demonstrate significant improvements in the audiovisual experience, usability, enjoyability and accomplishments. According to the students, the improvement in the audiovisual experience and usability can be attributed to the voice directions, but these do not produce a significant improvement in direction and feedback (see Fig. 9).

The results for all the four versions are presented together in a single graph (see Fig. 10). The results show that using the participatory design method facilitates the development of the BSMTS VR. Students evaluate the current version in terms of the audiovisual experience, usability, exploratory nature, direction and feedback, and enjoyability. Moreover, they also offer suggestions for future versions. In the development and evolution of previous versions, accomplishments improved significantly. The same method will be adopted in the development of future versions in order to further optimize the participatory design method.

Fig. 10.
figure 10

Mean value comparison of four versions

Full size image

6 Conclusion

VR has become a commonly adopted method in educational applications, but designers and developers typically emphasize the effects while neglecting the technology’s participatory nature and operation-based teaching. Hence, expected teaching goals are rarely achieved. This paper investigates and analyzes students’ experiences using VR educational applications in the belief that VR better suits operation and practice-based teaching activities. Finally, students majoring in subway maintenance and management are invited to involve themselves in the participatory design and development of the BSMTS VR application. Four versions are designed using the participatory design method. According to tests, students’ accomplishments are significantly improved and the use of participatory design in VR educational applications is efficient.