Storytelling Research of Virtual Image Based on User Experience

Abstract

The VR image not only has the watching characteristic of movie, the immersion of drama and the interaction of game, but also its unique conception. At present, a new type of narrative mode adapted to VR image features has not yet reached a consensus. The purpose of this paper is to analyze the importance of research on the user experience to VR image creation, so as to point out the urgency of using iterative thinking in creating works, at the same time, to explore the possibility of collaborative creation by utilizing the creators and users of intelligent tool. The key to construct a narrative systematic frame which accords with the VR image’s characteristics of visual, audition, touch, taste and touch is put forward. This paper adopts the methods of qualitative research, which includes desktop research and network user research as well as the quantitative researches, so as to conduct the validation studies about the habit, feeling, expectation and participation of users when they experience VR image. Finally, the conception of narrative systematic framework that is suitable for VR image and the suggestions for future research direction are put forward.

1 Introduction

With the innovative development of narrative theory and practice in the digital age, many related terms have emerged continuously, such as Cyberspace, Database Cinema, Post-plot Cinema, Participatory Audiences, Transmedia Storytelling, Synergistic Storytelling, Textual Poacher, Participatory Culture, Convergence Culture, Interactive Narrative, Cinema 3.0, All-encompassing Interactive Digital Medium, Viewers, AI Narrative and so on. The theoretical research of virtual image narrative is to think and summarize the newly emerging problems existing in film creation practice in time. At the present stage, the research on VR image narrative mainly focuses on discussing the difference between traditional film and the VR image. The discussion about view concentrates on the condition that when all of the following aspects become insignificant, for example, the “push, pull, swing, shift, follow” of camera’s movement, the “full view, distant view, middle view, close view, close-up shot, extreme close-up” of scenes, the depth of field conversion of focal length, the “out of picture” and “in the picture” of actors and the audiovisual language like “montage”, how to continue to innovate new narrative methods based on the traditional movies, there is little consensus at this stage. During the past ten years, the narrative characteristic, focus and mode of VR image have changed, while the audiences of VR image have changed even more. VR image has broken through the boundary restrictions of traditional movies and constructs virtual space, so that giving the audience a strong sense of presence. Michael Heym, an American scholar, firstly proposed three features of virtual reality in his famous book From Interface to Cyberspace: Metaphysics of Virtual Reality, that is, immersion, interaction and imagination. Therefore, the focus of VR image narrative will shift from “story” and “narration” to “environment” and “interaction”, and the focus of creation will shift from the design of “narration” to the interactive environment the narration happens, that is, the design of “narrative space”. However, the audience in the world of VR image embodies a great degree of “autonomy”. Users can walk into the scene of VR image and then looking at any angle around you, in addition, the users also can interact with person or thing in the scene and create a storyline that belongs to the specific personality, so as to recreate personal life, personal experience in the process of virtual experience.

VR image based on user experience need to establish the creative thinking that is centered by user experience. The goal of VR image creators should always be located on the position of verifying the creation process from a user’s point of view, rather than from a hypothetical perspective. The creator creates a typical user model by observing the behavior of audiences and listening to the attitudes as well as the viewpoints, thus drawing PERSONA, discovering and meeting the needs of users, optimizing the interactive design, and enhancing the user experience, so as to realize the goal of creation. The narrative of VR image should always be a continuous process, and the creators should adopt the systematic and iterative thinking to create the narrative of VR image. Making use of Deliberated Narrative, firstly, creators set a “story goal”, then the users participate in this frame and tell different stories by adopting different strategies at will. Simulation-based Approaches can also be used by different types of users according to their own characteristics, so that letting them participate in telling their own stories. The creation of VR image should break the traditional boundary between creator and user, which is no longer the relationship between creation and watching. Users should also be given the possibility to participate in creation. Through the expectation and experience on VR image, the users’ creative passion can be stimulated, so as to realize their own creative expression on the basis of narrative content provided by the creators. Through the continuous enrichment and perfection made by creators, this kind of expression put forward by the users can form new narrative content, which is connected and intertwined with each other, thus constructing a dynamic growth of story community at last. The creators and the audiences also have established a new relationship mode of collaborative creation. There is not only the watching characteristic of movies with VR image, but also the immersion of dramas and the interaction of games, at the same time, the VR image also has their own unique conception. Therefore, based on the characteristics of VR image, user experience research, iterative creative thinking, collaborative creation mode and intelligent tool, the research goal of this paper is to construct a narrative systematic frame that in accordance with the characteristics of visual, audition, touch, taste and touch with VR image.

This paper is composed of Third parts. First of all, I adopt the method of desktop research to collect papers that are related to VR narrative, conducting interviews of creators who create VR masterpieces, at the same time, I also summarize industry reports and network data, so as to probe into the narrative method and narrative characteristics of VR image works at the present stage. Secondly, through the network questionnaire survey of VR image users, quantitative studies on the motivation, habit and expectation of users to experience VR image. Thirdly, this paper selects the Chinese VR film “The Dream Collector” as the empirical research object, adopting the usability test method to carry on the field observation and the interview to VR users’ experience, qualitative studies on the motivation, habit and expectation of users to experience VR image as well as the possibility of participating in the creation. Several methods of combining analysis, investigation and usability tests are used to study the related researching points, so the reliability of the verification is ensured by verifying the same researching point through the results of different methods. At last, the frame of narrative system for VR image and suggestion for future research are put forward.

2 Research on Motivation, Habit and Expectation of VR Image Users

2.1 Experimental Design

• (1) Experimental objective and significance.

  The research aims to understand the key factors that affect the experience of VR image users, and to explore the motivation, demand as well as expectation of the user experience, so as to provide validation support and reference for the research of VR image narrative systematic frame.

• (2) Questionnaire design and distribution.

  There are 19 questions in total, of which objective questions (16) subjective questions (3). All of the questions are divided into the following eight parts in accordance with the aspect of investigation, that is, watching (3), narration (3), experience (4), interaction (3), attention (6), motivation (1), demand (1), expectation (1).

• (3) Experimental conditions and procedures.

  The online investigation service platform is used to create questionnaires, distribute questionnaires, retrieve questionnaires, export data, analyze data and write reports.

2.2 Data Analysis

• (1) Watching.

Data are showing in Figs. 1 and 2.

Fig. 1.

How many VR films have you seen? And how many times do you watch a VR film?

Fig. 2.

What is the purpose of watching the same VR film for many times? (multiple choices)

The users want to experience the perspectives of first person and the third person, even the second person, so as to play different roles from different perspectives to enhance the feeling of immersion and freshness in image.

• (2) Narration.

Data are showing in Figs. 3 and 4.

Fig. 3.

How long do you think VR movies should last?

Fig. 4.

What perspective do you want to use as watching VR film? And how do you want to see the story when you watch the film?

• (3) Experience.

Data are showing in Figs. 5, 6 and 7.

Fig. 5.

What’s the experiencing place do you like? (multiple choices)

Fig. 6.

What is your favorite way to watch? (multiple choices)

Fig. 7.

Do you like the process of watching VR (multiple choices)? And what other multiple dimensions for experience do you want to add in VR film in addition to the audiovisual experience? (multiple choices)

Human beings have very complex perceptual and behavioral systems, while the VR image extends the mental and spiritual systems of human beings, which can be thought as a comprehensive extension of various organs. Users want to walk, sit, run, touch the narrative contents, they want to experience the unprecedented new feeling brought about by the integration between virtual and reality.

• (4) Interaction.

Data are showing in Fig. 8.

Fig. 8.

(a) When you watch VR film, do you prefer the story telling or the interaction with you? (b) Do you like to be able to choose different development scenarios in the process of watching film? (c) What type of VR film do you like? (multiple choices)

• (5) Attention.

Data are showing in Figs. 9, 10 and 11.

Fig. 9.

(a) What type of VR film do you like? (multiple choices) (b) What are the features of VR film that appeal to you more? (multiple choices)

Fig. 10.

(a) What do you do first when you wear the helmet into the scene of film? (b) Which is the most attractive factor during the process of watching the film?

Fig. 11.

(a) When you watch the VR film, under what circumstances do you take the initiative to observe the environment and discover the details? (multiple choices) (b) Which is the attractive factor during the process of watching the film? (multiple choices)

• (6) Exploring the user’s motivations: why do you want to experience VR film?

As shown in Fig. 12(a), these key words have been mentioned for many times: curiosity, fun, new technology, being personally on the scene, shock and surprise before the feeling of freshness.

Fig. 12.

Keyword cloud map of user motivation and user’s demands

• (7) Exploring the user’s demands: what will attract you in VR film?

As shown in Fig. 12(b), these key words have been mentioned for many times: scene, interaction, plot, creativity, immersion, lifelike, story, creativity, being concerned.

• (8) Exploring the user’s expectations: what do you want from the VR experience?

As shown in Fig. 13, these key words have been mentioned for many times: sense of reality, feeling of freshness, being personally on the scene, interaction, happiness, more sensory experience, excitement, cool, craziness, pleasure, experience that is different from reality.

Fig. 13.

Keyword cloud map of user motivation and user’s expectations

2.3 Experimental Summary

• (1) The user’s demand for watching more than once verifies the necessity of adopting iterative method in VR image creation.

• (2) The user’s attitude towards content reflects user’s needs to enjoy freedom and be guided in virtual reality.

• (3) The user’s demand for experience expresses the user’s desire for a better immersive sense through multiple sensory experience.

• (4) The user’s choice of factors that influence attention verifies the effectiveness of guiding attention conducted by creator.

• (5) The user’s choice of interaction expresses the user’s desire for multiple ways of interaction, at the same time, it also shows user’s willingness to actively participate in the story, which verifies the feasibility of collaboration between the creator and the user.

3 Research on the Audience Availability of “The Dream Collector”

3.1 Experimental Design

• (1) Selection basis and analysis of the film

“The Dream Collector (2017)” produced by Pinta Studios and directed by Mi Li is selected as an object of empirical research. This film tells a story about chasing, giving up and regaining dream and it is shortlisted in the main competition unit of VR in the 74th Venice International Film Festival. The roles in film are designed for two leading roles and two supporting roles, what’s more, 4 scenes (general environment ahead and below the viaduct, inside the house of old man, inside the house of youth and the secret space) are built. The development of the story’s plot is to guide the audience to experience the four scenes in 7 times through the setting of “scene props”, the fade-in and fade-out arrangement as well as the reverse use of Spotlight Scope. Before and after the conversion of each scene, the design of camera location is used to ensure that the audience can pay attention to the leading role’s performance in main visual area when entering the next scene. The sound effect of this film is produced by the sound effect designer Chen Guang, who is “specially employed by Wang Jiawei”. The sound effect is designed to guide the audiences’ attention for many times. The soundtrack of 5 originally created films, “Dreams”, “Opening Title”, “Seasons Montage”, “Christmas Time”, “Ending Credits” are created by Guo Hao, a famous composer, so as to set off the whole atmosphere and guide the attention of audience.

• (2) Experimental purpose and significance

The purpose of this experiment is to explore whether the various kinds of narrative methods used in the VR image “The Dream Collector” can bring the audience experience of immersion, interaction and conception. In order to research on the influence generated by the narrative mode of VR image on the audiences’ experience, the following factors are tested: scene setting, sound, light, object movement, color, emotional guidance, transition follow and so on. The problems can be found through testing the availability, so as to enhance the satisfaction degree and the loyalty degree of audiences by improving the problems, thus reducing the cost of audiences’ experience. We hope to understand the needs and expectations of VR image audiences through experimental research, and to discover the problems existing in VR image creation from the perspective of user experience. This paper also tries to put forward the frame and creation process of VR image narrative and to provide a meaningful guide scheme for Chinese VR image creation.

• (3) Experimental objects and conditions

A total number of 16 users with no or less VR experience are selected to watch “The Dream Collector”. One room is selected as the observation room (as shown in Fig. 14), in addition, one observer, one recorder, one pico all-in-one head display, one liquid crystal television, one camera and one voice recorder.

Fig. 14.

User observation and interview sites

• (4) Experimental procedures

The average duration of this experiment is 25 min, including 10 min of watching and 15 min of interview. First of all, interviewing the audiences before watching the film. Secondly, the audiences are arranged to wear the head display to watch the film and enter into the official experiencing stage (observation and record). During the process of audiences’ watching, the observer obtains same picture as the audiences by watching the liquid crystal television and observes the changes in audiences’ attention during the watching process, at the same time, the recorder records key points in the audiences’ experience through on-the-spot observation and video playback. Finally, after watching the film, the feedback interviews about the audiences’ responses are conducted, so as to get the answers of questions in the test, focusing on the reasons behind the phenomenon and suggestions for the works. By observing what the audiences are doing and what the interviewed audiences are saying to make sure whether the audience act and think the same way. The audiences are surveyed before and after watching the film to assess their expectations and reactions to the work.

3.2 Data Analysis

According to the characteristics of user research, the creators have carried out a lot of presuppositions in the aspects of scene, role, moving object, sound, light, color design and so on. This chapter is based on the observation and interview of users after their watching, so as to verify whether the creator’s presupposition matches with the user’s response. Through conducting the research on users, the ability of creator in predicting and controlling the user’s attention in virtual scene can be further improved.

• (1) Scene design verification

The space can be divided into story space and mental space and the space is composed by the scene as well as the elements of scene (as shown in Fig. 15). According to the user’s response to four scenes in the film, it can be seen that the broader the scene and the more abundant the elements in the scene, the more distracting the user’s attention will be, so that the curiosity of user to explore freely is easier to be aroused. The general environment of viaduct and the interior space of the old man’s home account for the largest number of users and the largest number of visits.

Fig. 15.

Design layout of scenes

• (2) Leading role movement and Moving objects design and verification.

The change of distance between the roles and the audiences is a vertical movement, which will arose and attract the attention of audiences, while the horizontal movement will lead the audiences’ sight to follow the movement. Judging from the user’s experience response, we can see that the guidance of leading role’s movement is very effective, and the user will unconsciously follow the movement of leading role, looking at the direction where they go and the place where they look. When the leading roles disappear into the user’s vision, the users will take the initiative to look for them again. For example: the users look down following the young man Diuji, following the movement of the old man and the dog to come in and out, to enter the house and go out, following the old man to put things on the conveyor belt, thus thinking about where the things go through the conveyor belt. Following the old man and looking back, we can find more and more objects that represent dream are abandoned by others.

Through the response of audiences, we can know the moving object is very effective for guiding the audiences’ attention, no matter how small or unnoticeable the object is. As shown in Fig. 16, for example: throwing guitar, changes of young man’s clothes, falling leaves, dog soap, flying snowflakes and so on, all of these images have successfully attracted the attention of most users.

Fig. 16.

Role and object movement noticed by users in nine scene

• (3) Sound design verification

Users’ reactions also demonstrate that auditory cues are particularly effective in attracting the user’s attention and in preparing for some important events. A 360-degree VR picture often leaves the users unaware of the direction in which the leading role might be presented, at this point, the sound can be used to remind users. When the scene is changed, the well-timed music and sound effect can quickly pull the users back to the scene, this design also provides a short buffer time for the audiences’ adaptation (as shown in Fig. 17).

Fig. 17.

Sound noticed by users in nine scene

• (4) Light design verification

The light and shade contrast is mostly used in guiding users—setting light zone to highlight the places in the scene that need to be focused by audiences, and arranging the position relation between characters and light zone in accordance with the plot, so as to shape the characters. This work tries to use the Spotlight Scope, at the beginning of the film, the whole environment is completely dark, then it gradually expands into the visual range, but it’s not the light, just a range of sphere (as shown in Fig. 18). This shape can be customized and the visual range has a blurry edge, which can cooperate with the user to carry out a variety of visual expansion methods.

Fig. 18.

Light noticed by users in nine scene

• (5) Color design verification

Color can guide the mood and create atmosphere, the change of four seasons, the change of cloudy and sunny weather, the alternation of night and day, the change of black mysterious space and colorful gift shown by color play an interesting role in conveying atmosphere (as shown in Fig. 19).

Fig. 19.

Different color environments

3.3 Experimental Summary

• (1) Differences play an important role in guiding the user’s attention: one element in a scene is different from other elements, that is, the element can attract the attention of users. For example, the difference in grouping, in color, in proportion, in shape, in visibility, in motion and so on.

• (2) The creator should fully understand the difference between users and respect the difference in creation. The elements in the scene and the changes of elements should be enriched as much as possible, and the initiative of focusing the elements in the scene should be left to users as much as possible, so that the user can find more details that are concerned by them in the story and finding their unique pleasure.

• (3) The creator gives the audiences as much space as possible to move around in the story, and providing the users with the freedom degree to choose their own angle of view to discover the plot, so that each user has the opportunity to understand the story personally and deeply.

• (4) From the user interview, we found that many users show an interest in continuing the story. Many users have put forward their own ideas in the role design, scene setting, plot development and other aspects. The active participation of users in the creation of VR image proves the necessity of collaborative creation and conforms to the principle of iterative thinking.

• (5) From the user interview, many users hope to further expand the storyline and perfect the story of baseball gloves, skateboarding and football existed in the story. At the same time, I hope to increase the dream story related to me.

• (6) From the user interview, it is found that many users want to increase more participation opportunities, such as having users to participate in story from the first or second perspective, so as to pick up something they are interested in and experience a story about what happened to this object. The users can fix things by themselves, in addition, they can give gifts to people who have the same dreams and make dreams come true again.

4 Conclusion

The framework of VR image narrative system needs three important parts: adopting the VR image creation method on the basis of user experience, at the same time, using the collaborative creation platform of intelligent tools between creator and user to establish the VR image creation mechanism of iterative thinking. Further use of intelligent tools can achieve the natural interaction mode and socialized experience mode. The user research methods can be divided into 3-dimensional framework: Attitudinal vs. Behavioral, Qualitative vs. Quantitative, Context of Use, among which there are several methods that are adaptable to VR image user research, that is, participatory design, focus groups, interviews, eyetracking, moderated remote usability studies, unmoderated remote panel studies, concept testing, customer feedback, A/B testing and so on. Deliberated Narrative or Simulation-based Approaches can be adopted, at the same time, using speech recognition, graphic and image recognition, natural language processing, deep learning, artificial intelligence and other technologies to build the collaborative creation platform, so as to realize the vision of collaborative writing among creator, user and AI three aspects. Establishing the iterative thinking mechanism, in order to allow users to truly become creators, performers, improvers, experiencers, and sharers of VR image. Sharing the experience of VR video world with people around you will perpetuate the commercial value of VR image, which has a certain degree of persistence.