Keywords

1 Introduction

In digital content era, thousands physical bricks of information are converted to become a digital content. There are many devices and technologies which allow users to access data and to visualize and to interact with virtual world [1]. Virtual reality (VR) technologies have been used for various purposes. Virtual museum is one branch that applies VR for demonstrating actual objects or architecture with mock-up as three-dimensional models [2]. Virtual exhibition applies VR concepts with multimedia information and computer graphic technology. The design and development of virtual museum’s exhibition also considers interactivity in the system to allow users to learn about collection of history with content that they are interested in [2]. Virtual museum becomes a large multimedia center where users easily access information and can convey the contents through interaction between users [3]. It allows users to gain knowledge from the museum more easily but the keystone remains a good storytelling. We need a storytelling to drive virtual environment behavior while the interaction is the modality to follow this story. There is a kind of disconnection between the initial specification of the exhibition and its implementation and delivery. A good product development expects to fill this gap.

Storytelling becomes a major part to create interactive content for a virtual museum and use stories as instruments for suspenseful knowledge transferring [4]. In order to create a virtual museum exhibition, storytelling platforms will be the tool to help developer defining elements in a scene and makes stories. Non-linear storytelling is a kind of interactivity content [5], users get involved into the story to interact with a content. To design and develop an exhibition as non-linear storytelling, the contents will be considered in term of user interaction where interaction system is concerned. Here, devices and interaction techniques are also considered for designing interactive contents. Most of virtual museum’s exhibitions have been designed limited to selected devices. Maintenance and service of interaction system will be restricted to the contents of all development process that will be complicated for exhibition management due to the fact that there is no lifecycle management for an exhibition to support technologies change over time.

Therefore, storytelling platform acts as the major specification of any exhibition. We propose high-level abstraction to define all event and action in a scene that must be translated into low-level technical user interactions. Thus, end user (visitor) interactions will respect high-level abstraction whatever devices are finally used. The platform aims to be part of the lifecycle management of an exhibition.

2 Related Works

Authoring tools are majority to support an application development. We focus on manual narrative authoring [6] and interaction system is also concerned. Hence, the platform will provide not only story generation but also VR interaction system. Storytelling platform for virtual museum exhibition is related to VR platform which mainly provides functions to develop virtual environments and device plugin support. Educational game-authoring tool is another platform concerned to storytelling with an enjoyment goal. Advantages and disadvantages of each platform will be considered respect to application in the field of virtual museum exhibition.

2.1 VR Platforms

There are several platforms for creating a VE by a developer who handle several kinds of devices. VR Juggler [7] is a virtual platform providing a virtual reality environment independent of operating system. This platform supports many graphics engines and network distribution through the NetJuggler module but no high-level support for application distribution. AVANGO/NG [8] is a distributed scene graph framework. It applies a generic field container programming interface based on OpenSceneGraph and develops an entire application with Python scripting support. Vizard [9] is VR toolkit for interactive 3D content, scripting with Python undertaking many commercial devices and also achieving extraordinary rendering including multi-user, clustering, and multi-channel abilities. InVRs [10] implements Collaborative Virtual Environments (CVEs) approach in the form of a highly extensible, flexible, and modular framework with pre-defined navigation and interaction techniques. Configurable via XML, it has a network distributed virtual world using OpenSG as a scene graph engine. pSIVE [11] platform allows easy setting up of VEs with interactive content and also has a generic model to be applied in different contexts by non-expert users. pSIVE is a good platform to study device switching with pre-defined interaction techniques that selection and navigation techniques will be used to compare potential of each device. 3DVIA Studio [12] is an interactive 3D application platform by LUA scripting language. There are many GUI tools to work on models and animations in scenes. Providing integration with VR/AR systems and also realistic interactive 3D by advance rendering, physics and animation engine support to create immersive virtual reality project. Unity3D [13] is high performance 3D rendering and physics engine for game developer which can be applied for creating VR application. With various device plugin supports, Unity3D is able to export an application towards many operating systems, useful for interaction techniques studying. However, the application development remains based on pre-selected device. To switch device we have to edit programming part for a correct application control due to devices restriction.

Most VR platforms are proposed to be a tool for VEs development and support developer to handle devices configuration. However, storytelling to support and to organize story structure is still lacking. There is some platform providing high-level abstraction to define object behaviors in general for interaction which could be improved as educational game or storytelling platform.

2.2 Educational Game-Authoring Tools

WEEV [4] is an educational game creation framework. The system is built upon <e-Adventure> [14], a game authoring platform. WEEV implemented three tools to edit main element which are Actor editor, World editor and Story editor using visual programming language to represent a story based on interaction between user and game. Thinking Worlds [15] is an authoring platform for VEs focused on creating structured learning experiences. It is a commercial tool that facilitates development of serious 3D games. This platform attempts to create more complex scenes which required more 3D rendering engine and also camera and character control through the scene. Adventure Author [16] is a platform based on visual programming language as WEEV is. The development of this platform focuses on linear stories without real interactivity. Thus, it is easy to create educational stories rather than games. StoryTec [17] is another visual programming language like Adventure Author for serious game creation. However, it supports to create non-linear stories to encourage creativity rather than educational game. StoryTec has powerful expressiveness for story creation via a Story editor enabling non-linear stories. Storytelling Alice [18] is proposed to teach programming concept for students but also provides many features to create animated stories. Storytelling Alice includes high-level animation that enables users to program interaction between elements which provides 3D character and scenery with custom animations. Adventure Game Studio [19] and Adventure Maker [20] are platforms for game developer with 2D animation support. In contrast, these platforms do not provide tools for storytelling or educational features which are not targeted to educator. These platforms have GUI editor to support content editing on the scene based on point and click gaming. If we considered both, Table 1. shows educational game-authoring tools have abilities for developing structured learning and VR platform lacks. By the way, educational game-authoring tools tried to advance their engine to be high performance in 3D while most VR platforms support these features. The creation of an exhibition in virtual museum needs to have a storytelling platform which supports of educational features for user learning but also advanced 3D engine and device switching. Therefore, it would be great to combine good features of each platform together. Furthermore, development process is still lacking of lifecycle management which should be considered into storytelling platform. The process should carry out according to product lifecycle phase as well as conceive, design, realize and service. The design of a storytelling platform for virtual museum exhibition will have methodologies based on exhibition lifecycle management which improves exhibition development process as described in next section.

Table 1. Existing environment capacities
Full size table

3 Storytelling Platform

As a top–down design of engineering workflow, storytelling platform is focused on high-level functional requirements. High-level abstraction is decomposed into lower level structures and specifications until the physical implementation layer is reached. The specification phase is the definition of exhibition requirement. This starts with the design of scene components, defining basic entities and some event controlling and then high-level behaviors will be defined as action logics. From this specification, storytelling platform will be developed. Then the design phase is where the details design and development starts. An exhibition will be developed on the platform as a storytelling project. Next it comes the realization phase; physical implementation is handling here. The complete project will be launched into the low-level functions where interaction system is used. Finally, the service phase serves for interaction system providing system maintenance as well as reuse of project. Collaborative Virtual Environments (CVEs) may support this phase to transfer exhibition project to a specific interaction system. When technology is changed former exhibition project is transferred again for new technology.

3.1 Development of Storytelling Platform

The design of scene components will be applied to develop storytelling platform. The Viewer is a tool to display virtual environment in 3D. All entities are handled in the Asset manager while event and action part will be specified in the Event editor. Here, high-level abstract model will be defined into a scene with events and actions corresponding to storytelling needs. When all entities in a scene have been determined, an execution part will be addressed by a runtime engine to perform interactive stories in real time and interact at low-level connection. The translation between the high-level abstract model and low-level connections should be automated so far as we can do it. Figure 1. Shows the process and architecture of platform which has the following basic components:

Fig. 1.
figure 1

Exhibition development architecture

Full size image

A Viewer:

A viewer is a display window to present entities as virtual environments and cooperate with Asset manager for entities editing. Viewer provides graphic engine to handle rendering and lighting in the scene. There are 2 modes that use the same window which are storytelling mode for editing and runtime mode to present the result of storytelling mode.

Asset manager:

Asset manager is a tool panel to carry out all entities imported into the scene. When each model has been imported, entities will be loaded into a manager panel. Entities can be a representation of a building, artifact, actor, information or avatar. These will keep entities as scene graph data structure.

Event editor:

Event and action have been designed already then implemented to the visual setting in the Storytelling platform. Event editor provides GUI panel to connect each event component together: Entity, Event and Action. Entity will be selected directly in the 3D scene while events and actions are selected in a GUI template. Condition is added into an event for flexibility of logic assignment to be a rule for Action. Event editor is the tool for defining high-level abstractions for every scene, events and actions for all entities provide a common template for every storytelling.

Runtime engine:

Provides runtime mode in the Viewer to test the result from storytelling mode when event and action have been assigned. Event and action are interpreted to execute and perform entities in the scene in real-time interaction. We see how the story is going on and if interaction works well or not. Low-level functions can be implemented and tested on this stage. Before to execute the runtime engine, the connection with low-level interaction system is exported.

Low-level connection

is a part of a top-down design workflow. High-level abstraction is decomposed into a lower level for physical implementation. This is compatible with vertical transformation as propose in Model Driven approaches [21, 22]. High-level abstraction related to the entity as defined while event and action allow developer to connect the device which is independent of event setting anyway. Here, storytelling platform enables device changing through event and action by Event editor and low-level functions are applied by interaction follows device’s characteristic as shown in Fig. 2.

Fig. 2.
figure 2

Vertical transformation as model driven approach

Full size image

3.2 Development of Interaction System

Interaction system on our Storytelling platform uses Collaborative Virtual Environments (CVEs) to be the tool for running an application (This tool is internally developed at the laboratory). In order to connect devices to an application, CVE is expected to manage device connection in the system through a CVE’s server. Moreover, interaction techniques (ITs) are methods to let user perform a task within VEs via user interface. ITs are influenced by input devices, but the same device may be used for many ITs for the same task [23]. On the other hand, it may be possible to implement a given IT using several different input devices. ITs studying is essential for system development to use of various devices and to improve system when new technologies arise. Therefore, development of interaction system through CVE will support exhibition lifecycle management especially during maintenance and service phase.

ITs design:

Interaction tasks are the method to interact with VEs while ITs are the way to perform interaction tasks. To design ITs, the first step will begin with ITs classification. Universal interaction tasks are partitioned into separable subtasks, each of which represents technique component calls taxonomy. Taxonomy is not only a characterization of interaction tasks, but also as a design space. Taxonomy is the intermediate-level of ITs implementation to the lower level. In our Storytelling platform, interaction tasks are separated to be 3 genres of ITs which are Selection, Manipulation and Navigation [21].

Device setting on platform:

In Event editor, only connected device can be chosen and then ITs will be selected for managing scene behavior. A same input device can select different IT already designed and the same ITs can be used with different input device also. Storytelling platform provides ITs management. Some ITs are probably reuse with other devices by keeping all ITs into a database from where each device can be retrieved.

Device management:

Usually, CVE is used for collaboration and interaction of multiple users to work together on VR application [24]. However, CVE does not provide only multiple users in the system, but also enable using multiple devices together in the same application. Thus, CVE is the tool which facilitates flexibility when using devices and supports device management of Storytelling platform. Interaction system of our platform will address low-level implementation of device which supports devices service while high-level abstractions maintain the content events and actions in the exhibition.

4 Case Study

We expect to build an exhibition about history of Prathat temple, the crucial temple model of South East Asia. There have been various legends of Prathat temple. One legend mentioned that the Buddha relics (the teeth of the Lord Buddha) have been kept inside. The relics were moved from Tontha Buri by Prince Thanakuman and Princess Hem-chala. It has been assumed that the original form of Prathat was Mondop, a structure with four arches and a pyramidal roof topped with five tiers as Srivijaya style. Then, in 1700BE, Theravade doctrine of Lanka has been prosperous and spread to Nakhon Si Thammarat, as a result Chedi of Theravade was constructed to cover the original one. This becomes a typical model of Theravade in the south of Thailand and influentially spreads to adjacent areas.

The storyteller expects to make an exhibition to tell this story with Storytelling platform by transforming history in term of text to be an interactive learning exhibition. First, all entities models are loaded into the database as historical model which are able to edit transformation value within the scene. The story is modelled at a high-level abstraction by Event editor which has 3 main components: Entity, Event and Action. Each component is drag and drop node on a canvas. We make relations by linking nodes from Entity to Action through Event which is editable on setting menu. The example scene in Fig. 3 has 4 Entities which are two Actors Prince Thanakuman and Princess Hem-chala and two buildings named Mondop and JDPrathat. At beginning of the scene, we set two actors moving from Marker1 to Marker2 with 5 s duration to present the relics were moved from Tontha Buri by them. When approach to Marker2, Mondop will appear to show the original form of Prathat which kept the relics inside and these actors will fade away at the seconds 11. Then the building JDPrathat will be faded-in and we set more input event by adding Selection with respect to device. When this model is selected, the scene will play clicking sound and show message as defined.

Fig. 3.
figure 3

An example of Event editor on drag and drop canvas and selection setting for device and IT management.

Full size image

Selection event will depend on selected device which can be setting up the interaction technique to select the Building. Here, indication to object is a method to point to any object while indication to select is a selection method. These parameters of IT have been defined respect to device which can choose directly from setting menu. When the device is switched to other one and completely defines new IT. All actions that related to the event still work even device has been changed by flexibility of high-level design. In this example, Selection defined a mouse to be a default device. This platform allows storyteller to change device to other one e.g., a Haptic arm and assign indication to object and to select which calls IT. Although storyteller would like to use the old device, the platform provides capabilities to manage IT directly and still using the same logic. Finally, specification of ITs will be translated into executable interaction system follows selected devices. Figure 4. Shows high-level abstraction can be transformed the story to various devices.

Fig. 4.
figure 4

Transformation of storytelling model to low-level functions with various devices.

Full size image

5 Conclusion

We propose new methodologies for exhibition development in VM to support its management. We believe that it will also support the whole exhibition lifecycle. Devices organization and contents structure are addressed by the Storytelling platform which has efficient management to support alternative interaction in general. Engineering workflow as a top–down design provides a high- level interaction abstraction model where storytelling models specific interaction behavior to drive interaction in a scene. Devices and ITs are handled through a CVE.

According to product lifecycle phase, concept design is supported by our Storytelling platform which specifies the exhibition interaction. Service phase is processed after an evaluation method to keep an exhibition maintained and operational. We claim that Storytelling platform provides an adaptive interaction system which will supports user learning when applied to VM. This implementation is useful to deploy not only for the development of VM exhibition, but also for industrial engineering where interactive content and interaction system are required.

At this step the overall architecture of our exhibition development system is ready. A first demonstrator provides a storytelling modeler and we have a first converter of a story into a runtime environment. We continue developing this platform and we will analyze devices and ITs for each story to classify their specific potential to support user learning in VM.