1 Introduction

Methodology of a museum in which we hand real objects on to the next generation has progressed markedly by the virtual reality (VR) technology. A new method of an interactive exhibit [1, 2] in addition to a static exhibit has been explored at the museum by incorporating digital technology [3, 4]. A change from only seeing objects to experiencing them is gradually introduced to the field.

One of objectives of a museum is to organize and transfer mostly historical ‘object’ from the culture, fine art, science, industry, and life, to the mass of people. As a method to serve for this purpose, valuable objects were accumulated and preserved to show them to the public in a way that has been referred to as a museum format. A basic form is to transfer objects to people in the future, and to transfer something implied by those objects. This is an only way to transfer complete value of the authentic entity to the public. No other way may outweigh the method. The typical form is the exhibition of an original real thing (the primary source material) which can only be seen at the specific museum in the word, which is a symbolic style of the museum. The genuine object is an absolutely important and primary existence, and that is an untouchable for preservation. The authentic objects which can not be installed in the museum (e.g. an architecture or what already lost) have been embodied by physical models and transferred to following generations. The physical embodiment allows people a comprehensive use of multisensory perceptions, which introduces an extremely effective impact as an actual object (if succeeded). These are the most effective methods to transfer an object that has a three dimensional structure. They can transfer information not handled by an indirect representation of literal or visual media (the secondary material). The VR technology among digital technology allows humans to perceive the three dimensional structure of the real space as it is, so that it can best contribute to the transfer of object as discussed above. In addition, the interaction between objects and humans can be replicated by the VR technology. Since objects are usually artifacts that existed with humans, the interaction with humans has been essential for their existence. Interaction with objects is an experience for humans, and the objects are naturally exists for an experience. The real situation how objects are used, the type of interaction, the context of use, the sensations on phases, and overall impression produce the experience. To transfer such an experience is the essential inheritance of the cultural property of objects. Until now, objects themselves have been transferred in a museum since such an experience discussed above could not be handed on. The VR technology has enabled the museum to transfer the experience of objects.

2 Experience Transfer

There are two methods to transfer and replay the past experience. The first one is to build the environment of the past to experience the situation. If the same environment where the past user immersed is reproduced as similar as possible, almost the same experience is possible as the past users performed by using reproduced special environment. The traditional method for this direction was to build a real park in which original buildings are brought and repaired, or replicated to provide the past environment to the people in the present. The replication of a world by the VR technology drastically changed the methodology. It enabled to provide a space and an object that are flexibly reconstructed, and allowed people to experience the world.

The production of a space allows the pseudo experience of a specific architecture, a vehicle and a tool that are similar to original ones. The participant experiences the world by oneself in the space that is equivalent to that the past people encountered. Although the experience is similar to that the past people obtained, it is not the experience of the action taken by one’s own choice in the specific context in a particular background of the day. Even though the environment was the same, the experience is not the same one which the past people performed and obtained. The experience is not just what the past people really obtained.

The second way is to prepare the experience of the past itself for transfer in that a traditional reliving experience is involved. Although a reliving experience is to capture the experience of others vividly as if he/she oneself performed the experience, a traditional reliving experience was based on reception of information through a book or a movie audio-visually. This is just looking into the world the other person experienced during the presentation of a particular story. This activity is basically understanding the experience of others. It is an experience of observation as a bystander, although nominally toled as if he/she experienced by oneself. Of course, it is one form of experience transfer in a sense. However, a bodily sensation that can not be transferred exists inevitably in this type of reliving experience since the experience is inherently obtained through a body motion.

The reliving experience of an architecture or a tool may be truly obtained by doing completely the same physical motion as the past person did using the body to move around or manipulate in the same way. Thus, to transfer the past experience as it was is to reproduce the spatial motion of the person, specifically the past bodily motion, on to the other person’s body. Of course, sensory information of the environmental space is recorded and replicated as accurately as possible for the other person. This requires the same body motion of the past person in the space to be replicated completely, however it is usually impossible and inefficient to make the same body motion of the person who relives the past experience. Then, the problem is whether if the VR presentation of bodily movement is possible. A procedure for this presentation is first to record actual experience and then to reproduce it. The recording method has been discussed and developed in the context of lifelog.

A valuable experience for the museum may involve a special body motion skill such as an Olympic level athletic skill, and the body motion of a person with a particular skill which is recognized to be preserved to the next generation. More generally, a walk experience of world tourist sites or adventure activities in the frontier will be accepted as valuable experiences for many people. Since the diversity of bodily activities of humans is huge, to share such rich experiences produces a new unprecedented value.

3 Bodily Reliving Experience

We consider the transfer of experience that involves the body movement. The objective of the study is to make the past spatial activity of other persons the experience of the follower who traced the activity. We call this replication of the body motion the bodily reliving experience.

Since the experience is performed by the body, the body of the follower needs to be restricted to the motion of the precedent, and subject to it. Reliving is to become another person at the time, which means clearly that the follower person will not have a free experience. The experience itself already exists and the follower traces the experience, as the movie observer does not choose the behavior of the actor. The bodily sensation is also provided in the form of recorded motion in the past. The experience obtained is what the other person performed in the past real space. The reliving experience of this is a replication of a reality as what is created in the VR technology. The feature of the reliving experience is that the reality is filtered by the precedent person.

Fig. 1.
figure 1

VR experience

Full size image
Fig. 2.
figure 2

Bodily reliving experience

Full size image

Traditionally, the reliving media was based on a replication by a literal book or a movie. This paper proposes that the fusion of the body of an experiential follower itself and an advanced sensory display device makes a new presentation media for the reliving experience. We hypothesize that the own body is a part of the media. Figures 1 and 2 indicate models of a usual VR experience and the bodily reliving experience. A center circle indicates the human CNS (central nervous system) for the perception and cognition, and the ring means the body of the human. A half ring indicates the artificial VR space.

In Fig. 1 the VR space means a space of experience created by multisensory display. The experience is established by the interaction between the human (the CNS and the body) and the space. The experience will have an equivalent value of the real experience when the user feels the sense of presence to the space.

Figure 2 shows that the body of an experiential follower represented by a ring is fused with the VR space. This means that the body of the follower is a part of the presentation media that is a VR space by multisensory displays. This is a structure to transform the other person’s experience on to the own body experience by moving the body on the media side separating the body from the control of the CNS; the body receives stimulus for presenting the virtual body motion. Now that the CNS, the subject of cognition, obtains the bodily reliving experience rendered by the space and the body. The sense of presence in this setting is different from that of usual situation. In this sense, we call this the ultra reality. The approach to form the bodily reliving experience is largely different from the usual case of the virtual reality where the past environment is rebuilt for a new experience.

The reproduction of the environment of the virtual reality focuses an objective space replication, then the replicated space is conceptually independent from the person who receives the experience. The reality by the virtual reality is an objective reality, while the world created for the bodily reliving experience is a subjective reality of a predecessor. The latter reality has been given a meaning by anyone else; the reality was constructed by other people. The reality to experience is already established as in watching a movie. The difference is that the world is always rendered from the first person perspective.

4 Peculiarity of Bodily Reliving Experience and Hypothesis

4.1 Voluntariness

A method to realize the reliving experience is based on providing a variety of information to the five senses in addition to traditional audio visual presentation of the situation. Specifically the proprioception and tactile information of the body are displayed while the real body is totally reserved for them if the whole body is used as the media to present a physical experience. Although we have obtained an empirical knowledge through the TV or a movie in an ordinary reliving experience, in the bodily reliving experience, the bodily action of the other person needs to be reproduced through the body of the experiential follower. The body of the follower is a device within the presentation media to produce sensations. The reliving experience has a peculiarity in which the bodily experience is the first-person experience, and at the same time it is regulated heteronomously by a predecessor as discussed above. An ordinary experience is created and obtained solely due to the active physical motion selected by oneself in facing the space of the real world. In this sense, the reliving experience based on the existence of other person’s experience conflicts the nature of the active experience of the self. The reliving experience may not be recognized as an experience in that meaning. The point is a voluntariness of the experience. An experiential follower can not have voluntariness since the reliving experience is to trace the experience of others. If the essence of the reliving experience is to have the scenario of the other person, it may contradict the voluntariness essential for an ordinary experience. It is an interesting argument indeed; this inconsistency stemmed from the prerequisite we placed above where the self body is not one’s own.

4.2 Creation of Virtual Body

The first problem to install the bodily reliving experience is to what extent the body of the experiential follower should be identical to the predecessor’s body. The body of the follower is driven by the VR display device externally, which imparts the sensation of motion perceived and attended. A trivial method to make the sensation identical to the bodily sensation of the predecessor may be that the follower should move as the same amount as the predecessor. However, it is not an efficient solution, since it is not actually possible to force the body motion of the follower to be identical to the predecessor’s motion completely. The degrees of freedom of the human body motion is not small and the speed of the motion is fast. In addition, what is essential here is not the replication of the body motion itself, but the creation of the sensation of body motion. The point of the bodily reliving experience is to avoid the complete copy of motion by an efficient stimulation method to multisensory sensation related to the body motion.

The bodily reliving experience we pursue is essentially the projection image of the self body. It is the sensation of oneself who experience the objective space, and also the sensation of the self body. The self body is evident existence in the real world generally. The existence is definite, however it is only evident through the sensory system which has ultimate importance. However, this confidence is disrupted if the sensory system is fooled. Botvinick [5] demonstrated the illusion in which the user loses the body image by repeated false input to the sensory system. This phenomenon to recognize a rubber hand as ones own suggests that the body image is able to be projected to the other object.

Fig. 3.
figure 3

Virtual body by backward projection (revised from [6])

Full size image

The rubber hand illusion is an incredible fact since the hand is the key part of a human body to interact with the world. The fact that the body image is not firmly defined is usable for the creation of bodily reliving experience. The self projection by which the self body is visible in a virtual space is one of the VR features [7]. It is important in the sense that it enables interaction with the environment, and positions oneself in the space to introduce the sense of presence. In addition, it provides visibility from other people. The projected self is usually called a virtual body.

An avatar is also the body image of the player in the net game/space. The both represent the self body image, and they are the media of interaction with the space and other avatars. This type of projection represents the image of self body in the other space based on the intention of the user. This is a forward projection of the self control to the other space. On the other hand, the bodily reliving experience is a reverse (backward) projection where the past experience of the predecessor is projected to the current self body in the real world. Figure 3 indicates the relation.

The usual forward projection is used to drive body motion in the real space by a command from the CNS holding a self will. The real motion is projected again to the avatar motion in the other space. The avatar is the self body of the other space. The experience in the other space is created through the avatar. On the other hand in the backward projection, the experience in the past other space is projected to the current self body. The self body control is dominated by the external device and set under the passive state. The hypothesis of the bodily reliving experience states that the body imageFootnote 1 created by the self body motion should be that of the precedent body. The body image stored in the CNS is virtually replaced by projection of the other person’s body.

If the mapping of the body image from various personalities to the follower is ideally performed, the body image may be regarded as virtualized to accept nominal diversity. In Fig. 3, the virtual body in the CNS indicates the projected body image. A hypothesis may be presented that a great number of experiences would be obtained if the virtual body is rendered in any forms using the self body as a medium. The validation of the hypothesis has been performed in a small step.

5 Experience Simulator

5.1 Multisensory Display—FiveStar [8]

If the virtual body is controlled and projected, the system is able to create a variety of experiences. The system may be called an experience simulator in a sense where many experiences are generated by using the self body and its senses stimulated by a multisensory display.

Figure 4 indicates the direct-contact part of the multisensory display. The vestibular stimulation is produced by the seat motion. The proprioception of the lower limb is stimulated by the pedal and slider. The vibratory stimulation is added to the user at the skin of the back, thigh, and the sole of the feet. The visual display of 85-in 3-D (passive) LCD monitor (KD-84X9000, Sony) is placed about 120 cm in front of the user. In addition, a wind display of four circulators and a 5.1-ch audio system are installed around the seat. The user sits on the seat wearing polarized glasses, headphones, and shinbone stimulators to receive multisensory information. The visual angle is about 76\(^{\circ }\) in the horizontal direction.

Fig. 4.
figure 4

FiveStar Devices

Full size image

Figure 4 shows the motion seat and the user. The seat provides three dimensional motion (lift translation, roll, and pitch rotational motion) to the participant with three electric actuators (PWAM6H010MR-A, Oriental Motor, Inc.). The translational motion of the actuator is up to 200 mm/s with a maximum amplitude of 100 mm (position resolution: 0.01 mm) within a load of 600 N. The rotational speed is up to 1.25 rad/s with the maximum rotational angle of 0.3 rad. This three-dof motion creates motion stimulation of the head of a sitting user to impart controlled vestibular sensation in accordance with the rotation at the joint angle of the lower limbs of the real body to project a virtual body.

5.2 Bodily Reliving Experience—Diversity of Expression

Experiences replicated by the FiveStar system should be diverse specifically in the museum. To examine representation capability to provide bodily diversity created by the system, six conditions for presenting walking sensation were compared. Figure 5 shows the views that the participants watched. Eleven participants recruited in the university performed the experiment.

The objective of the experiment was to evaluate the sensation of walking body of six different body types: the self body, a slim body type, a fat body type, an adult (CG) body type, a child (CG) body type, and a sumo wrestler (CG) body type. Since each participant has a different body, the seat motion needed to be differently adjusted to present the sensation of walking in the six body types. We asked the participant to adjust the seat motion first to best imitate the walking in those bodies. Note that only the seat motion was used in this experiment to clarify the effect of vestibular stimulation. (The lower limb stimulator of proprioception and the tactile stimulators were not used.) The number of target bodies to represent was six, and the number of media conditions was five, then thirty settings in total to rate the sensation of walking were introduced in the experiment. The motion of the seat was regulated to the trajectory that each participant adjusted. The participant was instructed to get into (be immersed in) the target body and to walk as if he/she were in the body.

Fig. 5.
figure 5

Walking movies. (top) Left: first person view (1PP), center: male live video (3PP), right: fat man live video (3PP), (bottom) left: male adult CG (3PP), center: child CG (3PP), right: sumo wrestler CG (3PP)

Full size image

After the adjustment of the seat motion for all conditions, the participant was asked to rate the walking sensation on a visual analogue scale from ‘no walking sensation (not in the walking state)’ to ‘the real walking sensation (in the walking state)’. The ends of the line segment were marked 0 and 100.

Fig. 6.
figure 6

Walking sensation produced by different stimulations in visual and seat (vestibular) presentation.

Full size image

Figure 6 shows the result of the rating on the sensation of walking as the person shown in the view (with the visual mode) or designated by the experimenter (in the first person view mode). The result clearly indicates that the seat motion contributed markedly on the sensation of walking. Of course, a visual presentation has a certain effect on the sensation of walking as shown in the rating of the no-visual/with seat motion that was lower than the visual/with seat motion, however the seat motion had a very large effect. If the seat motion is not provided, the third person perspective was better in imparting the sensation of walking.

The rating varied differently between a self walking sensation and the other’s walking sensation. For the self-body walking sensation, the first person view (1PP) with seat motion condition produced the highest rating of a walking sensation. It was a natural view for the participant. As for the other’s body walking sensation, the sumo wrestler 3PP view with seat motion was the highest rating. This may be due to ease of qualitative understanding the bodily feature and motion based on the visual image by which the motion sensation of other’s was collected. Actually, the visual presentation provides most part of information on the walking of the others. It is suggested that the first person perspective is appropriate for replication of the self body walking, while the third person perspective is suited for the others where the feature of the bodies are easily understood.

6 Conclusion

The digital museum is a new direction of object exhibition, and its flexibility and interactivity of information presentation using ICT and/or VR technology will introduce unprecedented development of the museum exhibit. Visitor’s bodily experience is an essential direction for the purpose of the original museum objective. The present paper provides part of fundamental knowledge for sharing the experiences of action in the world. The motion of a seat could effectively enhanced the sensation of walking that may be applicable to the experience of seeing a large object such as architectures in the word heritage. As a preliminary result, the first person view of the participant with the seat motion created the highest sensation of walking that would introduce a vivid virtual tour of the world.

The next goal of the bodily reliving experience is to build a museum of body by projecting any body with body virtualization. It is to involve, as an objectives of VR, the conquest of the body in addition to that of the space and time. The body of the follower is made transparent and replaced by the other person’s body to obey the action of others. It has special characteristics in that the voluntariness of the body motion is lost although it is the experience of the self. It is related to cognitive property of the body ownership to define what is the self body, and the agency to define voluntariness. To provide an explanation between these issues and the bodily reliving experience is one of the important future works of the study.