Keywords

1 Introduction

Scroll Painting, as the most general type of Chinese traditional painting, has a narrative feature, related historical stories or image legends. These scroll paintings are not only beautiful artworks but also precious historical data. Viewers will learn relevant culture and history while appreciating paintings.

However, the public in modern society is more interested in novelties, and the attention of traditional Chinese paintings is getting lower and lower. As multimedia technology becomes more developed, we hope to find a kind of gamification design method to encourage viewers to explore these paintings proactively. Through the interaction, viewers can understand the stories of these traditional paintings better, so as to feel the charm of traditional Chinese paintings. We chose the famous Chinese scroll painting “Han Xizai Evening Banquet” as an example and designed “Digital Han Xizai Evening Banquet”.

Fig. 1.
figure 1

Han Xizai evening banquet

Full size image

1.1 Chinese Scroll Paintings

Chinese Scroll Paintings use large aspect ratios and composition to convey a special visual experience to the viewers. Unlike western oil paintings, Chinese Scroll paintings transform spatial and causal relationships through elaboration.

The most significant feature of Chinese painting is the cavalier perspective method (see Fig. 1). Different from the focus perspective method, cavalier perspective moves the viewpoint as the artist demands, and the scene in the picture would not be limit by the field of view. One picture may have several viewpoints. In addition, the different time and space in a painting is another feature of Chinese scroll painting, which means according to the story, events and characters appearing in different time and space in one painting, one character may appear on a picture many times. These two features provide conditions for storytelling, such as “Han Xizai Evening Banquet” and “Lo River Map”, and draw a lot of scenes from right to left to express the whole story.

1.2 “Han Xizai Evening Banquet”

“Han Xizai Evening Banquet” is one of the most famous Chinese traditional paintings in China. It is painted by Gu Hongzhong in Southern Tang Dynasty. This picture shows the scene of an evening banquet organized by Han Xizai, who is a famous chancellor in Southern Tang Dynasty (see Fig. 2).

The whole picture divided into 5 scenes: Listened to Pipa, Appreciated dancing, Take a break, Enjoyed melodious music, See guests off, showed stages from beginning to ending of the banquet, the painting style is graceful and meticulous and the composition of the picture is unique. There are many historical people and furniture in this painting, this painting is not only a beautiful painting had aesthetic values, but also a kind of precious historical data.

Fig. 2.
figure 2

Schematic of Cavalier perspective. One picture had multiple viewpoint.

Full size image

2 Related Works

Until recently, some museums and companies have also used traditional scroll paintings for digital exhibitions. Compared with traditional exhibitions, the digital exhibition avoids possible damage to the original things from touches. It also has the ability to reveal background knowledge [1]. At the 2010 Shanghai World Expo, Crystal Digital Technology Co., LTD made an animated film “Life Along the Bian River at the Pure Brightness Festival” (Bian River scroll for short, it is the most famous Scroll paintings). This exhibition displayed 3D animation on a giant white wall of 128 m long, using 12 projectors to present life of the ancient people. Likewise, the National Museum of China exhibited an animated film “The Qianlong Southern Inspection Tour”, displayed on a screen, which is 30 m long and 4 m high. However, both projects focus on watching, but there is no interactive design.

In 2007, the National Palace Museum in Taibei exhibited digital Bian River Scroll exhibition that could interact with it. When visitors touched some specific areas in this digital painting, the corresponding movies would jump out of the painting and start to play. Although this exhibition possessed interactive function, it had a poor immersive experience.

The Palace Museum in Beijing also exhibited an interactive project [2]. On a large screen with a higher resolution, viewers can appreciate the original Bian River Scroll image without animation. Viewers zoom in on the image with two hands and enjoy more details. The voice box would play corresponding environment sounds and dialogues, which helps viewers to understand the knowledge about the painting.

These projects are attractive and gained favorable evaluation, but they all have a monotonous way to interact with visitors. Visitors interaction is limited to gesture and hearing, and there is no way to experience diverse ways to enjoy these paintings. We hope the interactive patterns can be integrated into space, and it will take more novelty experiences and enhance the immersive feel.

3 Concept and Design Method

3.1 From “Seeing” to “Participating”

Each Chinese painting has many brilliant details, and they are emphasis area that the viewer should follow. Such as “Han Xizai Evening Banquet”, Encouraging viewers explore this area by themselves is thought when designed interactive way. Viewers will get the knowledge about these details.

Fig. 3.
figure 3

How to create “2.5D” vision: achromatic shadow layer overlaid with original painting.

Full size image

The traditional digital display of ancient Chinese art tends to focus on the “seeing” part, while we believe that “participating” can immerse viewers in the painting better. Such as “Han Xizai Evening Banquet”, it encourages viewers to explore these areas themselves while interacting. Viewers will gain knowledge about these details instead of just lightening the scenes. As the interactive area lights up, some other relevant information will be shown, including commentary voice, dance animations and conversion to other scenes.

3.2 Encouraging Exploration

Viewers should use the handles to activate these areas, as the handle items are a hint of the pointing device and the exploration activity. At the digital Han Xizai Evening Banquet, the handles were designed to be an electronic candle for the banquet theme. The viewers could use this electronic candle to illuminate the scene of this painting and explore freely. The intensity and angle of light will follow the position of viewers. This interactive approach immerses viewers as if they are participating in this banquet.

More than just lighting up the scene. When the interactive area is lightened, other related works will be displayed. Includes comment voice, dance animations or transitions to other scenes.

3.3 Interactive Narrative

In this painting, different time and space features will appear at the same time. Han Xizai and other characters in this scroll painting appear many times. It is an interesting way to analyze the story line through these characters with different postures and clothes in each scene. As a design concept, when viewers trigger specific interactive areas like some characters or furniture, the scene will be converted into other relevant scenes, and the scene that the viewer will see depends on themselves which is decided by their choice rather than in sequence. Multiple choices make various view sequence, providing viewers a different chance to understand. In order to make this narrative easier for the viewers, we divided it into five parts, representing five scenarios: banquets, watching dance, banquet break, flute blowing, and farewell to guests.

Fig. 4.
figure 4

Rebuild scene space, set 2D character in 3D space that we built. Look at this scene from above.

Full size image

3.4 Reappearance of Ancient Scene

To enhance immersion, we add a shadow layer on the original painting to create a sense of space. However, in order to preserve the plane style and scatter perspective of Chinese painting, we redesign the graphics and try to balance the 3D and 2D. Called “2.5D” vision. “2.5D” vision consists of 2 layers: shadow layer and original painting layer, achromatic shadow overlaid with original painting (see Fig. 3).

Original painting layer is reconstructed in 3D max instead of using original painting photo directly, so there is a perspective change and enhance a sense of space when the scene is converting. However, “scattering perspective” in scroll painting contradicts “focus perspective”, so things are distorted and weird in the rebuilt space. It will look like the original image only when the camera is set at a specific position.

When setting up 2D characters in the 3D space we built (see Fig. 4), we can move the camera in a limited area and render the sequence images. It seems original painting possess subtle perspective change. These images are played during scenes’ conversion.

4 Interaction with Candles

4.1 Candle and the Spatial Interface

Candle and The Spatial Interface The viewers’ activities within a certain range of space will react to the painting. We use handle prop as the medium between viewers and the painting. Considering the ancient Chinese style, we designed the shape of handle prop as a candle. Viewers use the electronic candle to interact with digital painting.

Fig. 5.
figure 5

Interaction styles

Full size image

4.2 Interaction Styles

Interaction Styles There are mainly four ways of interaction in the spatial interface, (a) Viewers use handle prop to activate some interactive points, then launch the interactive and encourage viewers to experience. (b) Viewers can use handle prop to explore the scene of painting freely. In order to encourage viewers to explore, the painting will be reflected in a moving way, such as the area where viewers stand will illuminate or sound the commentary voice played here. (c) Handle prop also used as the conversion switch, which is used by viewers use it to select a specific area to activate scene converting. (d) Designing special interactive effects in some wonderful scenes can make viewers deeply touched. For instance, there is a scene depicting four female company with Han Xizai resting in “Han Xizai Evening Banquet”. This scene has a privacy atmosphere, therefore the interactive effect is a dim light that squeezes through the door, trying to create a peeping effect for viewers (Fig. 5).

5 Implementation

Our installation consists of a high-precision indoor positioning system; two rear-projections units; an 8 m wide, 3 m high projection wall; two mainframes (for positioning and projection, respectively); six long cables and switches. When the device is arranged, the positioning system and the two hosts are connected through the switch and the network, and finally, the host obtains the candle position information and displays the corresponding picture on the projection wall with a projection.

5.1 Indoor Localization System

We utilize a high-accuracy indoor localization system for perspective-driven interaction. The system consists of four base stations and a tracking tag, which is embedded into a candle-like object (Fig. 6). With the aim of Ultra-Wide Bandwidth Microwave (UWB) Time Difference of Arrival (TDoA) Localization technology, we can acquire a localization accuracy of the tag around 10 cm. Computer calls display images according to the localization. Therefore, the location of viewers could be followed with this system and reflect in a display.

Fig. 6.
figure 6

A high-accuracy indoor localization system

Full size image

5.2 Light and Shadow Effect

In the actual implementation, in order to make the user get a better experience, and in order to make the interaction effect closer to the narrative, we have designed different light and shadow effects for the five scenes. As shown in the figure, in the first scene, the audience uses the candle to light the candlestick hidden in the picture, which finally makes the picture brighter and brighter and shows the effect of candlelight flickering. The candlelight of the audience in the second act will illuminate the face of each character on the screen to focus the attention of the audience. The third act, due to its special privacy atmosphere, is designed to look inward through the door slits, thus causing the bright part to appear as a sliver and moving in the opposite direction to the candle. The fourth act shows a grand performance, so the light effect is designed as a stage lighting that illuminates in a global context. The fifth act renders the parting atmosphere, so the shadow of the character is added to the effect of the illuminating effect. When the candlelight is far away from the character, the shadow will gradually lengthen, which further reflects the gradual drift away from the farewell.

Fig. 7.
figure 7

Different effect in each scene

Full size image

6 Observation

Our installations were successfully exhibited in Beijing, Shanghai, Nanjing, Shenzhen and other places and received feedback from lots of users. More than a thousand viewers experienced our interactive narrative system, and most of them thought that this interactive design had greatly enhanced their interest in the Han Xizai Evening Banquet. The viewers described their experiences in a smooth, natural, immersive, and interesting way (Fig. 7).

About 60\(\%\) of the viewers did not understand the story behind this painting. However, they showed further understanding and interest in the painting after this experience. Some viewers with Chinese art expertise said that the interaction created a good atmosphere and conveyed accurately. For example in the third act, the lighting simulation is taken from the crack at the door to Han Xizai’s house, which restores the perspective of the artist’s observation and recording of this evening banquet. 90\(\%\) of the viewers were interested in the positioning technology because of the smooth interaction. They also cared about other application scenarios of the technology (Fig. 8).

Fig. 8.
figure 8

A viewer experiences on site

Full size image

Some viewers give us many useful suggestions:

  • Add LED lighting effects off/on the candle to make it more realistic about the environment.

  • Lowering the screen so that the characters of the painting are as high as the viewers.

  • Narrowing the scope of interaction so that they moved within the space.

  • In terms of exhibitions, some curators hope that the exhibition environment will be more subtle and classical, such as adding some traditional ornaments to decorate.

There are still some disputes that need us to discuss in the future:

  • Whether characters can be increased more movements, such as interact with other characters in the painting or enlarge the local details.

  • Some viewers think that the different ways of interacting each scene are very attractive, eliminating fatigue, while others think that changes in interaction make them confused.

7 Discussion

For visual effects and device performance, one person handling interactive props can achieve the best effects in the current design. However, there are many viewers in a large hall, and only one interactive project is inefficient, therefore, multiple handles and multiple interaction methods should be considered.

This design encourages viewers to learn about Han Xizai Evening Banquet through multiple-choose narration to emphasize interesting interactions. While some art historians have the opposite opinion of this design style. Many traditional Chinese paintings have complex controversies in related academic circles. Some historians believe that show academic and objective knowledge accurately is more important than interesting popular education. Serious cultural education contradicts entertainment, which is more important. This is a discussion that worth exploring.

8 Conclusion

Through design “Digital Han Xizai Evening Banquet”, we hope our research method is suitable to other digital scroll paintings, with rich visual effects and interactive ways to present more brilliant characteristic and attract more public attention to Chinese scroll paintings.