Technology and Digital Art3D scan-based animation techniques for Chinese opera facial expression documentation
Section snippets
Chinese opera documentation
Chinese opera together with Greek tragic-comedy and Indian Sanskrit opera are the three oldest dramatic art forms in the world [1]. In the long development of Chinese opera, many different styles of opera appeared. Among the well-known styles are Beijing opera (actually the national form, Beijing city), Yule opera (Zhejiang province), Huangmei opera (Anhui province), Kunqu opera (Kunshan opera, popular mainly in Jiangsu province) and Sichuan Opera (Sichuan province). In the most flourishing
Facial make-up and facial expressions in Chinese opera
What appeal to people most in Chinese opera are the different styles of facial make-up, which is one of the major highlights of this art form and requires distinctive techniques of face painting. Exaggerated designs are painted on each performer's face to symbolize a character's personality, role and fate. This technique may have originated from ancient religions and dance.
Audiences who are familiar with opera will immediately know the character's personality, role and fate in the story by
Realistic data capture devices
Sensor devices are increasingly being used to capture realistic data in a number of applications. Motion capture devices can capture motion data at very high sampling rates. Similarly, 3D scanner devices can be used to capture geometric surface data, especially detailed and subtle surface variations in the surface geometry. While 3D scanning or digitizing is a fast process for capturing real life data, it has a number of difficulties associated with it. For example, for complete surface to be
Related work
Our system incorporates two core technical domains—point-based techniques for rendering and animation, and facial expression animation. Below we shall briefly review previous work reported in the literature, which is pertinent to these two areas.
Chinese facial expression documentation system
Our system is based on the use of 3D scanners for facial pose capture in the form of point-based models and the facial animation method adopted is that of simulating the Chinese opera facial expressions by direct morphing of the point-based face models. The main motivation is to reduce the requirement of human intervention in accurate documentation and preserve all realistic aspects captured through the use of 3D scanning technology.
Our system architecture is best represented in the form of a
3D scan of face poses (painted face) for each facial expression
The first task is to create the geometric data for modelling the performer's face, as needed in the animation using a 3D scan-based morphing technique. For this we scan the performer's facial expression in the desired number of key poses. The optimal number of poses has to be worked out in consultation with a suitable expert in the field of Chinese opera. Scanning face poses of different expressions is a complex task and requires patience. We need to scan the face model from different views,
Pre-processing and painting of 3D scan data to yield point-based geometric models
As we have mentioned before, we have to carry out certain pre-processing operations on the scanner output data before we can use them. These operations include cleaning, noise removal, smoothing and merging. We also estimate and associate a surface normal vector with each point and a radius of influence.
First of all, we have to select valid scanning volume and cut off some redundant 3D data such as the visible background and other extraneous objects that were visible to the camera along with
Interactive mark-up of feature points
In any two 3D scans of facial poses, even for poses belonging to a single expression, as it was briefly mentioned earlier, there is no one-to-one correspondence between their sampled points. The same facial region may be represented by different number of points in the two scans. There could even be some part of the face visible in one pose but absent in the second pose of the same expression animation, and hence has no geometric representation. See, for example, the first and the last poses in
Segmenting a point-sampled set into feature regions
Given two point-sampled face poses to be used for 3D morphing, the next major task is to partition the face poses into corresponding feature regions. For that, and before proceeding any further, the following terminology and definitions should be borne in mind:
- 1.
Every point in a face pose is designated either as a feature point or as a non-feature point.
- 2.
A discrete edge is a sequence of sample points from a start point to an end point, such that every point in the sequence lies within the
Building a feature-oriented Q-Splat hierarchy structure
Our hierarchical structure is an level of detail (LOD) structure that is used to optimize rendering speed, as it was introduced in [10]. For every 3D animation, we have to eventually render 2D images on the screen. With an LOD structure, we could get the screen resolution before drawing, and decide when the rendering process should stop at a certain layer. LOD structure accelerates the drawing speed and is widely used in most point-based rendering techniques. We have adopted the Q-Splat
Correspondence for animation purposes
We can easily find the corresponding nodes in the feature region sub-tree level. Since we subdivide the feature region according to the geometry position, we consider the nodes, which lie at the same position of the region sub-tree to correspond.
We have used the term scanner black holes to denote those parts of the facial surface that are absent in one of the poses, as those parts were not visible to the scanner. This is a common occurrence in facial expressions. For example, an expression
Level set driven animation
Once the correspondence between source and target points has been established, there are several methods to drive the animation. Linear interpolation is a simplest way, but it can cause unacceptable distortion such as shrinking, stretching and self-intersection of surface. Fig. 16 shows a rotating triangle, in which we can see the undesirable scaling.
Our solution is to use level set morphing method to drive the facial region animation [28] by tracking the flow of points on a level set in the
Conclusion
In this paper we have described significant technical challenges involved in the development of a system for using 3D scan-based animation techniques for documenting Chinese opera facial expressions. Since exaggerated face painting and expressions are among the most appealing aspects of this art form, our system is architected on the use of 3D scanners for realistic capture of facial expression poses from living human performers. We have prototype implementations of many parts of the system.
Acknowledgements
We gratefully acknowledge NSERC for an equipment grant that enabled the second author to set up the 3D scanning facility, and also Discovery grants for graduate research support. We are also grateful to the anonymous referees for their careful review and valuable suggestions for improvements in the presentation.
References (28)
- 〈http://www.travelchinaguide.com/intro/arts/chinese-opera.htm〉, retrieved on June 27,...
- 〈http://www.unesco.org/bpi/intangible_heritage/china.htm〉, retrieved on June 27,...
- 〈http://www.china.org.cn/english/2002/Sep/42032.htm〉, retrieved on June 27,...
- Thalmann D, Cetre R, Ulicny B, de Heras P, Clavien M. Creating a virtual audience for the heritage of ancient theaters...
- Kim H, Glauert J, de Heras Ciechomski P, Magnenat-Thalmann N, Thalmann D. “Real-time virtual characters for mixed...
- et al.
Procedural rhythmic character animation: an interactive Chinese lion dance
Computer Animation and Virtual Worlds
(2006) - Zhou H, Mudur SP. Application of 3D facial animation techniques for chinese opera documentation. In: Proceedings of the...
- Zhou H, Mudur SP. A system for Chinese Opera Facial Expression. In: Proceedings of Artech 2006, third international...
- Zhou H, Mudur SP. On the use of 3D scanner for chinese opera documentation. In: Proceedings of the 12th international...
- Ezzat T, Geiger G, Poggio T. Trainable videorealistic facial animation. In: Proceedings of ACM SIGGRAPH 2002,...
Cited by (4)
The Motivational Effects and Educational Affordance of Serious Games on the Learning of Cantonese Opera Movements
2024, International Journal of Human-Computer InteractionHCI in performance arts and the case of Illimitable Space System's multimodal interaction and visualization
2015, SA 2015 - SIGGRAPH ASIA 2015 Art PapersA crucial investigation of facial skin colour research trend and direction
2015, International Journal of Multimedia and Ubiquitous EngineeringDesign for three-dimensional facial expression acquiring system
2011, Guangxue Jingmi Gongcheng/Optics and Precision Engineering