Abstract
To synthesize real-time and realistic facial animation, we present an effective algorithm which combines image- and geometry-based methods for facial animation simulation. Considering the numerous motion units in the expression coding system, we present a novel simplified motion unit based on the basic facial expression, and construct the corresponding basic action for a head model. As image features are difficult to obtain using the performance driven method, we develop an automatic image feature recognition method based on statistical learning, and an expression image semi-automatic labeling method with rotation invariant face detection, which can improve the accuracy and efficiency of expression feature identification and training. After facial animation redirection, each basic action weight needs to be computed and mapped automatically. We apply the blend shape method to construct and train the corresponding expression database according to each basic action, and adopt the least squares method to compute the corresponding control parameters for facial animation. Moreover, there is a pre-integration of diffuse light distribution and specular light distribution based on the physical method, to improve the plausibility and efficiency of facial rendering. Our work provides a simplification of the facial motion unit, an optimization of the statistical training process and recognition process for facial animation, solves the expression parameters, and simulates the subsurface scattering effect in real time. Experimental results indicate that our method is effective and efficient, and suitable for computer animation and interactive applications.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Blanz, V., Basso, C., Poggio, T., Vetter, T., 2003. Reanimating faces in images and video. Comput. Graph. For., 22(3): 641–650. [doi:10.1111/1467-8659.t01-1-00712]
Blinn, J.F., 1977. Models of light reflection for computer synthesized pictures. ACM SIGGRAPH Comput. Graph., 11(2):192–198. [doi:10.1145/965141.563893]
Buck, I., Finkelstein, A., Jacobs, C., Klein, A., Salesin, D.H., Seims, J., Szeliski, R., Toyama, K., 2006. Performance-Driven Hand-Drawn Animation. Proc. 1st Int. Symp. on Non-photorealistic Animation and Rendering, p.101–108. [doi:10.1145/340916.340929]
Chai, X.J., Shan, S.G., Gao, W., Chen, X.L., 2005. Examplebased learning for automatic face alignment. J. Softw., 16(5):718–726 (in Chinese).
Cootes, T.F., Taylor, C.J., Cooper, D.H., Graham, J., 1995. Active shape models—their training and application. Comput. Vis. Image Understand., 61(1):38–59. [doi:10.1006/cviu.1995.1004]
Cootes, T.F., Edwards, G.J., Taylor, C.J., 2001. Active Appearance Models. IEEE Trans. Pattern Anal. Mach. Intell., 23(6):681–685. [doi:10.1109/34.927467]
Deng, Z.G., Chiang, P.Y., Fox, P., Neumann, U., 2006. Animating Blend Shape Faces by Cross-Mapping Motion Capture Data. Proc. Symp. on Interactive 3D Graphics and Games, p.43–48. [doi:10.1145/1111411.1111419]
d’Eon, E., Luebke, D., Enderton, E., 2007. Efficient Rendering of Human Skin. Proc. 18th Eurographics Conf. on Rendering Techniques, p.147–157. [doi:10.2312/EGWR/EGSR07/147-157]
Ekman, P., Friesen, W.V., Hager, J.C., 2002. Facial Action Coding System: the Manual. A Human Face. Research Nexus, Salt Lake City, p.1–197.
Guenter, B., Grimm, C., Wood, D., Malvar, H., Pighin, F., 2006. Making Faces. ACM SIGGRAPH, p.55–66. [doi:10.1145/1185657.1185858]
Jimenez, J., Sundstedt, V., Gutierrez, D., 2009. Screen-space perceptual rendering of human skin. ACM Trans. Appl. Percept., 6(4), Article 23. [doi:10.1145/1609967.1609970]
Lance, W., 1990. Performance-Driven Facial Animation. ACM SIGGRAPH, p.235–242. [doi:10.1145/97879.97906]
Lee, Y.C., Terzopoulos, D., Waters, K., 1995. Realistic Modeling for Facial Animation. Proc. 22nd Annual Conf. on Computer Graphics and Interactive Techniques, p.55–62. [doi:10.1145/218380.218407]
Little, A.C., Hancock, P.J.B., DeBruine, L.M., Jones, B.C., 2012. Adaptation to antifaces and the perception of correct famous identity in an average face. Front. Psychol., 3:19. [doi:10.3389/fpsyg.2012.00019]
Matthews, I., Baker, S., 2004. Active appearance models revisited. Int. J. Comput. Vis., 60(2):135–164. [doi:10.1023/B:VISI.0000029666.37597.d3]
Parke, F.I., 1972. Computer Generated Animation of Faces. Proc. ACM Annual Conf., p.451–457. [doi:10.1145/800 193.569955]
Penner, E., Borshukov, G., 2011. Pre-integrated Skin Shading. In: Engel, W. (Ed.), GPU Pro 2: Advanced Rendering Techniques. A K Peters/CRC Press, p.41–54. [doi:10.1201/b11325-1]
Pighin, F., Hecker, J., Lischinski, D., Szeliski, R., Salesin, D.H., 2005. Synthesizing Realistic Facial Expressions from Photographs. ACM SIGGRAPH, p.75–84. [doi:10.1145/1198555.1198589]
Viola, P., Jones, M., 2001. Rapid Object Detection Using a Boosted Cascade of Simple Features. Proc. IEEE Conf. on Computer Vision and Pattern Recognition, p.511–518. [doi:10.1109/cvpr.2001.990517]
Weise, T., Bouaziz, S., Li, H., Pauly, M., 2011. Realtime performance-based facial animation. ACM Trans. Graph., 30(4), Article 77, p.1–10. [doi:10.1145/2010324.1964972]
Yao, J.F., Chen, Q., 2008. Survey on computer facial expression animation technology. Appl. Res. Comput., 25(11): 3233–3237. [doi:10.3969/j.issn.1001-3695.2008.11.009]
Zhang, Q.S., Chen, G.L., 2003. Realistic 3D human facial animation. J. Softw., 14(3):643–650 (in Chinese).
Zhang, Q.S., Liu, Z.C., Guo, B.N., Demetri, T., Shum, H.Y., 2006. Geometry-driven photorealistic facial expression synthesis. IEEE Trans. Visual. Comput. Graph., 12(1):48–60. [doi:10.1109/TVCG.2006.9]
Author information
Authors and Affiliations
Corresponding author
Additional information
Project supported by the 2013 Annual Beijing Technological and Cultural Fusion for Demonstrated Base Construction and Industrial Nurture (No. Z131100000113007), and the National Natural Science Foundation of China (Nos. 61202324, 61271431, and 61271430)
Rights and permissions
About this article
Cite this article
Xu, S., Ma, G., Meng, W. et al. Statistical learning based facial animation. J. Zhejiang Univ. - Sci. C 14, 542–550 (2013). https://doi.org/10.1631/jzus.CIDE1307
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1631/jzus.CIDE1307