Abstract
We present a facial animation system for ordinary single-cameral videos based on 2D shape regression. Unlike some prior facial animation techniques, our system doesn’t need complex equipment. The system consists of firstly a Cascade Multi-Channel Convolutional Neural Network (CMC-CNN) model to accurately detect facial landmarks from 2D video frames. Based on these detected 2D points, the facial motion parameters, including the head pose and facial expressions, are recovered. Then the system animates a bone-driven 3D avatar with the facial motion parameters. Experiments show that our system can accurately detect facial landmarks and the animation results are visually plausible and similar to the user’s facial motion.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Bradley, D., Heidrich, W., Popa, T., Sheffer, A.: High resolution passive facial performance capture. ACM Trans Graph (TOG) 29 (2010). ACM. Article No. 41
Cao, C., Weng, Y., Lin, S., Zhou, K.: 3d shape regression for real-time facial animation. ACM Trans. Graph. (TOG) 32(4), 41 (2013)
Cao, X., Wei, Y., Wen, F., Sun, J.: Face alignment by explicit shape regression. Int. J. Comput. Vis. 107(2), 177–190 (2014)
Chrysos, G., Antonakos, E., Zafeiriou, S., Snape, P.: Offline deformable face tracking in arbitrary videos. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 1–9 (2015)
Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 6, 681–685 (2001)
Cootes, T.F., Taylor, C.J., Cooper, D.H., Graham, J.: Active shape models-their training and application. Comput. Vision Image Underst. 61(1), 38–59 (1995)
Dollár, P., Welinder, P., Perona, P.: Cascaded pose regression. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1078–1085. IEEE (2010)
Hou, Q., Wang, J., Cheng, L., Gong, Y.: Facial landmark detection via cascade multi-channel convolutional neural network. In: 2015 IEEE International Conference on Image Processing (ICIP), pp. 1800–1804. IEEE (2015)
Huang, H., Chai, J., Tong, X., Wu, H.T.: Leveraging motion capture and 3D scanning for high-fidelity facial performance acquisition. ACM Trans. Graphics (TOG) 30 (2011). ACM. Article No. 74
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T.: Caffe: convolutional architecture for fast feature embedding. In: Proceedings of the ACM International Conference on Multimedia, pp. 675–678. ACM (2014)
Lewis, J., Anjyo, K.: Direct manipulation blendshapes. IEEE Comput. Graph. Appl. 4, 42–50 (2010)
McLaughlin, T., Cutler, L., Coleman, D.: Character rigging, deformations, and simulations in film and game production. In: ACM SIGGRApPH 2011 Courses, p. 5. ACM (2011)
McLaughlin, T., Sumida, S.S.: The morphology of digital creatures. In: SIGGRAPH Courses, p. 1 (2007)
Orvalho, V., Bastos, P., Parke, F., Oliveira, B., Alvarez, X.: A facial rigging survey. In: Proceedings of the 33rd Annual Conference of the European Association for Computer Graphics-Eurographics, pp. 10–32 (2012)
Ren, S., Cao, X., Wei, Y., Sun, J.: Face alignment at 3000 fps via regressing local binary features. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1685–1692 (2014)
Sagonas, C., Tzimiropoulos, G., Zafeiriou, S., Pantic, M.: 300 faces in-the-wild challenge: the first facial landmark localization challenge. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 397–403 (2013)
Seo, J., Irving, G., Lewis, J., Noh, J.: Compression and direct manipulation of complex blendshape models. ACM Trans. Graph. (TOG) 30 (2011). ACM. Article No. 164
Shen, J., Zafeiriou, S., Chrysos, G.G., Kossaifi, J., Tzimiropoulos, G., Pantic, M.: The first facial landmark tracking in-the-wild challenge: benchmark and results. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 50–58 (2015)
Weise, T., Bouaziz, S., Li, H., Pauly, M.: Realtime performance-based facial animation. ACM Trans. Graph. (TOG) 30 (2011). ACM. Article No. 77
Weise, T., Li, H., Van Gool, L., Pauly, M.: Face/off: live facial puppetry. In: Proceedings of the 2009 ACM SIGGRAPH/Eurographics Symposium on Computer animation, pp. 7–16. ACM (2009)
Acknowledgement
This work is partially supported by the National Science Foundation of China under Grant No. 61473219, and the National High Technology Research and Development Program of China (863 Program) under Grant No. 2014AA015205.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Bai, R., Hou, Q., Wang, J., Gong, Y. (2016). Facial Animation Based on 2D Shape Regression. In: Chen, E., Gong, Y., Tie, Y. (eds) Advances in Multimedia Information Processing - PCM 2016. PCM 2016. Lecture Notes in Computer Science(), vol 9917. Springer, Cham. https://doi.org/10.1007/978-3-319-48896-7_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-48896-7_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-48895-0
Online ISBN: 978-3-319-48896-7
eBook Packages: Computer ScienceComputer Science (R0)