Skip to main content
SpringerLink
Log in

3D cartoon face rigging from sparse examples

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

We present a data-driven method for automatically constructing cartoonized 3D blendshapes of a subject’s face. Given a pre-defined blendshape template of the real facial expressions and corresponding cartoonized blendshape template created by an artist, we represent the blendshapes of an identity in the real and cartoon face spaces with the deformations of the blendshape template in each space and learn a mapping between the deformations in the two spaces. To this end, our method decomposes the deformations in each space into two parts: an identity-independent part that is represented with the deformation gradient of the blendshape template, and an identity-dependent part that is modeled by a low-rank linear model. We regress the linear model for the real expressions from a 3D facial expression dataset. An algorithm is then introduced to regress the mapping between the linear models in the two spaces from a small set of real expressions and their cartoonized counterparts. At run time, given the blendshapes of a subject’s real face and her 3D cartoon neutral face, our method automatically constructs the cartoonized blendshapes of the subject with the help of the cartoonized blendshape template and the learned mapping. Our method is user-independent and only requires a small set of 3D cartoonized expressions modeled by the artist for cartoon face rigging. We evaluate our method by creating cartoonized 3D facial animations for variant identities in two different artistic styles. The rigging results demonstrate that our method successfully preserves both artistic styles and personalized expressions of different identities.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Akleman, E.: Making caricatures with morphing. In: ACM SIGGRAPH 97 Visual Proceedings: The Art and Interdisciplinary Programs of SIGGRAPH’97, p. 145. ACM (1997)

  2. Akleman, E., Reisch, J.: Modeling expressive 3D caricatures. In: ACM SIGGRAPH 2004 Sketches, p. 61. ACM (2004)

  3. Alexander, O., Rogers, M., Lambeth, W., Chiang, M., Debevec, P.: The digital emily project: photoreal facial modeling and animation. In: Acm Siggraph 2009 Courses, p. 12. ACM (2009)

  4. Blanz, V., Vetter, T.: A morphable model for the synthesis of 3D faces. In: Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, pp. 187–194. ACM Press/Addison-Wesley Publishing Co. (1999)

  5. Bouaziz, S., Wang, Y., Pauly, M.: Online modeling for realtime facial animation. ACM Trans. Graph. (TOG) 32(4), 40 (2013)

    Article  MATH  Google Scholar 

  6. Cao, C., Bradley, D., Zhou, K., Beeler, T.: Real-time high-fidelity facial performance capture. ACM Trans. Graph. (ToG) 34(4), 46 (2015)

    Article  Google Scholar 

  7. Cao, C., Hou, Q., Zhou, K.: Displaced dynamic expression regression for real-time facial tracking and animation. ACM Trans. Graph. (TOG) 33(4), 43 (2014)

    MATH  Google Scholar 

  8. Cao, C., Weng, Y., Lin, S., Zhou, K.: 3D shape regression for real-time facial animation. ACM Trans. Graph. (TOG) 32(4), 41 (2013)

    Article  MATH  Google Scholar 

  9. Cao, C., Weng, Y., Zhou, S., Tong, Y., Zhou, K.: Facewarehouse: a 3D facial expression database for visual computing. IEEE Trans. Vis. Comput. Graph. 20(3), 413–425 (2014)

    Article  Google Scholar 

  10. Casas, D., Feng, A., Alexander, O., Fyffe, G., Debevec, P., Ichikari, R., Li, H., Olszewski, K., Suma, E., Shapiro, A.: Rapid photorealistic blendshape modeling from RGB-D sensors. In: Proceedings of the 29th International Conference on Computer Animation and Social Agents, pp. 121–129 (2016)

  11. Cosker, D., Krumhuber, E., Hilton, A.: A FACS valid 3D dynamic action unit database with applications to 3D dynamic morphable facial modeling. In: IEEE International Conference on Computer Vision (ICCV), 2011, pp. 2296–2303. IEEE (2011)

  12. Fujiwara, T., Koshimizu, H., Fujimura, K., Kihara, H., Noguchi, Y., Ishikawa, N.: 3D modeling system of human face and full 3D facial caricaturing. In: Proceedings of Third International Conference on 3-D Digital Imaging and Modeling, 2001, pp. 385–392. IEEE (2001)

  13. Fujiwara, T., Nishihara, T., Tominaga, M., Kato, K., Murakami, K., Koshirnizu, H.: On the detection of feature points of 3D facial image and its application to 3D facial caricature. In: Proceedings of Second International Conference on 3-D Digital Imaging and Modeling, 1999, pp. 490–496. IEEE (1999)

  14. Garrido, P., Zollhöfer, M., Casas, D., Valgaerts, L., Varanasi, K., Pérez, P., Theobalt, C.: Reconstruction of personalized 3D face rigs from monocular video. ACM Trans. Graph. 35(3), 28:1–28:15 (2016)

    Article  Google Scholar 

  15. Ichim, A.E., Bouaziz, S., Pauly, M.: Dynamic 3D avatar creation from hand-held video input. ACM Trans. Graph. (SIGGRAPH) 34(4), 45:1–45:14 (2015)

    Article  Google Scholar 

  16. Lewis, J.P., Anjyo, K., Rhee, T., Zhang, M., Pighin, F., Deng, Z.: Practice and theory of blendshape facial models. In: EuroGraphics (2014)

  17. Li, H., Weise, T., Pauly, M.: Example-based facial rigging. ACM Trans. Graph. (ToG) 29, 32 (2010)

    Google Scholar 

  18. Li, P., Chen, Y., Liu, J., Fu, G.: 3D caricature generation by manifold learning. In: IEEE International Conference on Multimedia and Expo, 2008, pp. 941–944. IEEE (2008)

  19. Li, T., Bolkart, T., Black, M.J., Li, H., Romero, J.: Learning a model of facial shape and expression from 4D scans. ACM Trans. Graph. 36(6), 194 (2017). https://doi.org/10.1145/3130800.3130813. (Proc. SIGGRAPH Asia)

    Google Scholar 

  20. Liu, J., Chen, Y., Miao, C., Xie, J., Ling, C.X., Gao, X., Gao, W.: Semi-supervised learning in reconstructed manifold space for 3D caricature generation. In: Computer Graphics Forum, vol. 28, pp. 2104–2116. Wiley Online Library (2009)

  21. Sadimon, S.B., Sunar, M.S., Mohamad, D., Haron, H.: Computer generated caricature: a survey. In: International Conference on Cyberworlds (CW), 2010, pp. 383–390. IEEE (2010)

  22. Sumner, R.W., Popović, J.: Deformation transfer for triangle meshes. ACM Trans. Graph. (TOG) 23, 399–405 (2004)

    Article  Google Scholar 

  23. Vlasic, D., Brand, M., Pfister, H., Popović, J.: Face transfer with multilinear models. ACM Trans. Graph. (TOG) 24, 426–433 (2005)

    Article  Google Scholar 

  24. Waltz, R.A., Morales, J.L., Nocedal, J., Orban, D.: An interior algorithm for nonlinear optimization that combines line search and trust region steps. Math. Program. 107(3), 391–408 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  25. Weise, T., Bouaziz, S., Li, H., Pauly, M.: Realtime performance-based facial animation. ACM Trans. Graph. (TOG) 30, 77 (2011)

    Article  Google Scholar 

  26. Xie, J., Chen, Y., Liu, J., Miao, C., Gao, X.: Interactive 3D caricature generation based on double sampling. In: Proceedings of the 17th ACM International Conference on Multimedia, pp. 745–748. ACM (2009)

  27. Zhou, J., Tong, X., Liu, Z., Guo, B.: 3D cartoon face generation by local deformation mapping. Vis. Comput. 32(6–8), 717–727 (2016)

    Article  Google Scholar 

Download references

Acknowledgements

We thank the anonymous reviewers for their constructive comments and suggestions. We also thank the graphics and parallel processing laboratory of Zhejiang University for sharing the FaceWarehouse dataset for our research. All cartoon exemplars used in our system are created by Xing Zhao and Shuitian Yan.

Author information

Authors and Affiliations

  1. Tsinghua University, Microsoft Research Asia, Beijing, China

    Jingyong Zhou

  2. Microsoft Research Asia, Beijing, China

    Hsiang-Tao Wu, Xin Tong & Baining Guo

  3. Microsoft Research, Redmond, USA

    Zicheng Liu

Authors
  1. Jingyong Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hsiang-Tao Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zicheng Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xin Tong
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Baining Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jingyong Zhou.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interests.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (mp4 39774 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, J., Wu, HT., Liu, Z. et al. 3D cartoon face rigging from sparse examples. Vis Comput 34, 1177–1187 (2018). https://doi.org/10.1007/s00371-018-1553-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-018-1553-3

Keywords

Search

Navigation