research-article

Compression and direct manipulation of complex blendshape models

Authors:

Geoffrey Irving,

Junyong NohAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 30, Issue 6

Pages 1 - 10

https://doi.org/10.1145/2070781.2024198

Published: 12 December 2011 Publication History

Abstract

We present a method to compress complex blendshape models and thereby enable interactive, hardware-accelerated animation of these models. Facial blendshape models in production are typically large in terms of both the resolution of the model and the number of target shapes. They are represented by a single huge blendshape matrix, whose size presents a storage burden and prevents real-time processing. To address this problem, we present a new matrix compression scheme based on a hierarchically semi-separable (HSS) representation with matrix block reordering. The compressed data are also suitable for parallel processing. An efficient GPU implementation provides very fast feedback of the resulting animation. Compared with the original data, our technique leads to a huge improvement in both storage and processing efficiency without incurring any visual artifacts. As an application, we introduce an extended version of the direct manipulation method to control a large number of facial blendshapes efficiently and intuitively.

Supplementary Material

JPG File (a164-seo.jpg)

Download
11.42 KB

MP4 File (a164-seo.mp4)

Download
66.36 MB

References

[1]

Autodesk, 2011. Autodesk Maya API White Paper.

[2]

Bainville, E., 2010. OpenCL Training Course: GPU matrix-vector product. http://www.bealto.com/.

[3]

Bergeron, P., and Lachapelle, P. 1985. Controlling facial expressions and body movements in the computer generated animated short 'Tony de Peltrie'. In SIGGRAPH 85 Tutorial Notes, Advanced Computer Animation Course.

[4]

Blanz, V., and Vetter, T. 1999. A morphable model for the synthesis of 3d faces. In Proc. of SIGGRAPH 99, ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, 187--194.

Digital Library

[5]

Börm, S., Grasedyck, L., and Hackbusch, W. 2003. Introduction to hierarchical matrices with applications. Engineering Analysis with Boundary Elements 27, 5, 405--422.

[6]

Börm, S. 2010. Efficient numerical methods for non-local operators: H2-matrix compression, algorithms and analysis. EMS Tracts in Math. European Mathematical Society.

[7]

Carr, J. C., Beatson, R. K., Cherrie, J. B., Mitchell, T. J., Fright, W. R., McCallum, B. C., and Evans, T. R. 2001. Reconstruction and representation of 3d objects with radial basis functions. In Proc. of SIGGRAPH 2001, ACM, Computer Graphics Proceedings, Annual Conference Series, 67--76.

Digital Library

[8]

Çivril, A., and Magdon-Ismail, M. 2009. On selecting a maximum volume sub-matrix of a matrix and related problems. Theor. Comput. Sci. 410 (November), 4801--4811.

Digital Library

[9]

Chandrasekaran, S., Gu, M., and Pals, T. 2004. Fast and stable algorithms for hierarchically semi-separable representations. Tech. rep., University of California, Berkeley.

[10]

Chuang, E. S. 2004. Analysis, synthesis, and retargeting of facial expressions. PhD thesis, Stanford, CA, USA. AAI3128633.

Digital Library

[11]

Deng, Z., Chiang, P.-Y., Fox, P., and Neumann, U. 2006. Animating blendshape faces by cross-mapping motion capture data. In Proc. of the 2006 Symp. on Interactive 3D graphics and games, ACM, New York, NY, USA, I3D '06, 43--48.

Digital Library

[12]

Ernst, J., 2011. Fast and efficient facial rigging. Talk at Game Developers Conference (GDC) 2011.

[13]

Feng, W.-W., Kim, B.-U., and Yu, Y. 2008. Real-time data driven deformation using kernel canonical correlation analysis. ACM Trans. Graph. 27 (August), 91:1--91:9.

Digital Library

[14]

Irving, G. 2011. Banded householder representation of linear subspaces. http://arxiv.org/abs/1108.5822.

[15]

Johnson, S. G., 2010. The NLopt nonlinear-optimization package. http://ab-initio.mit.edu/nlopt.

[16]

Joshi, P., Tien, W. C., Desbrun, M., and Pighin, F. 2003. Learning controls for blend shape based realistic facial animation. In Proc. of the 2003 ACM SIGGRAPH/Eurographics Symp. on Comp. Anim., Eurographics Association, SCA '03, 187--192.

Digital Library

[17]

Joshi, P., Meyer, M., DeRose, T., Green, B., and Sanocki, T. 2007. Harmonic coordinates for character articulation. ACM Trans. Graph. 26 (July).

Digital Library

[18]

Ju, T., Schaefer, S., and Warren, J. 2005. Mean value coordinates for closed triangular meshes. ACM Trans. Graph. 24 (July), 561--566.

Digital Library

[19]

Karni, Z., and Gotsman, C. 2000. Spectral compression of mesh geometry. In Proc. of SIGGRAPH 2000, ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, 279--286.

Digital Library

[20]

Kernighan, B. W., and Lin, S. 1970. An Efficient Heuristic Procedure for Partitioning Graphs. The Bell System Tech. J. 49, 1, 291--307.

[21]

Lau, M., Chai, J., Xu, Y.-Q., and Shum, H.-Y. 2009. Face poser: Interactive modeling of 3d facial expressions using facial priors. ACM Trans. Graph. 29 (December), 3:1--3:17.

Digital Library

[22]

Lewis, J., and Anjyo, K.-i. 2010. Direct manipulation blend-shapes. IEEE Comp. Graph. and Appl. 30, 4 (July-Aug.), 42--50.

Digital Library

[23]

Lewis, J. P., Mooser, J., Deng, Z., and Neumann, U. 2005. Reducing blendshape interference by selected motion attenuation. In Proc. of the 2005 Symp. on Interactive 3D graphics and games, ACM, New York, NY, USA, I3D '05, 25--29.

Digital Library

[24]

Li, H., Weise, T., and Pauly, M. 2010. Example-based facial rigging. ACM Trans. Graph. 29 (July), 32:1--32:6.

Digital Library

[25]

Lipman, Y., Kopf, J., Cohen-Or, D., and Levin, D. 2007. Gpu-assisted positive mean value coordinates for mesh deformations. In Proc. of the fifth Eurographics Symp. on Geom. Proc.

Digital Library

[26]

Loiola, E. M., Maria, N., Abreu, M., Boaventuranetto, P. O., Hahn, P., and Querido, T. 2007. An analytical survey for the quadratic assignment problem. European Journal Operational Research 176, 657--690.

[27]

Nguyen, H. 2007. GPU Gems 3. Addison-Wesley Professional.

Digital Library

[28]

NVIDIA. 2010. CUDA Compute Unified Device Architecture - Programming Guide.

[29]

Osipa, J. 2010. Stop Staring: Facial Modeling and Animation Done Right, 3rd Ed. Sybex.

Digital Library

[30]

Peng, J., Kim, C.-S., and Kuo, C.-C. J. 2005. Technologies for 3d mesh compression: A survey. Journal of Visual Communication and Image Representation 16, 6, 688--733.

Digital Library

[31]

Pighin, F., Hecker, J., Lischinski, D., Szeliski, R., and Salesin, D. H. 1998. Synthesizing realistic facial expressions from photographs. In Proc. of SIGGRAPH 98, ACM, New York, NY, USA, 75--84.

Digital Library

[32]

Powell, M. J. D. 2009. The BOBYQA algorithm for bound constrained optimization without derivatives. Tech. rep., Cambridge, England.

[33]

Raitt, B., 2004. The making of gollum. Presentation at U. Southern California Institute for Creative Technologies' Frontiers of Facial Animation Workshop, August.

[34]

Seol, Y., Seo, J., Kim, P. H., Lewis, J. P., and Noh, J. 2011. Artist friendly facial animation retargeting. ACM Trans. Graph. (Proceedings of SIGGRAPH ASIA 2011) 30, 6.

[35]

Sifakis, E., Neverov, I., and Fedkiw, R. 2005. Automatic determination of facial muscle activations from sparse motion capture marker data. ACM Trans. Graph. 24 (July), 417--425.

Digital Library

[36]

Sorkine, O., Cohen-Or, D., Irony, D., and Toledo, S. 2005. Geometry-aware bases for shape approximation. IEEE Trans. on Viz. and Comp. Graph. 11, 2 (March-April), 171--180.

Digital Library

[37]

Stefanoski, N., and Ostermann, J. 2008. Spatially and temporally scalable compression of animated 3d meshes with MPEG-4/FAMC. In ICIP '08 - IEEE International Conference on Image Processing, vol. 0.

[38]

Sumner, R. W., Zwicker, M., Gotsman, C., and Popović, J. 2005. Mesh-based inverse kinematics. ACM Trans. Graph. 24 (July), 488--495.

Digital Library

[39]

Vlasic, D., Brand, M., Pfister, H., and Popović, J. 2005. Face transfer with multilinear models. ACM Trans. Graph. 24 (July), 426--433.

Digital Library

[40]

Wu, Q., Xia, T., Chen, C., Lin, H.-Y. S., Wang, H., and Yu, Y. 2008. Hierarchical tensor approximation of multidimensional visual data. IEEE Trans. on Vis. and Comp. Graph. 14 (January), 186--199.

Digital Library

[41]

Xia, J., Chandrasekaran, S., Gu, M., and Li, X. S. 2010. Fast algorithms for hierarchically semiseparable matrices. Numerical Linear Algebra with Applications 17, 6, 953--976.

[42]

Zhang, L., Snavely, N., Curless, B., and Seitz, S. M. 2004. Spacetime faces: high resolution capture for modeling and animation. ACM Trans. Graph. 23 (August), 548--558.

Digital Library

Cited By

Kavan LDoublestein JPrazak MCioffi MRoble D(2024)Compressed Skinning for Facial BlendshapesACM SIGGRAPH 2024 Conference Papers10.1145/3641519.3657477(1-9)Online publication date: 13-Jul-2024
https://dl.acm.org/doi/10.1145/3641519.3657477
Ma WGhifary MLewis JChoi BEom H(2022)FDLS: A Deep Learning Approach to Production Quality, Controllable, and Retargetable Facial Performances.Proceedings of the 2022 Digital Production Symposium10.1145/3543664.3543672(1-9)Online publication date: 7-Aug-2022
https://dl.acm.org/doi/10.1145/3543664.3543672
Han JKim JSuh JKim H(2022)Customizing blendshapes to capture facial detailsThe Journal of Supercomputing10.1007/s11227-022-04885-779:6(6347-6372)Online publication date: 3-Nov-2022
https://dl.acm.org/doi/10.1007/s11227-022-04885-7
Show More Cited By

Index Terms

Compression and direct manipulation of complex blendshape models
1. Computing methodologies
  1. Computer graphics
    1. Animation

Recommendations

Direct Manipulation Blendshapes

This paper introduces a simple direct manipulation algorithm for the popular blendshape facial animation approach. As is the case for body animation, direct manipulation of blendshape models is an inverse problem: when a single vertex is moved, the ...
Compression and direct manipulation of complex blendshape models
SA '11: Proceedings of the 2011 SIGGRAPH Asia Conference

We present a method to compress complex blendshape models and thereby enable interactive, hardware-accelerated animation of these models. Facial blendshape models in production are typically large in terms of both the resolution of the model and the ...
Direct manipulation of blendshapes using a sketch-based interface
Web3D '18: Proceedings of the 23rd International ACM Conference on 3D Web Technology

We introduce a method that localizes the direct manipulation of blendshape models for facial animation with a customized sketch-based interface. Direct manipulation methods address the problem of cumbersome weight editing process using the traditional ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 30, Issue 6

December 2011

678 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/2070781

Issue’s Table of Contents

Copyright © 2011 ACM.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 December 2011

Published in TOG Volume 30, Issue 6

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

KOCCA/MCST

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

31
Total Citations
View Citations
946
Total Downloads

Downloads (Last 12 months)25
Downloads (Last 6 weeks)4

Reflects downloads up to 08 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Kavan LDoublestein JPrazak MCioffi MRoble D(2024)Compressed Skinning for Facial BlendshapesACM SIGGRAPH 2024 Conference Papers10.1145/3641519.3657477(1-9)Online publication date: 13-Jul-2024
https://dl.acm.org/doi/10.1145/3641519.3657477
Ma WGhifary MLewis JChoi BEom H(2022)FDLS: A Deep Learning Approach to Production Quality, Controllable, and Retargetable Facial Performances.Proceedings of the 2022 Digital Production Symposium10.1145/3543664.3543672(1-9)Online publication date: 7-Aug-2022
https://dl.acm.org/doi/10.1145/3543664.3543672
Han JKim JSuh JKim H(2022)Customizing blendshapes to capture facial detailsThe Journal of Supercomputing10.1007/s11227-022-04885-779:6(6347-6372)Online publication date: 3-Nov-2022
https://dl.acm.org/doi/10.1007/s11227-022-04885-7
Fu HBian SChaudhry EWang SYou LZhang J(2021)PDE Surface-Represented Facial BlendshapesMathematics10.3390/math92229059:22(2905)Online publication date: 15-Nov-2021
https://doi.org/10.3390/math9222905
Bailey SOmens DDilorenzo PO'Brien J(2020)Fast and deep facial deformationsACM Transactions on Graphics10.1145/3386569.339239739:4(94:1-94:15)Online publication date: 12-Aug-2020
https://dl.acm.org/doi/10.1145/3386569.3392397
Carrigan EZell EGuiard CMcDonnell R(2020)Expression Packing: As‐Few‐As‐Possible Training Expressions for Blendshape TransferComputer Graphics Forum10.1111/cgf.1392539:2(219-233)Online publication date: 13-Jul-2020
https://doi.org/10.1111/cgf.13925
Park JKim JSong JPark SYoon S(2019)Simultaneous Simplification of Multiple Triangle Meshes for Blend ShapeJournal of the Korea Computer Graphics Society10.15701/kcgs.2019.25.3.7525:3(75-83)Online publication date: 1-Jul-2019
https://doi.org/10.15701/kcgs.2019.25.3.75
Abdrashitov RJacobson ASingh K(2019)A System for Efficient 3D Printed Stop-motion Face AnimationACM Transactions on Graphics10.1145/336051039:1(1-11)Online publication date: 18-Oct-2019
https://dl.acm.org/doi/10.1145/3360510
Le BLewis J(2019)Direct delta mush skinning and variantsACM Transactions on Graphics10.1145/3306346.332298238:4(1-13)Online publication date: 12-Jul-2019
https://dl.acm.org/doi/10.1145/3306346.3322982
Reed KCosker D(2019)User‐Guided Facial Animation through an Evolutionary InterfaceComputer Graphics Forum10.1111/cgf.1361238:6(165-176)Online publication date: 19-Mar-2019
https://doi.org/10.1111/cgf.13612
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents