skip to main content
research-article
Open access

Dyna: a model of dynamic human shape in motion

Published: 27 July 2015 Publication History

Abstract

To look human, digital full-body avatars need to have soft-tissue deformations like those of real people. We learn a model of soft-tissue deformations from examples using a high-resolution 4D capture system and a method that accurately registers a template mesh to sequences of 3D scans. Using over 40,000 scans of ten subjects, we learn how soft-tissue motion causes mesh triangles to deform relative to a base 3D body model. Our Dyna model uses a low-dimensional linear subspace to approximate soft-tissue deformation and relates the subspace coefficients to the changing pose of the body. Dyna uses a second-order auto-regressive model that predicts soft-tissue deformations based on previous deformations, the velocity and acceleration of the body, and the angular velocities and accelerations of the limbs. Dyna also models how deformations vary with a person's body mass index (BMI), producing different deformations for people with different shapes. Dyna realistically represents the dynamics of soft tissue for previously unseen subjects and motions. We provide tools for animators to modify the deformations and apply them to new stylized characters.

Supplementary Material

ZIP File (a120-pons-moll.zip)
Supplemental files
MP4 File (a120.mp4)

References

[1]
Alexa, M., and Müller, W. 2000. Representing animations by principal components. Comp. Graph. Forum 19, 3, 411--418.
[2]
Allen, B., Curless, B., and Popović, Z. 2002. Articulated body deformation from range scan data. ACM Trans. Graph. 21, 3 (July), 612--619.
[3]
Allen, B., Curless, B., and Popović, Z. 2003. The space of human body shapes: Reconstruction and parameterization from range scans. ACM Transactions on Graphics (TOG) 22, 3, 587--594.
[4]
Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., and Davis, J. 2005. SCAPE: Shape Completion and Animation of PEople. ACM Trans. Graph. 24, 3, 408--416.
[5]
Assassi, L., Becker, M., and Magnenat-Thalmann, N. 2012. Dynamic skin deformation based on biomechanical modeling. In Proc. 25th Conf. Comp. Anim. and Social Agents.
[6]
Aubel, A., and Thalmann, D. 2001. Interactive modeling of the human musculature. In Proc. Comp. Anim., 167--255.
[7]
Bickel, B., Bächer, M., Otaduy, M. A., Matusik, W., Pfister, H., and Gross, M. 2009. Capture and modeling of non-linear heterogeneous soft tissue. ACM Trans. Graph. 28, 3 (July), 89:1--89:9.
[8]
Bogo, F., Romero, J., Loper, M., and Black, M. J. 2014. FAUST: Dataset and evaluation for 3D mesh registration. In CVPR, 3794--3801.
[9]
Capell, S., Green, S., Curless, B., Duchamp, T., and Popović, Z. 2002. Interactive skeleton-driven dynamic deformations. ACM Trans. Graph. 21, 3 (July), 586--593.
[10]
Capell, S., Burkhart, M., Curless, B., Duchamp, T., and Popović, Z. 2007. Physically based rigging for deformable characters. Graph. Models 69, 1 (Jan.), 71--87.
[11]
Chadwick, J. E., Haumann, D. R., and Parent, R. E. 1989. Layered construction for deformable animated characters. SIGGRAPH Comput. Graph. 23, 3 (July), 243--252.
[12]
Chen, Y., Liu, Z., and Zhang, Z. 2013. Tensor-based human body modeling. In CVPR, 105--112.
[13]
de Aguiar, E., and Ukita, N. 2012. Representing mesh-based character animations. Computers & Graphics 38 (Feb.), 10--17.
[14]
de Aguiar, E., Stoll, C., Theobalt, C., Ahmed, N., Seidel, H.-P., and Thrun, S. 2008. Performance capture from sparse multi-view video. ACM Trans. Graph. 27, 3 (Aug.), 98:1--98:10.
[15]
de Aguiar, E., Sigal, L., Treuille, A., and Hodgins, J. K. 2010. Stable spaces for real-time clothing. ACM Trans. Graph. 29, 4 (July), 106:1--106:9.
[16]
Fan, Y., Litven, J., and Pai, D. K. 2014. Active volumetric musculoskeletal systems. ACM Trans. Graph. 33, 4 (July), 152:1--152:9.
[17]
Guan, P., Reiss, L., Hirshberg, D., Weiss, A., and Black, M. J. 2012. Drape: Dressing any person. ACM Trans. Graph. 31, 4 (July), 35:1--35:10.
[18]
Hahn, F., Martin, S., Thomaszewski, B., Sumner, R., Coros, S., and Gross, M. 2012. Rig-space physics. ACM Trans. Graph. 31, 4 (July), 72:1--72:8.
[19]
Hasler, N., Stoll, C., Sunkel, M., Rosenhahn, B., and Seidel, H. 2009. A statistical model of human pose and body shape. Computer Graphics Forum 28, 2, 337--346.
[20]
Hirshberg, D. A., Loper, M., Rachlin, E., and Black, M. J. 2012. Coregistration: Simultaneous alignment and modeling of articulated 3D shape. In ECCV, vol. 7577 of LNCS. Springer, 242--255.
[21]
James, D. L., and Pai, D. K. 2002. Dyrt: Dynamic response textures for real time deformation simulation with graphics hardware. ACM Trans. Graph. 21, 3 (July), 582--585.
[22]
Karni, Z., and Gotsman, C. 2004. Compression of soft-body animation sequences. Computers & Graphics 28, 25--34.
[23]
Kim, T., and James, D. L. 2009. Skipping steps in deformable simulation with online model reduction. ACM Trans. Graph. 28, 5 (Dec.), 123:1--123:9.
[24]
Kim, T., and James, D. L. 2011. Physics-based character skinning using multi-domain subspace deformations. In Proc. ACM SIGGRAPH/Eurographics Symp. Comp. Anim., 63--72.
[25]
Kry, P. G., James, D. L., and Pai, D. K. 2002. EigenSkin: Real time large deformation character skinning in hardware. In Proc. 2002 ACM SIGGRAPH/Eurographics Symp. Comp. Anim., 153--159.
[26]
Larboulette, C., Cani, M.-P., and Arnaldi, B. 2005. Dynamic skinning: Adding real-time dynamic effects to an existing character animation. In Proc. 21st Spring Conf. Comp. Graph., ACM, SCCG '05, 87--93.
[27]
Lee, S.-H., Sifakis, E., and Terzopoulos, D. 2009. Comprehensive biomechanical modeling and simulation of the upper body. ACM Trans. Graph. 28, 4 (Sept.), 99:1--99:17.
[28]
Lewis, J. P., Cordner, M., and Fong, N. 2000. Pose space deformation: A unified approach to shape interpolation and skeleton-driven deformation. In Proc. SIGGRAPH, ACM, 165--172.
[29]
Loper, M. M., Mahmood, N., and Black, M. J. 2014. MoSh: Motion and shape capture from sparse markers. ACM Trans. Graph. 33, 6 (Nov.), 220:1--220:13.
[30]
Maurel, W., Wu, Y., Magnenat-Thalmann, N., and Thalmann, D. 1998. Biomechanical Models for Soft Tissue Simulation. Springer-Verlag, Berlin.
[31]
Metaxas, D., and Terzopoulos, D. 1993. Shape and nonrigid motion estimation through physics-based synthesis. IEEE Trans. Pattern Anal. Mach. Intell. 15, 6 (June), 580--591.
[32]
Neumann, T., Varanasi, K., Hasler, N., Wacker, M., Magnor, M., and Theobalt, C. 2013. Capture and statistical modeling of arm-muscle deformations. Computer Graphics Forum 32, 2 (May), 285--294.
[33]
Neumann, T., Varanasi, K., Wenger, S., Wacker, M., Magnor, M., and Theobalt, C. 2013. Sparse localized deformation components. ACM Trans. Graph. 32, 6 (Nov.), 179:1--179:10.
[34]
Park, S. I., and Hodgins, J. K. 2006. Capturing and animating skin deformation in human motion. ACM Trans. Graph. 25, 3 (July), 881--889.
[35]
Park, S. I., and Hodgins, J. K. 2008. Data-driven modeling of skin and muscle deformation. ACM Trans. Graph. 27, 3 (Aug.), 96:1--96:6.
[36]
Powell, M. 1970. A hybrid method for nonlinear equations. In Numerical Methods for Nonlinear Algebraic Equations, Gordon and Breach Science, London, P. Rabinowitz, Ed., 87--144.
[37]
Pratscher, M., Coleman, P., Laszlo, J., and Singh, K. 2005. Outside-in anatomy based character rigging. In Proc. 2005 ACM SIGGRAPH/Eurographics Symp. Comp. Anim., 329--338.
[38]
Robinette, K., Blackwell, S., Daanen, H., Boehmer, M., Fleming, S., Brill, T., Hoeferlin, D., and Burnsides, D. 2002. Civilian American and European Surface Anthropometry Resource (CAESAR) final report. Tech. Rep. AFRL-HE-WP-TR-2002-0169, US Air Force Research Laboratory.
[39]
Scheepers, F., Parent, R. E., Carlson, W. E., and May, S. F. 1997. Anatomy-based modeling of the human musculature. In Proc. SIGGRAPH, ACM, 163--172.
[40]
Shi, X., Zhou, K., Tong, Y., Desbrun, M., Bao, H., and Guo, B. 2008. Example-based dynamic skinning in real time. ACM Trans. Graph. 27, 3 (Aug.), 29:1--29:8.
[41]
Sifakis, E., Neverov, I., and Fedkiw, R. 2005. Automatic determination of facial muscle activations from sparse motion capture marker data. ACM Trans. Graph. 24, 3 (July), 417--425.
[42]
Stark, J., and Hilton, A. 2007. Surface capture for performance-based animation. IEEE Computer Graphics and Applications 27, 3, 21--31.
[43]
Sumner, R. W., and Popović, J. 2004. Deformation transfer for triangle meshes. ACM Trans. Graph. 23, 3, 399--405.
[44]
Teran, J., Sifakis, E., Blemker, S. S., Ng-Thow-Hing, V., Lau, C., and Fedkiw, R. 2005. Creating and simulating skeletal muscle from the visible human data set. IEEE Trans. Vis. and Comp. Graph. 11, 3 (May), 317--328.
[45]
Terzopoulos, D., and Waters, K. 1990. Physically-based facial modelling, analysis, and animation. J. Vis. and Comp. Anim. 1, 2 (Dec.), 73--80.
[46]
Tsoli, A., Mahmood, N., and Black, M. J. 2014. Breathing life into shape: Capturing, modeling and animating 3D human breathing. ACM Trans. Graph. 33, 4 (July), 52:1--52:11.
[47]
Wilhelms, J., and Van Gelder, A. 1997. Anatomically based modeling. In Proc. SIGGRAPH, ACM, 173--180.

Cited By

View all
  • (2025)Triangle Mesh Reconstruction by Fusion of Residual Graph Convolution and Edge Contraction PoolingThe European Journal on Artificial Intelligence10.1177/30504554241301393Online publication date: 4-Feb-2025
  • (2024)Ultra-dense Motion Capture: An exploratory full-automatic approach for dense tracking of breast motion in 4DPLOS ONE10.1371/journal.pone.029904019:2(e0299040)Online publication date: 26-Feb-2024
  • (2024)Enhancing 3D human pose estimation with NIR single-pixel imaging and time-of-flight technology: a deep learning approachJournal of the Optical Society of America A10.1364/JOSAA.49993341:3(414)Online publication date: 13-Feb-2024
  • Show More Cited By

Index Terms

  1. Dyna: a model of dynamic human shape in motion

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Transactions on Graphics
      ACM Transactions on Graphics  Volume 34, Issue 4
      August 2015
      1307 pages
      ISSN:0730-0301
      EISSN:1557-7368
      DOI:10.1145/2809654
      Issue’s Table of Contents
      Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 27 July 2015
      Published in TOG Volume 34, Issue 4

      Check for updates

      Author Tags

      1. human animation
      2. human shape
      3. motion capture
      4. soft-tissue motion

      Qualifiers

      • Research-article

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)424
      • Downloads (Last 6 weeks)58
      Reflects downloads up to 13 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2025)Triangle Mesh Reconstruction by Fusion of Residual Graph Convolution and Edge Contraction PoolingThe European Journal on Artificial Intelligence10.1177/30504554241301393Online publication date: 4-Feb-2025
      • (2024)Ultra-dense Motion Capture: An exploratory full-automatic approach for dense tracking of breast motion in 4DPLOS ONE10.1371/journal.pone.029904019:2(e0299040)Online publication date: 26-Feb-2024
      • (2024)Enhancing 3D human pose estimation with NIR single-pixel imaging and time-of-flight technology: a deep learning approachJournal of the Optical Society of America A10.1364/JOSAA.49993341:3(414)Online publication date: 13-Feb-2024
      • (2024)A Neural Network Model for Efficient Musculoskeletal-Driven Skin DeformationACM Transactions on Graphics10.1145/365813543:4(1-12)Online publication date: 19-Jul-2024
      • (2024)LayerNet: High-Resolution Semantic 3D Reconstruction of Clothed PeopleIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2023.333267746:2(1257-1272)Online publication date: 1-Feb-2024
      • (2024)GHNeRF: Learning Generalizable Human Features with Efficient Neural Radiance Fields2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)10.1109/CVPRW63382.2024.00287(2812-2821)Online publication date: 17-Jun-2024
      • (2024)Non-rigid point cloud registration module based on local human body parts2024 5th International Conference on Computer Vision, Image and Deep Learning (CVIDL)10.1109/CVIDL62147.2024.10604054(699-702)Online publication date: 19-Apr-2024
      • (2024)BLiSS: Bootstrapped Linear Shape Space2024 International Conference on 3D Vision (3DV)10.1109/3DV62453.2024.00018(569-580)Online publication date: 18-Mar-2024
      • (2024)Mesh deformation-based single-view 3D reconstruction of thin eyeglasses frames with differentiable renderingGraphical Models10.1016/j.gmod.2024.101225135(101225)Online publication date: Oct-2024
      • (2024)Variational autoencoders for 3D data processingArtificial Intelligence Review10.1007/s10462-023-10687-x57:2Online publication date: 8-Feb-2024
      • Show More Cited By

      View Options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Login options

      Full Access

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media