research-article

SkinMixer: Blending 3D Animated Models

Authors:

Stefano Nuvoli,

Riccardo Scateni,

Marco TariniAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 41, Issue 6

Article No.: 250, Pages 1 - 15

https://doi.org/10.1145/3550454.3555503

Published: 30 November 2022 Publication History

Abstract

We propose a novel technique to compose new 3D animated models, such as videogame characters, by combining pieces from existing ones. Our method works on production-ready rigged, skinned, and animated 3D models to reassemble new ones. We exploit mix-and-match operations on the skeletons to trigger the automatic creation of a new mesh, linked to the new skeleton by a set of skinning weights and complete with a set of animations. The resulting model preserves the quality of the input meshings (which can be quad-dominant and semi-regular), skinning weights (inducing believable deformation), and animations, featuring coherent movements of the new skeleton.

Our method enables content creators to reuse valuable, carefully designed assets by assembling new ready-to-use characters while preserving most of the hand-crafted subtleties of models authored by digital artists. As shown in the accompanying video, it allows for drastically cutting the time needed to obtain the final result.

Supplemental Material

MP4 File

presentation

Download
201.28 MB

References

[1]

Kfir Aberman, Yijia Weng, Dani Lischinski, Daniel Cohen-Or, and Baoquan Chen. 2020. Unpaired Motion Style Transfer from Video to Animation. ACM Trans. Graph. 39, 4, Article 64 (2020), 12 pages.

Digital Library

[2]

Adib Akl, Charles Yaacoub, Marc Donias, Jean-Pierre Da Costa, and Christian Germain. 2018. A survey of exemplar-based texture synthesis methods. Computer Vision and Image Understanding 172 (2018), 12--24.

[3]

Baptiste Angles, Marco Tarini, Brian Wyvill, Loïc Barthe, and Andrea Tagliasacchi. 2017. Sketch-Based Implicit Blending. ACM Trans. Graph. 36, 6, Article 181 (2017), 13 pages.

Digital Library

[4]

Autodesk. 2019. Maya.

[5]

Seungbae Bang and Sung-Hee Lee. 2018. Spline Interface for Intuitive Skinning Weight Editing. ACM Trans. Graph. 37, 5, Article 174 (2018), 14 pages.

Digital Library

[6]

Ilya Baran and Jovan Popović. 2007. Automatic Rigging and Animation of 3D Characters. ACM Trans. Graph. 26, 3 (2007), 72-es.

Digital Library

[7]

Gavin Barill, Neil G. Dickson, Ryan Schmidt, David I. W. Levin, and Alec Jacobson. 2018. Fast Winding Numbers for Soups and Clouds. ACM Trans. Graph. 37, 4, Article 43 (2018), 12 pages.

Digital Library

[8]

Adrien Bernhardt, Loïc Barthe, Marie-Paule Cani, and Brian Wyvill. 2013. Implicit Blending Revisited. Comput. Graph. Forum 29, 2 (2013), 367--376.

[9]

Stephan Bischoff and Leif Kobbelt. 2005. Structure Preserving CAD Model Repair. Comput. Graph. Forum 24, 3 (2005), 527--536.

[10]

David Bommes, Bruno Lévy, Nico Pietroni, Enrico Puppo, Cláudio T. Silva, Marco Tarini, and Denis Zorin. 2013. Quad-Mesh Generation and Processing: A Survey. Comput. Graph. Forum 32, 6 (2013), 51--76.

Digital Library

[11]

Mazen Borno, Ludovic Righetti, Michael Black, Scott Delp, Eugene Fiume, and Javier Romero. 2018. Robust Physics-based Motion Retargeting with Realistic Body Shapes. Comput. Graph. Forum 37, 8 (2018), 81--92.

[12]

Péter Borosán, Ming Jin, Doug DeCarlo, Yotam Gingold, and Andrew Nealen. 2012. RigMesh: Automatic Rigging for Part-Based Shape Modeling and Deformation. ACM Trans. Graph. 31, 6, Article 198 (2012), 9 pages.

Digital Library

[13]

Sofien Bouaziz, Yangang Wang, and Mark Pauly. 2013. Online Modeling for Realtime Facial Animation. ACM Trans. Graph. 32, 4, Article 40 (2013), 10 pages.

Digital Library

[14]

Adnane Boukhayma, Jean-Sébastien Franco, and Edmond Boyer. 2017. Surface Motion Capture Transfer with Gaussian Process Regression. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 3558--3566.

[15]

Marcel Campen and Leif Kobbelt. 2010. Exact and Robust (Self-)Intersections for Polygonal Meshes. Comput. Graph. Forum 29, 2 (2010), 397--406.

[16]

Siddhartha Chaudhuri, Daniel Ritchie, Kai Xu, and Hao (Richard) Zhang. 2019. Learning Generative Models of 3D Structures. In Eurographics 2019 - Tutorials, Wenzel Jakob and Enrico Puppo (Eds.). The Eurographics Association.

[17]

Zhiqin Chen and Hao Zhang. 2019. Learning Implicit Fields for Generative Shape Modeling. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 5939--5948.

[18]

Gianmarco Cherchi, Marco Livesu, Riccardo Scateni, and Marco Attene. 2020. Fast and Robust Mesh Arrangements Using Floating-Point Arithmetic. ACM Trans. Graph. 39, 6, Article 250 (2020), 16 pages.

Digital Library

[19]

Kwang-Jin Choi and Hyeong-Seok Ko. 2000. Online motion retargetting. The Journal of Visualization and Computer Animation 11, 5 (2000), 223--235.

[20]

CNR. 2013. The Visualization and Computer Graphics Library. http://vcg.isti.cnr.it/vcglib/

[21]

Blender Online Community. 2018. Blender - a 3D modelling and rendering package. Blender Foundation, Stichting Blender Foundation, Amsterdam.

[22]

Brian Curless and Marc Levoy. 1996. A Volumetric Method for Building Complex Models from Range Images. In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '96). Association for Computing Machinery, New York, NY, USA, 303--312.

Digital Library

[23]

Olivier Dionne and Martin de Lasa. 2013. Geodesic Voxel Binding for Production Character Meshes. In Proceedings of the 12th ACM SIGGRAPH/Eurographics Symposium on Computer Animation (Anaheim, California) (SCA '13). Association for Computing Machinery, New York, NY, USA, 173--180.

Digital Library

[24]

Noah Duncan, Lap-Fai Yu, and Sai-Kit Yeung. 2016. Interchangeable Components for Hands-on Assembly Based Modelling. ACM Trans. Graph. 35, 6, Article 234 (2016), 14 pages.

Digital Library

[25]

Thomas A. Funkhouser, Michael M. Kazhdan, Philip Shilane, Patrick Min, William Kiefer, Ayellet Tal, Szymon Rusinkiewicz, and David P. Dobkin. 2004. Modeling by example. ACM Trans. Graph. 23, 3 (2004), 652--663.

Digital Library

[26]

Lin Gao, Jie Yang, Yi-Ling Qiao, Yu-Kun Lai, Paul L. Rosin, Weiwei Xu, and Shihong Xia. 2018. Automatic Unpaired Shape Deformation Transfer. ACM Trans. Graph. 37, 6, Article 237 (2018), 15 pages.

Digital Library

[27]

Michael Gleicher. 1998. Retargetting Motion to New Characters. In Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '98). Association for Computing Machinery, New York, NY, USA, 33--42.

Digital Library

[28]

Olivier Gourmel, Loic Barthe, Marie-Paule Cani, Brian Wyvill, Adrien Bernhardt, Mathias Paulin, and Herbert Grasberger. 2013. A Gradient-Based Implicit Blend. ACM Trans. Graph. 32, 2, Article 12 (2013), 12 pages.

Digital Library

[29]

Keith Grochow, Steven L. Martin, Aaron Hertzmann, and Zoran Popović. 2004. Style-Based Inverse Kinematics. ACM Trans. Graph. 23, 3 (2004), 522--531.

Digital Library

[30]

Gael Guennebaud, Benoit Jacob, et al. 2014. Eigen: a C++ linear algebra library. http://eigen.tuxfamily.org

[31]

LLC Gurobi Optimization. 2018. Gurobi Optimizer Reference Manual.

[32]

Chris Hecker, Bernd Raabe, Ryan W. Enslow, John DeWeese, Jordan Maynard, and Kees van Prooijen. 2008. Real-Time Motion Retargeting to Highly Varied User-Created Morphologies. In ACM SIGGRAPH 2008 Papers (Los Angeles, California) (SIGGRAPH '08). Association for Computing Machinery, New York, NY, USA, Article 27, 11 pages.

Digital Library

[33]

Daniel Holden, Jun Saito, and Taku Komura. 2016. A Deep Learning Framework for Character Motion Synthesis and Editing. ACM Trans. Graph. 35, 4, Article 138 (2016), 11 pages.

Digital Library

[34]

Alec Jacobson, Ilya Baran, Jovan Popović, and Olga Sorkine. 2011. Bounded Biharmonic Weights for Real-Time Deformation. ACM Trans. Graph. 30, 4, Article 78 (2011), 8 pages.

Digital Library

[35]

Alec Jacobson, Ladislav Kavan, and Olga Sorkine-Hornung. 2013. Robust inside-outside segmentation using generalized winding numbers. ACM Trans. Graph. 32, 4, Article 33 (2013), 12 pages.

Digital Library

[36]

Alec Jacobson, Daniele Panozzo, et al. 2016. libigl: A simple C++ geometry processing library. https://libigl.github.io

[37]

Won-Seob Jang, Won-Kyu Lee, In-Kwon Lee, and J. Lee. 2008. Enriching a motion database by analogous combination of partial human motions. Vis. Comput. 24 (2008), 271--280.

Digital Library

[38]

R. Kenny Jones, Theresa Barton, Xianghao Xu, Kai Wang, Ellen Jiang, Paul Guerrero, Niloy J. Mitra, and Daniel Ritchie. 2020. ShapeAssembly: Learning to Generate Programs for 3D Shape Structure Synthesis. ACM Trans. Graph. 39, 6, Article 234 (2020), 20 pages.

Digital Library

[39]

Ladislav Kavan, Steven Collins, Jiří Žára, and Carol O'Sullivan. 2007. Skinning with Dual Quaternions. In Proceedings of the 2007 Symposium on Interactive 3D Graphics and Games (Seattle, Washington) (I3D '07). Association for Computing Machinery, New York, NY, USA, 39--46.

Digital Library

[40]

Ladislav Kavan and Olga Sorkine. 2012. Elasticity-Inspired Deformers for Character Articulation. ACM Trans. Graph. 31, 6, Article 196 (2012), 8 pages.

Digital Library

[41]

Vladislav Kreavoy, Dan Julius, and Alla Sheffer. 2007. Model Composition from Interchangeable Components. In 15th Pacific Conference on Computer Graphics and Applications (PG'07). 129--138.

Digital Library

[42]

Lijuan Liu, Youyi Zheng, Di Tang, Yi Yuan, Changjie Fan, and Kun Zhou. 2019. NeuroSkinning: Automatic Skin Binding for Production Characters with Deep Graph Networks. ACM Trans. Graph. 38, 4, Article 114 (2019), 12 pages.

Digital Library

[43]

Andrea Maggiordomo, Paolo Cignoni, and Marco Tarini. 2021. Texture Defragmentation for Photo-Reconstructed 3D Models. Comput. Graph. Forum 40, 2 (2021), 65--78.

[44]

Giorgio Marcias, Nico Pietroni, Daniele Panozzo, Enrico Puppo, and Olga Sorkine-Hornung. 2013. Animation-Aware Quadrangulation. Comput. Graph. Forum 32, 5 (2013), 167--175.

Digital Library

[45]

Andelo Martinovic and Luc Van Gool. 2013. Bayesian Grammar Learning for Inverse Procedural Modeling. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 201--208.

Digital Library

[46]

Simone Melzi, Riccardo Marin, Pietro Musoni, Filippo Bardon, Marco Tarini, and Umberto Castellani. 2020. Intrinsic/extrinsic embedding for functional remeshing of 3D shapes. Computers & Graphics 88 (2020), 1--12.

[47]

Eray Molla, Henrique Galvan Debarba, and Ronan Boulic. 2018. Egocentric Mapping of Body Surface Constraints. IEEE Transactions on Visualization and Computer Graphics 24, 7 (2018), 2089--2102.

[48]

Pascal Müller, Peter Wonka, Simon Haegler, Andreas Ulmer, and Luc Van Gool. 2006. Procedural Modeling of Buildings. ACM Trans. Graph. 25, 3 (2006), 614--623.

Digital Library

[49]

Alessandro Muntoni and Stefano Nuvoli. 2021. CG3Lib: A C++ geometry processing library.

[50]

Ken Museth, Jeff Lait, John Johanson, Jeff Budsberg, Ron Henderson, Mihai Alden, Peter Cucka, David Hill, and Andrew Pearce. 2013. OpenVDB: An Open-Source Data Structure and Toolkit for High-Resolution Volumes. In ACM SIGGRAPH 2013 Courses (Anaheim, California) (SIGGRAPH '13). Association for Computing Machinery, New York, NY, USA, Article 19, 1 pages.

Digital Library

[51]

Stefano Nuvoli, Alex Hernandez, Claudio Esperança, Riccardo Scateni, Paolo Cignoni, and Nico Pietroni. 2019. QuadMixer: Layout Preserving Blending of Quadrilateral Meshes. ACM Trans. Graph. 38, 6, Article 180 (2019), 13 pages.

Digital Library

[52]

Junjun Pan, Lijuan Chen, Yuhan Yang, and Hong Qin. 2018. Automatic Skinning and Weight Retargeting of Articulated Characters Using Extended Position-Based Dynamics. Vis. Comput. 34, 10 (2018), 1285--1297.

Digital Library

[53]

Daniele Panozzo, Yaron Lipman, Enrico Puppo, and Denis Zorin. 2012. Fields on Symmetric Surfaces. ACM Trans. Graph. 31, 4, Article 111 (2012), 12 pages.

Digital Library

[54]

Yoav I. H. Parish and Pascal Müller. 2001. Procedural Modeling of Cities. In Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '01). Association for Computing Machinery, New York, NY, USA, 301--308.

Digital Library

[55]

Jeong Joon Park, Peter R. Florence, Julian Straub, Richard A. Newcombe, and S. Love-grove. 2019. DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 165--174.

[56]

Darko Pavic, Marcel Campen, and Leif Kobbelt. 2010. Hybrid Booleans. Comput. Graph. Forum 29, 1 (2010), 75--87.

[57]

Nico Pietroni, Stefano Nuvoli, Thomas Alderighi, Paolo Cignoni, and Marco Tarini. 2021. Reliable Feature-Line Driven Quad-Remeshing. ACM Trans. Graph. 40, 4, Article 155 (2021), 17 pages.

Digital Library

[58]

Daniel Ritchie, Sarah Jobalia, and Anna Thomas. 2018. Example-based Authoring of Procedural Modeling Programs with Structural and Continuous Variability. Comput. Graph. Forum 37, 2 (2018), 401--413.

[59]

Christian Roessl, Leif Kobbelt, and Hans-Peter Seidel. 2000. Extraction of feature lines on triangulated surfaces using morphological operators. In Papers from 2000 AAAI Spring Symposium: Smart Graphics. AAAI, 71--75.

[60]

Ryan Schmidt and Karan Singh. 2010. Meshmixer: An Interface for Rapid Mesh Composition. In ACM SIGGRAPH 2010 Talks (Los Angeles, California) (SIGGRAPH '10). ACM, New York, NY, USA, Article 6, 1 pages.

Digital Library

[61]

Andrei Sharf, Marina Blumenkrants, Ariel Shamir, and Daniel Cohen-Or. 2006. Snap-Paste: an interactive technique for easy mesh composition. Vis. Comput. 22, 9--11 (2006), 835--844.

Digital Library

[62]

Karan Singh and Richard E. Parent. 2001. Joining polyhedral objects using implicitly defined surfaces. Vis. Comput. 17, 7 (2001), 415--428.

[63]

Sebastian Starke, Yiwei Zhao, Fabio Zinno, and Taku Komura. 2021. Neural Animation Layering for Synthesizing Martial Arts Movements. ACM Trans. Graph. 40, 4, Article 92 (2021), 16 pages.

Digital Library

[64]

Robert W. Sumner and Jovan Popović. 2004. Deformation Transfer for Triangle Meshes. ACM Trans. Graph. 23, 3 (2004), 399--405.

Digital Library

[65]

Sarah Taylor, Taehwan Kim, Yisong Yue, Moshe Mahler, James Krahe, Anastasio Garcia Rodriguez, Jessica Hodgins, and Iain Matthews. 2017. A Deep Learning Approach for Generalized Speech Animation. ACM Trans. Graph. 36, 4, Article 93 (2017), 11 pages.

Digital Library

[66]

Ruben Villegas, Jimei Yang, Duygu Ceylan, and Honglak Lee. 2018. Neural Kinematic Networks for Unsupervised Motion Retargetting. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 8639--8648.

[67]

The Foundry Visionmongers. 2018. Modo 12.1.

[68]

Jungdam Won and Jehee Lee. 2019. Learning Body Shape Variation in Physics-Based Characters. ACM Trans. Graph. 38, 6, Article 207 (2019), 12 pages.

Digital Library

[69]

Jiajun Wu, Chengkai Zhang, Tianfan Xue, William T. Freeman, and Joshua B. Tenenbaum. 2016. Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling. In Proceedings of the 30th International Conference on Neural Information Processing Systems (NIPS'16). Curran Associates Inc., Red Hook, NY, USA, 82--90.

[70]

Zhan Xu, Yang Zhou, Evangelos Kalogerakis, Chris Landreth, and Karan Singh. 2020. RigNet: Neural Rigging for Articulated Characters. ACM Trans. Graph. 39, 4, Article 58 (2020), 14 pages.

Digital Library

[71]

Zhan Xu, Yang Zhou, Evangelos Kalogerakis, and Karan Singh. 2019. Predicting Animation Skeletons for 3D Articulated Models via Volumetric Nets. In 2019 International Conference on 3D Vision (3DV). 298--307.

[72]

Kangxue Yin, Zhiqin Chen, Siddhartha Chaudhuri, Matthew Fisher, Vladimir G. Kim, and Hao Zhang. 2020. COALESCE: Component Assembly by Learning to Synthesize Connections. In 2020 International Conference on 3D Vision (3DV). 61--70.

[73]

Juyong Zhang, Chunlin Wu, Jianfei Cai, Jianmin Zheng, and Xue-Cheng Tai. 2010. Mesh Snapping: Robust Interactive Mesh Cutting Using Fast Geodesic Curvature Flow. Comput. Graph. Forum 29, 2 (2010), 517--526.

[74]

Jiaran Zhou, Marcel Campen, Denis Zorin, Changhe Tu, and Cláudio T. Silva. 2018. Quadrangulation of non-rigid objects using deformation metrics. Computer Aided Geometric Design 62 (2018), 3--15.

Digital Library

[75]

Chenyang Zhu, Kai Xu, Siddhartha Chaudhuri, Renjiao Yi, and Hao Zhang. 2018. SCORES: Shape Composition with Recursive Substructure Priors. ACM Trans. Graph. 37, 6, Article 211 (2018), 14 pages.

Digital Library

Cited By

Cao LPeng C(2024)Hierarchical Spherical Cross‐Parameterization for Deforming CharactersComputer Graphics Forum10.1111/cgf.1519743:6Online publication date: 19-Sep-2024
https://doi.org/10.1111/cgf.15197
Zhan XFu RRitchie D(2024)CharacterMixer: Rig‐Aware Interpolation of 3D CharactersComputer Graphics Forum10.1111/cgf.1504743:2Online publication date: 30-Apr-2024
https://doi.org/10.1111/cgf.15047
Xu RDou ZWang NXin SChen SJiang MGuo XWang WTu C(2023)Globally Consistent Normal Orientation for Point Clouds by Regularizing the Winding-Number FieldACM Transactions on Graphics10.1145/359212942:4(1-15)Online publication date: 26-Jul-2023
https://dl.acm.org/doi/10.1145/3592129

Index Terms

SkinMixer: Blending 3D Animated Models
1. Computing methodologies
  1. Computer graphics
    1. Animation
    2. Shape modeling
      1. Mesh models

Recommendations

High Performance Automatic Character Skinning Based on Projection Distance

Skeleton-driven-deformation methods have been commonly used in the character deformations. The process of painting skin weights for character deformation is a long-winded task requiring manual tweaking. We present a novel method to calculate skinning ...
An art-directed wrinkle system for CG character clothing and skin

We present a kinematic system for creating art-directed clothing and skin wrinkles on CG characters used in the production of computer-animated feature films. This system employs a curve-based method for generating wrinkles on reference poses, which are ...
Simulation-based in-between creation for CACAni system
SIGGRAPH ASIA '09: ACM SIGGRAPH ASIA 2009 Sketches

In-between creation in traditional cel animation based on the hand-drawn key-frames is a fundamental element for the actual production and plays a symbolic role in an artistic interpretation of the scene. To create impressive in-betweens, however, ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 41, Issue 6

December 2022

1428 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/3550454

Issue’s Table of Contents

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 November 2022

Published in TOG Volume 41, Issue 6

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
391
Total Downloads

Downloads (Last 12 months)80
Downloads (Last 6 weeks)14

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Cao LPeng C(2024)Hierarchical Spherical Cross‐Parameterization for Deforming CharactersComputer Graphics Forum10.1111/cgf.1519743:6Online publication date: 19-Sep-2024
https://doi.org/10.1111/cgf.15197
Zhan XFu RRitchie D(2024)CharacterMixer: Rig‐Aware Interpolation of 3D CharactersComputer Graphics Forum10.1111/cgf.1504743:2Online publication date: 30-Apr-2024
https://doi.org/10.1111/cgf.15047
Xu RDou ZWang NXin SChen SJiang MGuo XWang WTu C(2023)Globally Consistent Normal Orientation for Point Clouds by Regularizing the Winding-Number FieldACM Transactions on Graphics10.1145/359212942:4(1-15)Online publication date: 26-Jul-2023
https://dl.acm.org/doi/10.1145/3592129

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents