Survey PaperParametric modeling of 3D human body shape—A survey
Graphical abstract
Introduction
3D human body shape modeling is a classical problem in both academia and industry. In the past, representing body shape with high realism required a professional artist to manually model and animate the human body, a highly skilled task. The advent of 3D scanning has created the opportunity to capture human body geometry and texture in detail, but it still typically involves a professional acquisition process. In particular, holes and gaps are always present in the scanned data due to self-occlusion and inaccessibility of places such as the armpits. Self-contact causing topological changes is also problematic. The use of traditional scanning techniques results in artifacts, primarily missing regions of a non-trivial size.
Fortunately, (unclothed) 3D bodies share a common structure, both in terms of identity-dependent body shape, and pose-dependent shape as they animate. So, researchers have proposed use of parametric models to represent 3D human body shape, building upon statistical analysis of high-quality 3D body training data. In this paper, we consider to survey such parametric modeling technique, which can represent a range of identity-dependent body shapes, and deform them naturally into various poses. When used in conjunction with data capture systems, the advantage over traditional scanning is the robust ability to automatically reconstruct a complete 3D body model from incomplete data. High fidelity results are achieved due to such models being data-driven, using a high-quality body shape dataset to learn the models. Nevertheless, to capture the intricacies of human body shape, the mathematical descriptions used by parametric models are quite complicated. Building a model thus involves many issues, including 3D body training dataset preparation, designing a proper body model, and training the model to fit the prepared data.
Anguelov [1] pioneered parametric modeling methods of 3D body shape, introducing the fundamental SCAPE (Shape Completion and Animation for PEople) method. SCAPE is a statistical model that captures correlations of shape deformations between different individual bodies as well as correlations of pose deformations. Many following works have improved upon SCAPE, which provides a highly flexible and realistic body model. Our survey reviews existing methods for parametric modeling of 3D body shape, and their wide applications to human body processing tasks. We introduce the necessary mathematical concepts as well as current methods, and use various high-level criteria to organize existing work into several categories, emphasizing their similarities and differences. Our goal is that this comprehensive survey will help readers navigate the constantly expanding literature on parametric 3D body shape modeling, and inspire researchers to contribute to this promising field in the future.
Other surveys have to some degree reviewed the topics covered in this report. We discuss them here to explain their differences and the need for our paper.
The recent course in [2] provides a deep discussion on learning human body shapes in motion. This course is the most similar work to our survey, and includes a solid introduction to the parametric modeling of 3D body shapes. However, it focuses on work and progress in the Perceiving Systems department at the Max Planck Institute for Intelligent Systems, while we aim to provide a comprehensive survey which considers work from multiple research teams, analyzing them as a whole. More importantly, we wish to highlight future perspectives based on an analysis of the strengths and limitations of existing works.
The original SCAPE-based methods are briefly introduced along with other data driven methods in [3], but since then, important developments have been made beyond the original SCAPE model.
Brunton [4] presents an overview of statistical analysis, especially the PCA technique for face processing, but does not give a profound discussion of techniques for human body shape.
A 3D human body can be represented as an explicit surface, e.g. a triangular mesh, which is the focus of this survey, but other representations such as implicit surfaces and volumetric models may also be used. Experiments show that using an explicit surface gives the highest fidelity among different representations. We briefly consider a few illustrative methods using these other representations, to give a broader view of 3D body parametric models.
For implicit surfaces, Gaussians and other parametric proxies have been used to reconstruct 3D body shape, without training on the 3D body dataset. For example, [5] proposes an articulated soft object model, where many 3D Gaussian proxies (also known as metaballs or soft objects [6]) are attached to an articulated skeleton to provide an anatomically-based approximation. Each soft object defines a field and the body skin surface is taken to be a level set of the sum of these proxies. However, the reconstructed body is only a torso, since the head, hands and feet are explicit meshes that are attached to the torso. A similar approach is used by Ilic and Fua [7] to model upper body shape with details. Stoll et al. [8] proposes a sums of Gaussians (SoG) model, which approximates the whole-body shape and can be reconstructed from a sparse set of images, aided by the kinematic skeleton. In related approaches, [9] and [10] use super-quadric proxies for the representation of 3D body shape.
A volumetric Gaussians density body model [11] has recently been proposed for skeletal pose estimation from sparse views; it has been extended by Rhodin et al. [12], using fitting to a registered mesh database [13] for human shape reconstruction.
The rest of our survey is structured as follows. In Section 2, we start with fundamentals: basic definitions used in parametric body shape modeling, and initial works that provide the technical foundations. Section 3 reviews existing parametric models, and elucidates the key similarities and differences between them. 3D body processing applications that make use of parametric models are summarized in Section 4. In Section 5, we show our analysis of the state-of-the-art and elaborate on future perspectives, conclusions are finally made in Section 6.
Section snippets
Fundamentals
This section introduces fundamental information about parametric 3D body shape modeling, including basic definitions, datasets available for training, and preliminary work. Note: shape is a general term, and we usually use it to mean both the identity-dependent shape and pose-dependent shape. In addition, when pose refers to the skeletal pose as used in traditional motion capture, it is prefixed by the term skeletal.
Parametric models
We now discuss the pioneering SCAPE model in more detail, and its variants, and then compare typical parametric models of 3D body shape.
Applications
Parametric modeling of 3D body shape has been applied to many tasks. As shown in Table 2, existing applications can be classified into four areas: 3D body shape recovery, motion capture, semantic editing, and other applications.
Analysis and future perspectives
To allow reconstruction of complete 3D body shapes even from incomplete capture data, the research community has focused on and successively developed outstanding 3D parametric body models which statistically analyze available body shape datasets composed of different persons in a varied set of poses.
Overall, existing 3D parametric models body models can be divided into triangle-based and vertex-based types: one considers triangle deformations and the other considers variations in vertex
Conclusions
Parametric modeling of 3D body shape provides the basis for a robust approach to creating realistic body shapes which look and behave like those of real humans. Existing parametric models can represent different body shapes, deform naturally into different poses. Such models which unify identity-dependent and pose-dependent shape generalize to new body shapes not present in the training data set.
We have summarized the common definitions used in such parametric models, and listed 3D body
Acknowledgment
This work was supported by the Natural Science Foundation of China (No. 61602507), Natural Science Foundation of Jiangsu Province (No. BK20150723), Key Researching plan (International and regional cooperation) of Hunan Provincial Department of science and technology (No. 2016WK2038), China Postdoctoral Science Foundation (No. 2016M602555), National Key Research and Development Program of China (No. 2017YFB1103600), Shenzhen Science Plan (JSGG20150925164740726), and the National Natural Science
References (101)
- et al.
Building statistical shape spaces for 3d human modeling
Pattern Recognit
(2017) - et al.
Scape-based human performance reconstruction
Comput Graph
(2014) - et al.
Estimation of human body shape and cloth field in front of a kinect
Neurocomputing
(2015) - et al.
Semantic parametric body shape estimation from noisy depth sequences
Robot Auton Syst
(2016) - et al.
Estimating body shape of dressed humans
Comput Graphics
(2009) - et al.
Estimation of human body shape and posture under clothing
Comput Vis Image Understand
(2014) - et al.
Detailed, accurate, human shape estimation from clothed 3d scan sequences
Proceedings of the CVPR
(2017) - et al.
SCAPE: shape completion and animation of people
ACM Trans Graphics
(2005) - et al.
Learning human body shapes in motion
ACM SIGGRAPH Courses
(2016) - et al.
Data-driven shape analysis and processing
Comput Graph Forum
(2017)
Statistical shape spaces for 3d data: a review
Articulated soft objects for multiview shape and motion capture
IEEE Trans Pattern Anal Mach Intell
A generalization of algebraic surface drawing
ACM Trans Graph
Implicit meshes for surface reconstruction
IEEE Trans Pattern Anal Mach Intel
Fast articulated motion tracking using a sums of Gaussians body model
ICCV
Human pose estimation from silhouettes - a consistent approach using distance level sets
WSCG
Estimating articulated human motion with covariance scaled sampling
Int J Robot Res
A versatile scene model with differentiable visibility applied to generative pose estimation
Proceedings of the ICCV
General automatic human shape and motion capture using volumetric contour cues
Proceedings of the ECCV
A statistical model of human pose and body shape
Comput Graph Forum
The caesar project: a 3-d surface anthropometry survey
Proceedings of the 3DIM
Dyna: a model of dynamic human shape in motion
ACM Trans Graph
Mosh: motion and shape capture from sparse markers
ACM Trans Graph
FAUST: dataset and evaluation for 3d mesh registration
Proceedings of the CVPR
Dynamic faust: registering human bodies in motion
Proceedings of the CVPR
Shape retrieval of non-rigid 3d human models
Int J Comput Vis
Pose space deformation: a unified approach to shape interpolation and skeleton-driven deformation
Proceedings of the SIGGRAPH
Articulated body deformation from range scan data
ACM Trans Graph
The space of human body shapes: reconstruction and parameterization from range scans
ACM Trans Graph
Deformation transfer for triangle meshes
ACM Trans Graph
Mesh editing with poisson-based gradient field manipulation
ACM Trans Graph
Expression cloning
SIGGRAPH
Tensor-based human body modeling
CVPR
Coregistration: simultaneous alignment and modeling of articulated 3d shape
ECCV
Multilinear pose and body shape estimation of dressed subjects from image sets
Proceedings of the CVPR
OpenDR: An approximate differentiable renderer
Proceedings of the ECCV
Breathing life into shape: capturing, modeling and animating 3d human breathing
ACM Trans Graph
Lie bodies: a manifold representation of 3d human shape
Proceedings of the ECCV - Volume Part I
Moviereshape: tracking and reshaping of humans in videos
ACM Trans Graph
SMPL: a skinned multi-person linear model
ACM Trans Graph
Learning a correlated model of identity and pose-dependent body shape variation for real-time synthesis
Proceedings of the SCA
Smooth skinning decomposition with rigid bones
ACM Trans Graph
Realtime reconstruction of an animating human body from a single depth camera
IEEE Trans Visual Comput Graph
Detailed human shape and pose from images
Proceedings of the CVPR
Combined discriminative and generative articulated pose and non-rigid shape estimation
Proceedings of the NIPS
Shining a light on human pose: on shadows, shading and the estimation of pose and shape
Proceedings of the ICCV
The naked truth: estimating body shape under clothing
Proceedings of the ECCV
Cited by (48)
Temporally consistent reconstruction of 3D clothed human surface with warp field
2023, Image and Vision ComputingFast NURBS-based parametric modeling of human calves with high-accuracy for personalized design of graduated compression stockings
2023, Computer Methods and Programs in BiomedicineCitation Excerpt :Normally, massive data are required to represent detailed shapes, increasing both processing cost and application cost. In response to this, various template model-based scaling models have been proposed [23–26], which introduced several registered key parameters to construct target shapes. However, very few works on accurate parametric models of human shapes for fit design of apparels can be found in literature [23,27].
DIMNet: Dense implicit function network for 3D human body reconstruction
2021, Computers and Graphics (Pergamon)Citation Excerpt :Among them, [17] can restore detailed features such as facial expressions. While this method can capture the shape and posture information of the human body [18], it can hardly capture personalized characteristics such as texture, hairstyle, and clothing. In order to solve the problem of inconspicuous personalized features, [19,20], proposes SMPL+D(i.e. Displacement) to enrich personalized features.
Status of research on parametric methods for the reconstruction of 3D models of the human body for virtual fitting
2024, International Journal of Clothing Science and TechnologyThe construction of a three-dimensional human body based on semantic-driven parameters to improve virtual fitting
2024, Textile Research JournalPose and Shape Estimation of Occluded Humans with Attention and Conditional GAN
2024, Jisuanji Fuzhu Sheji Yu Tuxingxue Xuebao/Journal of Computer-Aided Design and Computer Graphics