Parametric modeling of 3D human body shape—A survey

Highlights

• A survey of parametric 3D body shape modeling approaches, with emphasis on their use in body shape capture.

• An analysis of 3D body shape processing applications based on parametric Modeling.

• Outline industrial applications and areas needing more research.

Abstract

Parametric modeling of 3D body shape is widely used to create realistic human bodies. It furthermore permits robust reconstruction of complete 3D body shapes even from incomplete capture data—traditional scanning techniques result in gaps and missing regions due to occlusion and inaccessibility of certain areas of the body. Numerous methods of parametric modeling have been proposed for a wide variety of 3D body processing tasks. They provide the ability to represent a range of identity-dependent body shapes, and to deform them naturally into various poses. This report surveys and classifies recent developments in parametric 3D body shape modeling. We focus on elucidating the key similarities and differences between existing methods as an aid to understanding their relationships. We also discuss a variety of 3D body shape processing applications that benefit from parametric modeling. Our analysis of the strengths and limitations of existing algorithms also lets us highlight opportunities for future research.

Introduction

3D human body shape modeling is a classical problem in both academia and industry. In the past, representing body shape with high realism required a professional artist to manually model and animate the human body, a highly skilled task. The advent of 3D scanning has created the opportunity to capture human body geometry and texture in detail, but it still typically involves a professional acquisition process. In particular, holes and gaps are always present in the scanned data due to self-occlusion and inaccessibility of places such as the armpits. Self-contact causing topological changes is also problematic. The use of traditional scanning techniques results in artifacts, primarily missing regions of a non-trivial size.

Fortunately, (unclothed) 3D bodies share a common structure, both in terms of identity-dependent body shape, and pose-dependent shape as they animate. So, researchers have proposed use of parametric models to represent 3D human body shape, building upon statistical analysis of high-quality 3D body training data. In this paper, we consider to survey such parametric modeling technique, which can represent a range of identity-dependent body shapes, and deform them naturally into various poses. When used in conjunction with data capture systems, the advantage over traditional scanning is the robust ability to automatically reconstruct a complete 3D body model from incomplete data. High fidelity results are achieved due to such models being data-driven, using a high-quality body shape dataset to learn the models. Nevertheless, to capture the intricacies of human body shape, the mathematical descriptions used by parametric models are quite complicated. Building a model thus involves many issues, including 3D body training dataset preparation, designing a proper body model, and training the model to fit the prepared data.

Anguelov [1] pioneered parametric modeling methods of 3D body shape, introducing the fundamental SCAPE (Shape Completion and Animation for PEople) method. SCAPE is a statistical model that captures correlations of shape deformations between different individual bodies as well as correlations of pose deformations. Many following works have improved upon SCAPE, which provides a highly flexible and realistic body model. Our survey reviews existing methods for parametric modeling of 3D body shape, and their wide applications to human body processing tasks. We introduce the necessary mathematical concepts as well as current methods, and use various high-level criteria to organize existing work into several categories, emphasizing their similarities and differences. Our goal is that this comprehensive survey will help readers navigate the constantly expanding literature on parametric 3D body shape modeling, and inspire researchers to contribute to this promising field in the future.

Other surveys have to some degree reviewed the topics covered in this report. We discuss them here to explain their differences and the need for our paper.

The recent course in [2] provides a deep discussion on learning human body shapes in motion. This course is the most similar work to our survey, and includes a solid introduction to the parametric modeling of 3D body shapes. However, it focuses on work and progress in the Perceiving Systems department at the Max Planck Institute for Intelligent Systems, while we aim to provide a comprehensive survey which considers work from multiple research teams, analyzing them as a whole. More importantly, we wish to highlight future perspectives based on an analysis of the strengths and limitations of existing works.

The original SCAPE-based methods are briefly introduced along with other data driven methods in [3], but since then, important developments have been made beyond the original SCAPE model.

Brunton [4] presents an overview of statistical analysis, especially the PCA technique for face processing, but does not give a profound discussion of techniques for human body shape.

A 3D human body can be represented as an explicit surface, e.g. a triangular mesh, which is the focus of this survey, but other representations such as implicit surfaces and volumetric models may also be used. Experiments show that using an explicit surface gives the highest fidelity among different representations. We briefly consider a few illustrative methods using these other representations, to give a broader view of 3D body parametric models.

For implicit surfaces, Gaussians and other parametric proxies have been used to reconstruct 3D body shape, without training on the 3D body dataset. For example, [5] proposes an articulated soft object model, where many 3D Gaussian proxies (also known as metaballs or soft objects [6]) are attached to an articulated skeleton to provide an anatomically-based approximation. Each soft object defines a field and the body skin surface is taken to be a level set of the sum of these proxies. However, the reconstructed body is only a torso, since the head, hands and feet are explicit meshes that are attached to the torso. A similar approach is used by Ilic and Fua [7] to model upper body shape with details. Stoll et al. [8] proposes a sums of Gaussians (SoG) model, which approximates the whole-body shape and can be reconstructed from a sparse set of images, aided by the kinematic skeleton. In related approaches, [9] and [10] use super-quadric proxies for the representation of 3D body shape.

A volumetric Gaussians density body model [11] has recently been proposed for skeletal pose estimation from sparse views; it has been extended by Rhodin et al. [12], using fitting to a registered mesh database [13] for human shape reconstruction.

The rest of our survey is structured as follows. In Section 2, we start with fundamentals: basic definitions used in parametric body shape modeling, and initial works that provide the technical foundations. Section 3 reviews existing parametric models, and elucidates the key similarities and differences between them. 3D body processing applications that make use of parametric models are summarized in Section 4. In Section 5, we show our analysis of the state-of-the-art and elaborate on future perspectives, conclusions are finally made in Section 6.