Elsevier

Computers & Graphics

Volume 97, June 2021, Pages 258-267
Computers & Graphics

Special Section on CAD & Graphics 2021
Craniofacial reconstruction based on heat flow geodesic grid regression (HF-GGR) model

https://doi.org/10.1016/j.cag.2021.04.029Get rights and content

Highlights

  • Propose an encode the craniofacial geometry method by geodesic grid made by the angle division map.

  • Give a more compact geodesic grid regression model for craniofacial reconstruction.

  • The extensive experimental results show that our algorithm can achieve accurate reconstruction results with faster speed and fewer geodesic grid points than the state-of-the-art method.

Abstract

Craniofacial reconstruction is to predict the 3D facial geometry according to the internal relationship between the skull and face, which is widely applied in the field of criminal investigation, archaeology, forensic medicine and so on. In this paper, utilizing the inherent advantage of geodesic to encode craniofacial geometry, we propose a heat flow geodesic grid regression (HF-GGR) model to facilitate craniofacial reconstruction. Our algorithm consists of three steps. In the first step, we extract the nose-tip rooted geodesic distance field and discretize it into a radial grid representation. Then in the second step, we generate geodesic grid of target skull appearance by utilizing the partial least squares regression (PLSR) method. Finally in the third step, we reconstruct the face of target skull according to the geodesic grid and face statistical model. We have conducted experiments on a data set with 213 pairs of craniofacial data. The extensive experimental results show that our algorithm can achieve accurate reconstruction results with faster speed and fewer geodesic grid points than the state-of-the-art method.

Introduction

Craniofacial reconstruction is a challenging yet fascinating research task and becomes increasingly important in the field of forensic medicine, archaeology, criminal investigation and so on. In the field of archaeology, reconstructing the remains of mausoleums cannot only satisfy archaeologists desire to explore ancient peoples appearance, but also acquire the analysis of cultural environment of that time by combining the reconstructed appearances and historical facts. In the field of forensic medicine, in order to help patients with large-scale burn or cosmetic surgery to carry out skin transplantation, we can also utilize the craniofacial reconstruction technique to reconstruct patient’s original face, which can help doctors carry out the next step of operation. In the field of criminal investigation, craniofacial reconstruction can help policemen to identify the unknown corpse.

However, craniofacial reconstruction is not easy. Traditionally, craniofacial reconstruction, as a professional craft, relies on sculptors’ anatomy knowledge and carving skill, which is a tedious and time consuming process. Furthermore, the final reconstruction is inevitably subjective. With the advancement of digital scanning and geometry processing technology, it becomes possible to emancipate sculptors from the hard work. The basic principle to design an automatic craniofacial reconstruction algorithm is based on the observation that there is an interesting coupling relationship between the skull geometry and the facial geometry. Therefore, the algorithm design includes at least two considerations: (1) which kind of information is able to better encode the skull/facial geometry, and (2) how the statistical model is designed to make the reconstruction as fast/accurate as possible.

Geodesic is an important intrinsic measure in differential geometry. Although it can characterize how the surface is curved, the expensive computational cost restricts its use in a wide range of applications. Balance between accuracy and run-time performance has been a central problem of CG (compute geodesic) for decades (since the early beginning). Many excellent geodesic algorithms are proposed for solving the different problems. In this paper, considering the running speed and the smoothness of geodesics, we use the heat flow algorithm to extract the geodesic distances at a group of radially distributed grid points and further encode the geometry of craniofacial training data into an array-like structure. After that, we predict geodesic grid corresponding to the user-specified skull data based on the partial least squares regression model. Finally, we perform the real craniofacial reconstruction operation by utilizing the geodesic grid and the face statistical model. Experimental results validate the effectiveness of the novel algorithm framework.

  • 1.

    We use the angle division map to encode the craniofacial geometry, facilitated by the heat flow method that can report the geodesic distances/paths very quickly.

  • 2.

    We give a more compact geodesic grid regression model. Rather than use tens of thousands of mesh vertices, we sample the geodesic distances at a group of radially distributed grid points, which is totally independent of the mesh resolution. In this way, the computational cost is greatly reduced.

  • 3.

    Experiments on 213 pairs of craniofacial data were conducted and the experimental results show that our method can reconstruct facial models more accurate than classic craniofacial reconstruction methods.

The sections of this paper are arranged as follows. Section 2 introduces related works about our method. Our algorithm is elaborated in Section 3, which consists of (a) building craniofacial geodesic grid, (b) computing geodesic distance by heat flow method, (c) generating facial geodesic grid of the user-specified skull by partial least squares regression and (d) reconstructing face by utilizing face statistical model. Next, we present experimental results in Section 4. Finally, we conclude this paper in Section 5.

Section snippets

Craniofacial reconstruction

Craniofacial reconstruction methods include knowledge-based craniofacial reconstruction and statistical model-based craniofacial reconstruction. Knowledge-based craniofacial reconstruction [1], [2], [3], [4], [5], [6] depends on learning of soft tissues location and depth. It involves accurate skull registration and professional knowledge. With the development of science and technology, more and more statistical models are proposed for craniofacial reconstruction.

Statistical model-based

Method

In this paper, we encode craniofacial geometry as a geodesic grid, which built on craniofacial training data by heat flow method. The geodesic grid of the face surface corresponding to user-specified skull is established by partial least squares regression model, and then the facial appearance of target skull is estimated utilizing face statistical model. The flow chart of our method is shown by Fig. 1. N is the number of training craniofacial data.

Experimental results and analysis

In experiments, we firstly compute geodesic distance, extract geodesic path based on heat flow method for establishing geodesic grid. Then we establish face statistical model and utilize generated geodesic grid for craniofacial reconstruction. PLSR is not utilized for reconstructing face directly, but utilized for generate facial geodesic grid of target skull. The final craniofacial reconstruction results can be obtained by utilizing face statistical model and generated geodesic grid.

Conclusion

In this paper, a craniofacial reconstruction based on heat flow geodesic grid regression model is proposed. We utilize the inherent advantage of geodesic to encode craniofacial geometry automatically. And facilitated by the heat flow method, the calculated geodesic distance, geodesic path for establishing geodesic grid are smoother than calculated by FMM. PLSR is not utilized for reconstructing facial surface directly, but be utilized for generating geodesic grid. With the help of geodesic grid

CRediT authorship contribution statement

Bin Jia: Methodology, Software, Validation, Writing - original draft. Junli Zhao: Methodology, Conceptualization, Investigation, Writing - review & editing. Shiqing Xin: Methodology, Writing - review & editing. Fuqing Duan: Methodology, Supervision, Writing - review & editing. Zhenkuan Pan: . Zhongke Wu: Methodology, Supervision. Jinhua Li: Methodology, Supervision. Mingquan Zhou: Data curation.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

The authors gratefully appreciated the anonymous reviewers for all of their helpful comments. This work was supported by the National Key R&D Program of China under Grant No. 2019YFC1521103, the National Natural Science Foundation of China under Grant Nos. 61702293, 11472144, 61772016 61772294, 61572078, the National Statistical Science Research Project (No.2020355) and Development Plan - Major Scientific and Technological Innovation Projects of Shandong Province (No.2019JZZY02010), and the

References (39)

  • L. Cao et al.

    Computing smooth quasi-geodesic distance field (qgdf) with quadratic programming

    Comput-Aided Des

    (2020)
  • R. Huang et al.

    Automatic craniofacial registration based on radial curves

    Comput Graph

    (2019)
  • T. Gietzen et al.

    A method for automatic forensic facial reconstruction based on dense statistics of soft tissue thickness

    PloS One

    (2019)
  • P. Paysan et al.

    Face reconstruction from skull shapes and physical attributes

    Joint pattern recognition symposium

    (2009)
  • Z. Xiao et al.

    Craniofacial reconstruction using gaussian process latent variable models

    International conference on computer analysis of images and patterns

    (2015)
  • Y.-f. Zhang et al.

    Face appearance reconstruction based on a regional statistical craniofacial model (rcsm)

    2010 20th international conference on pattern recognition

    (2010)
  • Y. Hu et al.

    A hierarchical dense deformable model for 3d face reconstruction from skull

    Multim Tools Appl

    (2013)
  • D. Madsen et al.

    Probabilistic joint face-skull modelling for facial reconstruction

    Proceedings of the IEEE conference on computer vision and pattern recognition

    (2018)
  • W. Shui et al.

    A computerized craniofacial reconstruction method for an unidentified skull based on statistical shape models

    Multim Tools Appl

    (2020)
  • Cited by (5)

    • CR-GAN: Automatic craniofacial reconstruction for personal identification

      2022, Pattern Recognition
      Citation Excerpt :

      Therefore, developing a mathematical model for the assumed relationship between the face and the underlying skull substrate based on CT scan data is the most critical task for computerized CR. In recent decades, there has been increasing interest in using classical machine learning-based methods to perform CR tasks, for example, least squares support vector regression [6], latent root regression [7], and partial least squares regression [8,9]. These approaches represent the 3D geometry of the skull and craniofacial surfaces in two different rough shape parameter spaces, and then fit the correspondence between these two shape parameter spaces using a statistical model.

    • Facial approximation of a Zaghunluq mummy in Xinjiang, China

      2024, Archaeological and Anthropological Sciences
    • An End-to-End Conditional Generative Adversarial Network Based on Depth Map for 3D Craniofacial Reconstruction

      2022, MM 2022 - Proceedings of the 30th ACM International Conference on Multimedia
    View full text