Special Section on CAD & Graphics 2021Craniofacial reconstruction based on heat flow geodesic grid regression (HF-GGR) model
Graphical abstract
Introduction
Craniofacial reconstruction is a challenging yet fascinating research task and becomes increasingly important in the field of forensic medicine, archaeology, criminal investigation and so on. In the field of archaeology, reconstructing the remains of mausoleums cannot only satisfy archaeologists desire to explore ancient peoples appearance, but also acquire the analysis of cultural environment of that time by combining the reconstructed appearances and historical facts. In the field of forensic medicine, in order to help patients with large-scale burn or cosmetic surgery to carry out skin transplantation, we can also utilize the craniofacial reconstruction technique to reconstruct patient’s original face, which can help doctors carry out the next step of operation. In the field of criminal investigation, craniofacial reconstruction can help policemen to identify the unknown corpse.
However, craniofacial reconstruction is not easy. Traditionally, craniofacial reconstruction, as a professional craft, relies on sculptors’ anatomy knowledge and carving skill, which is a tedious and time consuming process. Furthermore, the final reconstruction is inevitably subjective. With the advancement of digital scanning and geometry processing technology, it becomes possible to emancipate sculptors from the hard work. The basic principle to design an automatic craniofacial reconstruction algorithm is based on the observation that there is an interesting coupling relationship between the skull geometry and the facial geometry. Therefore, the algorithm design includes at least two considerations: (1) which kind of information is able to better encode the skull/facial geometry, and (2) how the statistical model is designed to make the reconstruction as fast/accurate as possible.
Geodesic is an important intrinsic measure in differential geometry. Although it can characterize how the surface is curved, the expensive computational cost restricts its use in a wide range of applications. Balance between accuracy and run-time performance has been a central problem of CG (compute geodesic) for decades (since the early beginning). Many excellent geodesic algorithms are proposed for solving the different problems. In this paper, considering the running speed and the smoothness of geodesics, we use the heat flow algorithm to extract the geodesic distances at a group of radially distributed grid points and further encode the geometry of craniofacial training data into an array-like structure. After that, we predict geodesic grid corresponding to the user-specified skull data based on the partial least squares regression model. Finally, we perform the real craniofacial reconstruction operation by utilizing the geodesic grid and the face statistical model. Experimental results validate the effectiveness of the novel algorithm framework.
- 1.
We use the angle division map to encode the craniofacial geometry, facilitated by the heat flow method that can report the geodesic distances/paths very quickly.
- 2.
We give a more compact geodesic grid regression model. Rather than use tens of thousands of mesh vertices, we sample the geodesic distances at a group of radially distributed grid points, which is totally independent of the mesh resolution. In this way, the computational cost is greatly reduced.
- 3.
Experiments on 213 pairs of craniofacial data were conducted and the experimental results show that our method can reconstruct facial models more accurate than classic craniofacial reconstruction methods.
The sections of this paper are arranged as follows. Section 2 introduces related works about our method. Our algorithm is elaborated in Section 3, which consists of (a) building craniofacial geodesic grid, (b) computing geodesic distance by heat flow method, (c) generating facial geodesic grid of the user-specified skull by partial least squares regression and (d) reconstructing face by utilizing face statistical model. Next, we present experimental results in Section 4. Finally, we conclude this paper in Section 5.
Section snippets
Craniofacial reconstruction
Craniofacial reconstruction methods include knowledge-based craniofacial reconstruction and statistical model-based craniofacial reconstruction. Knowledge-based craniofacial reconstruction [1], [2], [3], [4], [5], [6] depends on learning of soft tissues location and depth. It involves accurate skull registration and professional knowledge. With the development of science and technology, more and more statistical models are proposed for craniofacial reconstruction.
Statistical model-based
Method
In this paper, we encode craniofacial geometry as a geodesic grid, which built on craniofacial training data by heat flow method. The geodesic grid of the face surface corresponding to user-specified skull is established by partial least squares regression model, and then the facial appearance of target skull is estimated utilizing face statistical model. The flow chart of our method is shown by Fig. 1. is the number of training craniofacial data.
Experimental results and analysis
In experiments, we firstly compute geodesic distance, extract geodesic path based on heat flow method for establishing geodesic grid. Then we establish face statistical model and utilize generated geodesic grid for craniofacial reconstruction. PLSR is not utilized for reconstructing face directly, but utilized for generate facial geodesic grid of target skull. The final craniofacial reconstruction results can be obtained by utilizing face statistical model and generated geodesic grid.
Conclusion
In this paper, a craniofacial reconstruction based on heat flow geodesic grid regression model is proposed. We utilize the inherent advantage of geodesic to encode craniofacial geometry automatically. And facilitated by the heat flow method, the calculated geodesic distance, geodesic path for establishing geodesic grid are smoother than calculated by FMM. PLSR is not utilized for reconstructing facial surface directly, but be utilized for generating geodesic grid. With the help of geodesic grid
CRediT authorship contribution statement
Bin Jia: Methodology, Software, Validation, Writing - original draft. Junli Zhao: Methodology, Conceptualization, Investigation, Writing - review & editing. Shiqing Xin: Methodology, Writing - review & editing. Fuqing Duan: Methodology, Supervision, Writing - review & editing. Zhenkuan Pan: . Zhongke Wu: Methodology, Supervision. Jinhua Li: Methodology, Supervision. Mingquan Zhou: Data curation.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgments
The authors gratefully appreciated the anonymous reviewers for all of their helpful comments. This work was supported by the National Key R&D Program of China under Grant No. 2019YFC1521103, the National Natural Science Foundation of China under Grant Nos. 61702293, 11472144, 61772016 61772294, 61572078, the National Statistical Science Research Project (No.2020355) and Development Plan - Major Scientific and Technological Innovation Projects of Shandong Province (No.2019JZZY02010), and the
References (39)
- et al.
Facial soft tissue thickness database for craniofacial reconstruction in the turkish adult population
Forensic Sci Int
(2014) - et al.
Densely calculated facial soft tissue thickness for craniofacial reconstruction in chinese adults
Forensic Sci Int
(2016) - et al.
A spatially-dense regression study of facial form and tissue depth: towards an interactive tool for craniofacial reconstruction
Forensic Sci Int
(2014) - et al.
Construction and analysis of a head ct-scan database for craniofacial reconstruction
Forensic Sci Int
(2009) - et al.
Facial approximation: soft tissue thickness values for caucasian males using cone beam computer tomography
Legal Med
(2019) - et al.
Craniofacial reconstruction using a combined statistical model of face shape and soft tissue depths: methodology and validation
Forensic Sci Int
(2006) - et al.
Bayesian estimation of optimal craniofacial reconstructions
Forensic Sci Int
(2010) - et al.
3d face reconstruction from skull by regression modeling in shape parameter spaces
Neurocomputing
(2015) - et al.
A regional method for craniofacial reconstruction based on coordinate adjustments and a new fusion strategy
Forensic Sci Int
(2016) - et al.
Applying the improved chen and hans algorithm to different versions of shortest path problems on a polyhedral surface
Comput-Aided Des
(2010)
Computing smooth quasi-geodesic distance field (qgdf) with quadratic programming
Comput-Aided Des
Automatic craniofacial registration based on radial curves
Comput Graph
A method for automatic forensic facial reconstruction based on dense statistics of soft tissue thickness
PloS One
Face reconstruction from skull shapes and physical attributes
Joint pattern recognition symposium
Craniofacial reconstruction using gaussian process latent variable models
International conference on computer analysis of images and patterns
Face appearance reconstruction based on a regional statistical craniofacial model (rcsm)
2010 20th international conference on pattern recognition
A hierarchical dense deformable model for 3d face reconstruction from skull
Multim Tools Appl
Probabilistic joint face-skull modelling for facial reconstruction
Proceedings of the IEEE conference on computer vision and pattern recognition
A computerized craniofacial reconstruction method for an unidentified skull based on statistical shape models
Multim Tools Appl
Cited by (5)
CR-GAN: Automatic craniofacial reconstruction for personal identification
2022, Pattern RecognitionCitation Excerpt :Therefore, developing a mathematical model for the assumed relationship between the face and the underlying skull substrate based on CT scan data is the most critical task for computerized CR. In recent decades, there has been increasing interest in using classical machine learning-based methods to perform CR tasks, for example, least squares support vector regression [6], latent root regression [7], and partial least squares regression [8,9]. These approaches represent the 3D geometry of the skull and craniofacial surfaces in two different rough shape parameter spaces, and then fit the correspondence between these two shape parameter spaces using a statistical model.
Facial approximation of a Zaghunluq mummy in Xinjiang, China
2024, Archaeological and Anthropological SciencesAn End-to-End Conditional Generative Adversarial Network Based on Depth Map for 3D Craniofacial Reconstruction
2022, MM 2022 - Proceedings of the 30th ACM International Conference on MultimediaEnhanced head-skull shape learning using statistical modeling and topological features
2022, Medical and Biological Engineering and Computing