Abstract
Generating face images from skull images have many applications in fields such as archaeology, anthropology and especially forensics, etc. However, face/skull images generation remain a challenging problem due to the fact that face image and skull image have different characteristics and the data on skull images is also limited. Therefore, we consider this transformation as an unpaired image-to-image translation problem and research the recently popular generative models (GANs) to generate face images from skull images. To this end, we use a novel synthesis framework called U-GAT-IT, a new framework for unsupervised image-to-image translation. This framework use AdaLIN (Adaptive Layer-Instance Normalization), which a new normalization function to focus on more important regions between source and target domains. Furthermore, to visualize the generated face in many other aspects, we use an additional 3D facial generation model called DECA (Detailed Expression Capture and Animation), which is a model for 3D facial reconstruction that is trained to robustly produce a UV displacement map from a low-dimensional latent representation. Experimental results show that the proposed method achieves positive results compared to the current unpaired image-to-image translation models.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abate, A.F., et al.: FACES: 3D FAcial reConstruction from anciEnt Skulls using content based image retrieval. J. Vis. Lang. Comput. 15(5), 373–389 (2004)
Andersson, B., Valfridsson, M.: Digital 3D facial reconstruction based on computed tomography (2005)
Biederman, I., Kalocsai, P.: Neural and psychophysical analysis of object and face recognition. In: Wechsler, H., Phillips, P.J., Bruce, V., Soulié, F.F., Huang, T.S. (eds.) Face Recognition, pp. 3–25. Springer, Heidelberg (1998). https://doi.org/10.1007/978-3-642-72201-1_1
Bińkowski, M., et al.: Demystifying MMD GANs. In: International Conference on Learning Representations (2018)
Buzug, T.M., et al.: Reconstruction of soft facial parts (2005)
Feng, Y., et al.: Learning an animatable detailed 3D face model from in the- wild images. ACM Trans. Graph. (TOG) 40(4), 1–13 (2021)
Grüner, O.: Identification of skulls: a historical review and practical applications. In: Iscan, M.Y., Helmer, R.P. (eds.) Forensic Analysis of the Skull: Craniofacial Analysis, Reconstruction, and Identification. Wiley- Liss, New York (1993)
Huang, X., Belongie, S.: Arbitrary style transfer in real-time with adaptive instance normalization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1501–1510 (2017)
Karras, T., Laine, S., Aila, T.: A style-based generator architecture for generative adversarial networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4401–4410 (2019)
Kim, J., et al.: U-GAT-IT: unsupervised generative attentional networks with adaptive layer-instance normalization for image-to-image translation (2020)
Li, T., et al.: Learning a model of facial shape and expression from 4D scans. ACM Trans. Graph. 36(6), 194–1 (2017)
Mao, X., et al.: Least squares generative adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2794–2802 (2017)
Miyasaka, S., et al.: The computer-aided facial reconstruction system. Forensic Sci. Int. 74(1–2), 155–165 (1995)
Nagpal, S., et al.: On matching skulls to digital face images: a preliminary approach. In: 2017 IEEE International Joint Conference on Biometrics (IJCB), pp. 813–819. IEEE (2017)
Delgado, A.N.: The problematic use of race in facial reconstruction. Sci. Cult. 29(5), 568–593 (2020)
Paoletti, M.E., et al.: Deep learning classifiers for hyperspectral imaging: a review. ISPRS J. Photogramm. Remote. Sens. 158, 279–317 (2019)
Pearson, K.: On the skull and portraits of George Buchanan. Biometrika, 233–256 (1926)
Salimans, T., et al.: Improved techniques for training GANs. Adv. Neural. Inf. Process. Syst. 29, 2234–2242 (2016)
Singh, M., et al.: Learning a shared transform model for skull to digital face image matching. In: 2018 IEEE 9th International Conference on Biometrics Theory, Applications and Systems (BTAS), pp. 1–7. IEEE (2018)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)
Verzé, L.: History of facial reconstruction. Acta Biomed. 80(1), 5–12 (2009)
Wang, L., Sindagi, V., Patel, V.: High-quality facial photo-sketch synthesis using multi-adversarial networks. In: 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), pp. 83–90. IEEE (2018)
Wilkinson, C.: Facial reconstruction-anatomical art or artistic anatomy? J. Anat. 216(2), 235–250 (2010)
Zhu, J.-Y., et al.: Unpaired image-to-image translation using cycle- consistent adversarial networks. In: 2017 IEEE International Conference on Computer Vision (ICCV) (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Vo, D.K., Bui, L.T., Le, T.H. (2023). Face Generation from Skull Photo Using GAN and 3D Face Models. In: Arai, K. (eds) Proceedings of the Future Technologies Conference (FTC) 2022, Volume 1. FTC 2022 2022. Lecture Notes in Networks and Systems, vol 559. Springer, Cham. https://doi.org/10.1007/978-3-031-18461-1_2
Download citation
DOI: https://doi.org/10.1007/978-3-031-18461-1_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-18460-4
Online ISBN: 978-3-031-18461-1
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)