Unsupervised Lightweight Face 3D Reconstruction From a Single Uncalibrated Image
Authors: 
Yuhang Shi, Hua Jin, Dapeng TaoAuthors Info & Claims
Yuhang Shi, Hua Jin, Dapeng TaoAuthors Info & ClaimsPages 170 - 176
Abstract
Reconstruct 3D model from 2D image is an important task in the field of deep learning, which aims to make computers have the ability to perceive the 3D world like human-beings. In this paper, a lightweight method is proposed for 3D face reconstruction from a single image without any supervision. Specifically, our method employs encoder-decoder architectures to extract depth map, light condition, transformation matrix and albedo from input image. According to the principle of rendering, we can obtain the connection between 2D image coordinates and 3D model vertices coordinates, using the above elements, we can get the projection image of the reconstructed model. Then the reconstruction loss can be used to optimize the network parameters. Experiments show that our method surpasses the previous work in reconstruction speed and the size of model.
References
[1]
Brunton A, Bolkart T, and Wuhrer S. 2014. Multilinear wavelets: A statistical shape space for human faces. In Proc. of ECCV. 297–312.
[2]
Hadi N A, Ibrahim A, and Yahya F. 2013. Centroid Based on Branching Contour Matching for 3D Reconstruction using Beta-spline. Journal of Image and Graphics 1, 3 (2013), 138–142.
[3]
Morales A, Piella G, and Sukno F M. 2021. Survey on 3D face reconstruction from uncalibrated images. Computer Science Review 40 (2021), 100400.
[4]
Cao C, Weng Y, and Zhou S. 2014. Facewarehouse: A 3D facial expression database for visual computing. IEEE Trans. Vis. Comput. Graphics 20, 3 (2014), 413–425.
[5]
Ferrari C, Lisanti G, and Berretti S. 2017. A dictionary learning-based 3D morphable shape model. IEEE Trans. Multimedia 19, 12 (2017), 2666–2679.
[6]
Sariyanidi E, Zampella C J, and Schultz R T. 2020. Inequality-constrained and robust 3D face model fitting. In Proc. of ECCV.
[7]
Rotger Moll G, Moreno-Noguer F, and Lumbreras F. 2019. Detailed 3D face reconstruction from a single RGB image. In Proc. of WSCG. 103–112.
[8]
Dai H, Pears N, and Smith W A P. 2017. A 3D morphable model of Craniofacial Shape and Texture Variation. In Proc. of IEEE ICCV. 3104–3112.
[9]
Jin H, Wang X, and Zhong Z. 2017. Robust 3D face modeling and reconstruction from frontal and side images. Computer Aided Geometric Design 50 (2017), 1–13.
[10]
Jiang Z H, Wu Q, and Chen K. 2019. Disentangled representation learning for 3D face shape. In Proc. of IEEE CVPR. 11957–11966.
[11]
Kato H, Ushiku Y, and Harada T. 2018. Neural 3d mesh renderer. In Proc. of IEEE CVPR. 3907–3916.
[12]
Kemelmacher-Shlizerman I and Seitz S M. 2011. Face reconstruction in the wild. In Proc. of IEEE ICCV. 1746–1753.
[13]
Kemelmacher-Shlizerman I and Basri R. 2011. 3D face reconstruction from a single image using a single reference face shape. IEEE Trans. Pattern Anal. Mach. Intell 33, 2 (2011), 394–405.
[14]
Booth J, Roussos A, and Ponniah A. 2017. Large scale 3D morphable models. Int. J. Comput. Vis 126, 2 (2017), 233–254.
[15]
Woodham R J. 1980. Photometric method for determining surface orientation from multiple images. Opt. Eng 19, 1 (1980), 139–144.
[16]
Lee M and Choi C H. 2014. Real-time facial shape recovery from a single image under general, unknown lighting by rank relaxation. Comput. Vis. Image Underst 120 (2014), 59–69.
[17]
Lüthi M, Gerig T, and Jud C. 2018. Gaussian process morphable models. IEEE Trans. Pattern Anal. Mach. Intell. 40, 8 (2018), 1860–1873.
[18]
Ho L N, Tran A T, and Phung Q. 2021. Toward Realistic Single-View 3D Object Reconstruction with Unsupervised Learning from Multiple Images. In Proc. ICCV. 12600–12610.
[19]
Sinha S N, Ramnath K, and Szeliski R. 2012. Detecting and reconstructing 3d mirror symmetric objects. In Proc. ECCV. 586–600.
[20]
Aldrian O and Smith W A. 2010. A linear approach of 3D face shape and texture recovery using a 3D morphable model. In Proc. of BMVC. 75:1–75:10.
[21]
Dou P, Shah S K, and Kakadiaris I A. 2017. End-to-end 3D face reconstruction with deep neural networks. In Proc. of IEEE CVPR. 5908–5917.
[22]
Huber P, Hu G, and Tena R. 2016. A multiresolution 3D morphable face model and fitting framework. In Proc. of VISAPP. 79–86.
[23]
Paysan P, Knothe R, and Amberg B. 2009. A 3D face model for pose and illumination invariant face recognition. In Proc of IEEE AVSS. 296–301.
[24]
Wu S, Rupprecht C, and Vedaldi A. 2020. Unsupervised learning of probably symmetric deformable 3d objects from images in the wild. In Proc. of IEEE CVPR. 1–10.
[25]
Gerig T, Morel-Forster A, and Blumer C. 2018. Morphable face models - An open framework. In Proc. of IEEE FG. 75–82.
[26]
Li T, Bolkart T, and Black M J. 2017. Learning a model of facial shape and expression from 4D scans. ACM Trans. Graph 36, 6 (2017).
[27]
Blanz V and Vetter T. 1999. A morphable model for the Synthesis of 3D Faces. In Proc. of ACM SIGGRAPH. 187–194.
[28]
Zhu X, Lei Z, and Liu X. 2016. Face alignment across large poses: A 3D solution. In Proc. of IEEE CVPR. 146–155.
[29]
Sato Y and Bao Y. 2018. 3D Face Recognition without Using the Positional Relation of Facial Elements. Journal of Image and Graphics 6, 1 (2018), 33–38.
[30]
Zhou Y, Deng J, and Kotsia I. 2019. Dense 3D face decoding over 2500FPS: Joint texture and shape convolutional mesh decoders. In Proc. of IEEE CVPR. 1097–1106.
[31]
Liu Z, Luo P, and Wang X. 2015. Deep learning face attributes in the wild. In Proc. ICCV. 3730–3738.
- Unsupervised Lightweight Face 3D Reconstruction From a Single Uncalibrated Image
Recommendations
3D Face Reconstruction from a Single Image Using a Single Reference Face Shape
Human faces are remarkably similar in global properties, including size, aspect ratio, and location of main features, but can vary considerably in details across individuals, gender, race, or due to facial expression. We propose a novel method for 3D ...
An Unsupervised Approach for 3D Face Reconstruction from a Single Depth Image
Advances in Computer GraphicsAbstractIn this paper, we propose a convolutional encoder network to learn a mapping function from a noisy depth image to a 3D expressive facial model. We formulate the task as an embedding problem and train the network in an unsupervised manner by ...
3D face reconstruction from a single non-frontal face image
SIGGRAPH '15: ACM SIGGRAPH 2015 PostersA reconstruction of a human face shape from a single image is an important theme for criminal investigation such as recognition of suspected people from surveillance cameras with only a few frames. It is, however, still difficult to recover a face shape ...
Comments
Information & Contributors
Information
Published In
February 2022
253 pages
ISBN:9781450395809
DOI:10.1145/3529570
Copyright © 2022 ACM.
© 2022 Association for Computing Machinery. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of a national government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 29 June 2022
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
- Yunnan Natural Science Funds
- 1. Yunnan provincial major science and technology special plan projects: digitization research and application demonstration of Yunnan characteristic industry
- Key project of basic research plan of Yunnan Province - Research on data-driven intelligent postoperative analgesia scheme
- National Natural Science Foundation of China
Conference
ICDSP 2022
ICDSP 2022: 2022 6th International Conference on Digital Signal Processing
February 25 - 27, 2022
Chengdu, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 66Total Downloads
- Downloads (Last 12 months)12
- Downloads (Last 6 weeks)0
Reflects downloads up to 05 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format