skip to main content
10.1145/3301506.3301519acmotherconferencesArticle/Chapter ViewAbstractPublication PagesicvipConference Proceedingsconference-collections
research-article

3D Face Reconstruction from Low-Resolution Images with Convolutional Neural Networks

Published: 29 December 2018 Publication History

Abstract

During the past years, convolutional neural networks (CNNs) have widely spread as a powerful tool for tackling a variety of challenges posed in computer vision. Consequently, the trend neither does stop at 3D face reconstruction: Recently, several CNN-based approaches for reconstructing the dense 3D geometry of a face from only a single image have been introduced. However, while all of these methods deal with 3D face reconstruction in the high-resolution (HR) case, reconstruction in low-resolution (LR) surveillance scenarios by means of CNNs has not received any attention so far.
With this work, we address that gap, being the first to propose a CNN architecture specifically tailored to LR 3D face reconstruction: We introduce an end-to-end trainable CNN capable of simultaneously estimating 3D geometry and pose of a face given a single LR image. By coupling our network with a state-of-the-art LR face detector, we build a 3D face reconstruction pipeline ready for integration into real-world applications.
We conduct systematic evaluation on LR versions of the in-the-wild AFLW2000-3D dataset, considering decreasing interocular distances (IODs) down to three pixels. The results show superior performance of the proposed method in the LR domain over state-of-the-art approaches, for both 3D face reconstruction and the closely related face alignment task.

References

[1]
D. Baltieri, R. Vezzani, and R. Cucchiara, "3DPeS: 3D people dataset for surveillance and forensics," in J-HGBU, 2011.
[2]
A. Bulat and G. Tzimiropoulos, "How far are we from solving the 2D & 3D face alignment problem? (and a dataset of 230,000-3D facial landmarks)," in ICCV, 2017.
[3]
A. Bulat and G. Tzimiropoulos. (2017). Super-FAN: Integrated facial landmark localization and super-resolution of real-world low resolution faces in arbitrary poses with GANs. arXiv: 1712.02765 {cs.CV}.
[4]
P. Dou, S. K. Shah, and I. A. Kakadiaris, "End-to-end 3D face reconstruction with deep neural networks," in CVPR, 2017.
[5]
R. Gross, I. Matthews, J. Cohn, T. Kanade, and S. Baker, "Multi-PIE," Image and Vision Computing, 2010.
[6]
C. Herrmann, C. Qu, and J. Beyerer, "Low-resolution video face recognition with face normalization and feature adaptation," in ICSIPA, 2015.
[7]
C. Herrmann, D. Willersinn, and J. Beyerer, "Residual vs. inception vs. classical networks for low-resolution face recognition," in SCIA, 2017.
[8]
G. Hu, "Face analysis using 3D morphable models," PhD thesis, University of Surrey, 2015.
[9]
G. Hu, C. H. Chan, J. Kittler, and W. Christmas, "Resolution-aware 3D morphable model," in BMVC, 2012.
[10]
P. Hu and D. Ramanan, "Finding tiny faces," in CVPR, 2017.
[11]
S. Ioffe and C. Szegedy, "Batch normalization: Accelerating deep network training by reducing internal covariate shift," in ICML, 2015.
[12]
A. S. Jackson, A. Bulat, V. Argyriou, and G. Tzimiropoulos, "Large pose 3D face reconstruction from a single image via direct volumetric CNN regression," in ICCV, 2017.
[13]
Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, and T. Darrell. (2014). Caffe: Convolutional architecture for fast feature embedding. arXiv: 1408.5093 {cs.CV}.
[14]
Y. Jin and C.-S. Bouganis, "Robust multi-image based blind face hallucination," in CVPR, 2015.
[15]
A. Jourabloo and X. Liu, "Large-pose face alignment via CNN-based dense 3D model fitting," in CVPR, 2016.
[16]
D. P. Kingma and J. Ba. (2014). Adam: A method for stochastic optimization. arXiv: 1412.6980 {cs.CV}.
[17]
P. Mortazavian, J. Kittler, and W. Christmas, "3D morphable model fitting for low-resolution facial images," in ICB, 2012.
[18]
C. Qu, C. Herrmann, E. Monari, T. Schuchert, and J. Beyerer, "Robust 3D patch-based face hallucination," in WACV, 2017.
[19]
E. Richardson, M. Sela, R. Or-El, and R. Kimmel, "Learning detailed face reconstruction from a single image," in CVPR, 2017.
[20]
E. Richardson, M. Sela, and R. Kimmel, "3D face reconstruction by learning from synthetic data," in 3DV, 2016.
[21]
K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," in ICLR, 2015.
[22]
C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, "Rethinking the inception architecture for computer vision," in CVPR, 2016.
[23]
Z. Wang, Z. Miao, Q. M. J. Wu, Y. Wan, and Z. Tang, "Low-resolution face recognition: A review," Visual Computer, 2014.
[24]
H. Yang, X. Jia, C. C. Loy, and P. Robinson. (2015). An empirical study of recent face alignment methods. arXiv: 1511.05049 {cs.CV}.
[25]
X. Zhu, Z. Lei, X. Liu, H. Shi, and S. Z. Li, "Face alignment across large poses: A 3D solution," in CVPR, 2016.

Cited By

View all
  • (2023)Automating the Statutory Warning Messages in the Movie using Object Detection Techniques2023 Second International Conference on Electrical, Electronics, Information and Communication Technologies (ICEEICT)10.1109/ICEEICT56924.2023.10157446(1-7)Online publication date: 5-Apr-2023
  • (2020)Analysis of video surveillance images using computer vision in a controlled security environment2020 15th Iberian Conference on Information Systems and Technologies (CISTI)10.23919/CISTI49556.2020.9141068(1-6)Online publication date: Jun-2020
  • (2020)Contour-based 3D Modeling through Joint Embedding of Shapes and ContoursSymposium on Interactive 3D Graphics and Games10.1145/3384382.3384518(1-10)Online publication date: 5-May-2020

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
ICVIP '18: Proceedings of the 2018 2nd International Conference on Video and Image Processing
December 2018
252 pages
ISBN:9781450366137
DOI:10.1145/3301506
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

In-Cooperation

  • Kyoto University: Kyoto University
  • TU: Tianjin University

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 December 2018

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. 3D face reconstruction
  2. CNN
  3. low-resolution

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

ICVIP 2018

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)3
  • Downloads (Last 6 weeks)1
Reflects downloads up to 20 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2023)Automating the Statutory Warning Messages in the Movie using Object Detection Techniques2023 Second International Conference on Electrical, Electronics, Information and Communication Technologies (ICEEICT)10.1109/ICEEICT56924.2023.10157446(1-7)Online publication date: 5-Apr-2023
  • (2020)Analysis of video surveillance images using computer vision in a controlled security environment2020 15th Iberian Conference on Information Systems and Technologies (CISTI)10.23919/CISTI49556.2020.9141068(1-6)Online publication date: Jun-2020
  • (2020)Contour-based 3D Modeling through Joint Embedding of Shapes and ContoursSymposium on Interactive 3D Graphics and Games10.1145/3384382.3384518(1-10)Online publication date: 5-May-2020

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media