skip to main content
10.1145/3373509.3373572acmotherconferencesArticle/Chapter ViewAbstractPublication PagesiccprConference Proceedingsconference-collections
research-article

Face Frontalization with Inpainting Method

Published: 25 March 2020 Publication History

Abstract

Problems caused by large profile views are always holdback for face recognition and face reconstruction, such as texture missing problem. In this paper, we propose a method to generate frontal faces from faces of different poses in the wild. Different from previous works for face frontalization which only use GANs or only render a rotated 3D face model to an image, we combine face reconstruction and generative adversarial model, which can preserve the shape information from 3D space and refine the texture through learned distribution from GAN. The task of our GAN is to inpaint the incomplete frontal images which are rendered from the reconstructed 3D model. As the results show, our inpainting method has a better performance than the Poisson editing method on face verification.

References

[1]
.H Larochelle A Larsen, S Sonderby and O Winther. 2016. Autoencoding beyond pixels using learned similarity metric In ICML
[2]
S Chintala A.Radford, L.Metz. 2016. Unsupervised representation learning with deep convolutional generative adversarial networks. In ICLR
[3]
S. Zafeiriou, C. Sagonas, G. Tzimiropoulos and M. Pantic, P. 2013. 300 faces in-the-wild challenge: The first facial landmark localization challenge. In ICCVW
[4]
S. Zhou, Y. Tong, K. Zhou, C. Cao, Y. Wang. 2014. Face Warehouse: A 3D facial expression database for visual computing. IEEE Transactions on Visualization and Computer Graphics,
[5]
S. Mohamed, D. J. Rezende and D. Wierstra. 2014. Stochastic backpropagation and approximate inference in deep generative models. In ICML
[6]
S. Liao D. Yi, Z. Lei and S. Z. Li. 2014. Learning face representation from scratch. arXiv:1411.7923.
[7]
J. G. Daugman. 1985. Certainty relation for resolution in space, spatial frequency and orientation optimized by two-dimension visual cortical filters. JOSA.
[8]
D. Kalenichenko, F. Schroff and J. Philbin. 2015. Facenet: A unified embedding for face recognition and clustering. In CVPR
[9]
T. Berg G. B. Huang, M. Ramesh and E. Learned-Miller. 2007. Labeled faces in the wild: A database for studying face recognition in unconstrained environment. Technical Report 07--49, University of Massachusetts, Amherst.
[10]
I. J. Goodfellow, M. Mirza B. Xu D. W. Farley, S. Ozair, A. Courville, J. P. Abadie and Y. Bengio.2014. Generative Adversarial Nets. In NIPS.
[11]
P. Isola, A. A. Efos, J. Zhu and T. Park. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. In ICCV.
[12]
D. P. Kingma and M. Welling. 2014. Auto-encoding Variational Bayes. In ICLR
[13]
L. Jiang, J. Zhang, B. Deng, H. Li and L. Liu. 2017. 3D Face Reconstruction with Geometry Details from A Single Image. arXiv:1702.0
[14]
X. Liu and L. Tran. 2018. Nonlinear 3D Face Morphable Model. In CVPR
[15]
X. Liu, L. Tran and X.Yin. 2017. Disentangled representation learning GAN for pose-invariant face recognition. In ICCV
[16]
M. Mirza and S. Osindero. 2014. Conditional generative adversarial nets. In NIPS
[17]
Q. Wu, Y. Lai, J. Zheng, J. Zhang and J. Cai. 2018. Alive caricature from 2d to 3d. In CVPR.
[18]
T. Li, R. Huang, S. Zhang and H. Ran. 2017. Beyond face rotation: Global and local perception GAN for photorealistic and identity preserving frontal view synthesis. In ICCV
[19]
L.Hu, K. Nagano, H. Li, S. Satio and L. Wei. 2017 Photorealistic facial texture inference using deep neural networks. In CVPR
[20]
A. Hadid, T. Ahonen and M. Pietikainen. 2006. Face description with local binary patterns: Application to face recognition. TPAMI
[21]
E. Paz T. Hassner, S. Harel and R. Enbar. 2015. Effective Face frontalization in Unconstrained Images. In IEEE Conference on Computer Vision and Pattern Recognition, 4295--4304
[22]
V. Blanz and T. Vetter. 1999. A Morphable Model for the Synthesis of 3D faces. In SIGGRAPH
[23]
Z. Yu, M. Li, B. Raj, W. Liu, Y. Wen and L. Song. 2017. SphereFace: Deep hypersphere embedding for face recognition. In CVPR
[24]
X. Yu, K. Sohn, X. Liu, X, Yin and M. Chandraker. 2017. Towards Large-pose Face Frontalization in the Wild. In ICCV
[25]
K. Sohn, D. N. Metaxas, X. Peng, X. Yu and M. Chandraker. 2017. Reconstruction-based Disentanglement for pose-invariant face recognition. In ICCV
[26]
X. Zhu, J. Yan, D. Yi, Z. Lei and S. Z. Li. 2015. High-fidelity pose and expression normalization for face recognition in the wild. In IEEE Conference on Computer Vision and Pattern Recognition, 787--796.
[27]
Y. Guo, B. Jiang, J. Zhang, J. Zhen and J. Cai. 2017. CNN-based real-time dense face reconstruction with inverse- rendered photo-realistic face images. arXiv:1708.00980v3.
[28]
B. Jiang, Y. Guo, J. Zhang, J. Zheng and J. Cai. 2017. 3Dfacenet: Real-time Dense Face Reconstruction via Synthesizing Photorealistic Face Images. arXiv:1708.00980v2
[29]
W. Ren, Y. Liu, A. Jourabloo and X. Liu. 2017. Dense face alignment. In ICCVW.
[30]
Z. F. Li, Y. Wen, K. P. Zhang and Y. Qiao. 2016. A discriminative feature learning approach for deep face recognition. In ECCV.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
ICCPR '19: Proceedings of the 2019 8th International Conference on Computing and Pattern Recognition
October 2019
522 pages
ISBN:9781450376570
DOI:10.1145/3373509
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

In-Cooperation

  • Hebei University of Technology
  • Beijing University of Technology

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 March 2020

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Face Frontalization
  2. Face Reconstruction
  3. GAN
  4. Inpainting

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Funding Sources

  • National Natural Science Foundation of China

Conference

ICCPR '19

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 76
    Total Downloads
  • Downloads (Last 12 months)4
  • Downloads (Last 6 weeks)0
Reflects downloads up to 20 Jan 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media