research-article

Pose Invariant 3D Facial Landmark Detection Via Pose Normalization and Deep Regression

Authors:

Jingchen Zhang,

Daning WangAuthors Info & Claims

IPMV '20: Proceedings of the 2020 2nd International Conference on Image Processing and Machine Vision

Pages 74 - 78

https://doi.org/10.1145/3421558.3421570

Published: 25 November 2020 Publication History

Abstract

Despite the rapid development of detecting facial landmarks on 2D images in the past decade, detecting the landmarks on 3D faces under varying poses is still challenging. Existing methods rely either on pose-invariant feature descriptors, which are vulnerable to missing data (e.g., due to self-occlusion at large poses), or on multi-view landmark models, which are complicated to apply. In this paper, instead, we propose to first normalize the 3D face to frontal pose and then use a single deep model to regress the facial landmarks. We adapt a pose estimation network to estimate the yaw and pitch angles of the input 3D face based on its depth image, and then rotate the face to frontal pose and encode it as a position map. We employ another deep network to predict the location of landmarks on the position map, which is projected back onto the input 3D face to get the final detection results. Evaluation experiments using three public datasets (FRGCv2, UND and Bosphorus) prove the superiority of the proposed method for pose-invariant 3D facial landmark detection.

References

[1]

Mauricio P. Segundo, Chau C. Queirolo, Olga Regina Pereira Bellon, and Luciano Silva, “Automatic 3d facial segmentation and landmark detection,” in ICIAP. IEEE, 2007, pp. 431–436.

[2]

Marcelo Romero and Nick E. Pears, “Landmark localization in 3d face data,” inAVSS.IEEE,2009,pp.73–78.

[3]

Nese Alyüz, Berk Gökberk, and Lale Akarun, “Regional registration for expression resistant 3-d face recognition,” in Trans. Information Forensics and Security. IEEE, 2010, pp. 425–440.

Digital Library

[4]

Xi Zhao, Emmanuel Dellandréa, Liming Chen, and Ioannis A. Kakadiaris, “Accurate landmarking of three dimensional facial data in the presence of facial expressions and occlusions using a three-dimensional statistical facial feature model,” in Trans. Systems, Man, and Cybernetics, Part. IEEE, 2011, pp. 1417–1428.

Digital Library

[5]

Xin Fan, Qi Jia, Kang Huyan, Xianfeng Gu, and Zhongxuan Luo, “3d facial landmark localization using texture regression via conformal mapping,” in Pattern Recognition Letters. ELSEVIER, 2016, pp. 395–402.

Digital Library

[6]

Jinwen Xu, Qijun Zhao, Xiaofeng Li, and Yang Wang, “2.5d cascaded regression for robust facial landmark detection,” in IJCB. IEEE, 2017, pp. 124–132.

[7]

Clement Creusot, Nick E. Pears, and Jim Austin, “A machine-learning approach to keypoint detection and landmarking on 3d meshes,” in IJCV. Springer, 2013, pp. 146–179.

[8]

G. Passalis, P. Perakis, T. Theoharis, and I. A. Kakadiaris, “Using facial symmetry to handle pose variations in real-world 3d face recognition,” inTPAMI.IEEE,2013, pp. 1938–951.

[9]

P. Perakis, G. Passalis, T. Theoharis, and I. Kakadiaris, “3d facial landmark detection under large yaw and expression variations,” in TPAMI. IEEE, 2013, pp. 1552– 564.

Digital Library

[10]

Janez Krizaj, Ziga Emersicy, Simon Dobrisek, Peter Peery, and Vitomir Struc, “Localization of facial landmarks in depth images using gated multiple ridge descent,” in IWOBI. IEEE, 2018, pp. 1–8.

[11]

Marek Kowalski, Jacek Naruniec, and Tomasz Trzcinski, “Deep alignment network: A convolutional neural network for robust face alignment,” in CVPR. IEEE, 2017, pp. 2034–2043.

[12]

Xu Luo, Pengfei Li, Fuxuan Chen, and Qijun Zhao, “Improving large pose face alignment by regressing 2d and 3d landmarks simultaneously and visibility refinement,” in CCBR, 2018, pp. 349–357.

[13]

Nataniel Ruiz, Eunji Chong, and James M.Rehg, “Fine grained head pose estimation without keypoints,” in CVPR. IEEE, 2018, pp. 2074–2083.

[14]

Kangkang Gao, Shan-Ming Yang, Keren Fu, and Peng Cheng, “Deep 3d facial landmark detection on position maps,” in IScIDE, 2019, pp. 299–311.

[15]

Xiangyu Zhu, Zhen Lei, Xiaoming Liu, Hailin Shi, and Stan Z. Li, “Face alignment across large poses: A 3d solution,” in CVPR. IEEE, 2016, pp. 146–155.

[16]

P. Jonathon Phillips, Patrick J. Flynn, W. Todd Scruggs, Kevin W.Bowyer, JinChang, Kevin Hoffman, Joe Marques, Jaesik Min, and William J. Worek, “Overview of the face recognition grand challenge,” in CVPR. IEEE, 2005, pp. 947–954.

[17]

Ping Yan and Kevin W. Bowyer, “Empirical evaluation of advanced ear biometrics,” in CVPR.IEEE,2005, p. 41.

[18]

Arman Savran, Nese Alyüz, Hamdi Dibeklioglu, Oya Çeliktutan, Berk Gökberk, Bülent Sankur, and Lale Akarun, “Bosphorus database for 3d face analysis,” in BIOID, 2008, pp. 47–56.

[19]

Federico M. Sukno, John L. Waddington, and Paul F. Whelan, “3-d facial landmark localization with asymmetry patterns and shape regression from incomplete local features,” in Trans. Cybernetics. IEEE, 2015, pp. 1717–1730.

[20]

Kai Wang, Xi Zhao, Wanshun Gao, and Jianhua Zou, “Acoarse-to-fineapproachfor3dfaciallandmarkingby using deep feature fusion,” in Symmetry, 2018, p. 308.

[21]

M. P. Segundo, C. Queirolo, O. R. P. Bellon, and L. Silva, “Automatic 3d facial segmentation and landmark detection,” in ICIAP. IEEE, 2007, pp. 431–36.

Cited By

Jiao XZhao JLv CDuan FPan ZLi X(2023)Robust 3D Craniofacial Landmarks Localization by An End-to-End Regression Network2023 IEEE International Conference on Multimedia and Expo (ICME)10.1109/ICME55011.2023.00159(900-905)Online publication date: Jul-2023
https://doi.org/10.1109/ICME55011.2023.00159
Mehta ABi RMoamen SAbdelaal MHerndon N(2023)Automatic Detection of Facial Landmarks for Denture ModelsData Management Technologies and Applications10.1007/978-3-031-37890-4_6(114-133)Online publication date: 23-Jul-2023
https://doi.org/10.1007/978-3-031-37890-4_6

Recommendations

Dynamic Weighting of Facial Features for Automatic Pose-Invariant Face Recognition
CRV '12: Proceedings of the 2012 Ninth Conference on Computer and Robot Vision

This paper proposes an automatic pose-invariant face recognition system. In our approach, we consider the texture information around the facial features to compute the similarity measure between the probe and gallery images. The weight of each facial ...
Rough set based pose invariant face recognition with mug shot images

This paper presents a pose invariant face recognition system to annihilate pose problem by using a modified log Gabor algorithm and the concepts of rough sets together. The proposed system consists of four separate modules; face detection, pose ...
Fully automatic pose-invariant face recognition via 3D pose normalization
ICCV '11: Proceedings of the 2011 International Conference on Computer Vision

An ideal approach to the problem of pose-invariant face recognition would handle continuous pose variations, would not be database specific, and would achieve high accuracy without any manual intervention. Most of the existing approaches fail to match ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

IPMV '20: Proceedings of the 2020 2nd International Conference on Image Processing and Machine Vision

August 2020

194 pages

ISBN:9781450388412

DOI:10.1145/3421558

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 November 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

IPMV 2020

IPMV 2020: 2020 2nd International Conference on Image Processing and Machine Vision

August 5 - 7, 2020

Bangkok, Thailand

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
80
Total Downloads

Downloads (Last 12 months)5
Downloads (Last 6 weeks)0

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Jiao XZhao JLv CDuan FPan ZLi X(2023)Robust 3D Craniofacial Landmarks Localization by An End-to-End Regression Network2023 IEEE International Conference on Multimedia and Expo (ICME)10.1109/ICME55011.2023.00159(900-905)Online publication date: Jul-2023
https://doi.org/10.1109/ICME55011.2023.00159
Mehta ABi RMoamen SAbdelaal MHerndon N(2023)Automatic Detection of Facial Landmarks for Denture ModelsData Management Technologies and Applications10.1007/978-3-031-37890-4_6(114-133)Online publication date: 23-Jul-2023
https://doi.org/10.1007/978-3-031-37890-4_6

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten