Abstract
3D facial landmark detection is a crucial step for many computer vision applications, such as 3D facial expression analysis, 3D face recognition, and 3D reconstruction. Pose variations, expression changes and self-occlusion yet make 3D facial landmark detection a very challenging task. In this paper, we propose a novel 3D Face Landmark Localization Network (3DLLN), which is robust to the above challenges. Different from existing methods, the proposed 3DLLN utilizes the position maps as an intermediate representation, from which 3DLLN detects 3D landmark coordinates. Further, we demonstrate the usage of a deep regression architecture to improve the accuracy and robustness of a large number of landmarks. The proposed scheme is evaluated on two public datasets FRGCv2 and BU_3DFE and achieves superior results to state-of-the-arts.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Berretti, S., Amor, B.B., Daoudi, M., Del Bimbo, A.: 3D facial expression recognition using sift descriptors of automatically detected keypoints. Vis. Comput. 27(11), 1021 (2011)
Blanz, V., Vetter, T., et al.: A morphable model for the synthesis of 3D faces. In: SIGGRAPH, vol. 99, pp. 187–194 (1999)
Booth, J., Zafeiriou, S.: Optimal UV spaces for facial morphable model construction. In: 2014 IEEE International Conference on Image Processing (ICIP), pp. 4672–4676. IEEE (2014)
Bulat, A., Tzimiropoulos, G.: Binarized convolutional landmark localizers for human pose estimation and face alignment with limited resources. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3706–3714 (2017)
Bulat, A., Tzimiropoulos, G.: How far are we from solving the 2D & 3D face alignment problem? (and a dataset of 230,000 3D facial landmarks). In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1021–1030 (2017)
Chang, K.I., Bowyer, K.W., Flynn, P.J.: Multiple nose region matching for 3D face recognition under varying facial expression. IEEE Trans. Pattern Anal. Mach. Intell. 28(10), 1695–1700 (2006)
Cihan Camgoz, N., Struc, V., Gokberk, B., Akarun, L., Alp Kindiroglu, A.: Facial landmark localization in depth images using supervised ridge descent. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 136–141 (2015)
Colombo, A., Cusano, C., Schettini, R.: 3D face detection using curvature analysis. Pattern Recogn. 39(3), 444–455 (2006)
Creusot, C., Pears, N., Austin, J.: A machine-learning approach to keypoint detection and landmarking on 3D meshes. Int. J. Comput. Vis. 102(1–3), 146–179 (2013)
Dibeklioğlu, H., Gökberk, B., Akarun, L.: Nasal region-based 3D face recognition under pose and expression variations. In: Tistarelli, M., Nixon, M.S. (eds.) ICB 2009. LNCS, vol. 5558, pp. 309–318. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-01793-3_32
Emambakhsh, M., Evans, A.N., Smith, M.: Using nasal curves matching for expression robust 3D nose recognition. In: 2013 IEEE Sixth International Conference on Biometrics: Theory, Applications and Systems (BTAS), pp. 1–8. IEEE (2013)
Fan, X., Jia, Q., Huyan, K., Gu, X., Luo, Z.: 3D facial landmark localization using texture regression via conformal mapping. Pattern Recogn. Lett. 83, 395–402 (2016)
Feng, Y., Wu, F., Shao, X., Wang, Y., Zhou, X.: Joint 3D face reconstruction and dense alignment with position map regression network. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 534–551 (2018)
Feng, Z.H., Hu, G., Kittler, J., Christmas, W., Wu, X.J.: Cascaded collaborative regression for robust facial landmark detection trained using a mixture of synthetic and real images with dynamic weighting. IEEE Trans. Image Process. 24(11), 3425–3440 (2015)
Grewe, C.M., Zachow, S.: Fully automated and highly accurate dense correspondence for facial surfaces. In: Hua, G., Jégou, H. (eds.) ECCV 2016. LNCS, vol. 9914, pp. 552–568. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-48881-3_38
Harris, C.G., Stephens, M., et al.: A combined corner and edge detector. In: Alvey Vision Conference, vol. 15, pp. 10–5244. Citeseer (1988)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Jahanbin, S., Choi, H., Bovik, A.C.: Passive multimodal 2-D+ 3-D face recognition using gabor features and landmark distances. IEEE Trans. Inf. Forensics Secur. 6(4), 1287–1304 (2011)
Križaj, J., Emeršič, Ž., Dobrišek, S., Peer, P., Štruc, V.: Localization of facial landmarks in depth images using gated multiple ridge descent. In: 2018 IEEE International Work Conference on Bioinspired Intelligence (IWOBI), pp. 1–8. IEEE (2018)
Li, H., Huang, D., Morvan, J.M., Wang, Y., Chen, L.: Towards 3D face recognition in the real: a registration-free approach using fine-grained matching of 3D keypoint descriptors. Int. J. Comput. Vis. 113(2), 128–142 (2015)
Mian, A., Bennamoun, M., Owens, R.: An efficient multimodal 2D–3D hybrid approach to automatic face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 29(11), 1927–1943 (2007)
Newell, A., Yang, K., Deng, J.: Stacked hourglass networks for human pose estimation. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9912, pp. 483–499. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46484-8_29
Passalis, G., Perakis, P., Theoharis, T., Kakadiaris, I.A.: Using facial symmetry to handle pose variations in real-world 3D face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 33(10), 1938–1951 (2011)
Paulsen, R.R., Juhl, K.A., Haspang, T.M., Hansen, T., Ganz, M., Einarsson, G.: Multi-view consensus CNN for 3D facial landmark placement. In: Jawahar, C.V., Li, H., Mori, G., Schindler, K. (eds.) ACCV 2018. LNCS, vol. 11361, pp. 706–719. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20887-5_44
Perakis, P., Passalis, G., Theoharis, T., Kakadiaris, I.A.: 3D facial landmark detection under large yaw and expression variations. IEEE Trans. Pattern Anal. Mach. Intell. 35(7), 1552–1564 (2012)
Segundo, M.P., Queirolo, C., Bellon, O.R., Silva, L.: Automatic 3D facial segmentation and landmark detection. In: 14th International Conference on Image Analysis and Processing (ICIAP 2007), pp. 431–436. IEEE (2007)
Segundo, M.P., Silva, L., Bellon, O.R.P., Queirolo, C.C.: Automatic face segmentation and facial landmark detection in range images. IEEE Trans. Syst. Man Cybern. Part B (Cybern.) 40(5), 1319–1330 (2010)
Terada, T., Chen, Y.W., Kimura, R.: 3D facial landmark detection using deep convolutional neural networks. In: 2018 14th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD), pp. 390–393. IEEE (2018)
Zhao, X., Dellandrea, E., Chen, L., Kakadiaris, I.A.: Accurate landmarking of three-dimensional facial data in the presence of facial expressions and occlusions using a three-dimensional statistical facial feature model. IEEE Trans. Syst. Man Cybern. Part B (Cybern.) 41(5), 1417–1428 (2011)
Zhu, X., Liu, X., Lei, Z., Li, S.Z.: Face alignment in full pose range: a 3D total solution. IEEE Trans. Pattern Anal. Mach. Intell. 41(1), 78–92 (2019)
Zulqarnain Gilani, S., Shafait, F., Mian, A.: Shape-based automatic detection of a large number of 3D facial landmarks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4639–4648 (2015)
Acknowledgement
This work was supported by the National Science Foundation of China, No. 61703077, U1833128, the Fundamental Research Funds for the Central Universities, No. YJ201755, and the Sichuan Science and Technology Major Projects (2018GZDZX0029).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Gao, K., Yang, S., Fu, K., Cheng, P. (2019). Deep 3D Facial Landmark Detection on Position Maps. In: Cui, Z., Pan, J., Zhang, S., Xiao, L., Yang, J. (eds) Intelligence Science and Big Data Engineering. Visual Data Engineering. IScIDE 2019. Lecture Notes in Computer Science(), vol 11935. Springer, Cham. https://doi.org/10.1007/978-3-030-36189-1_25
Download citation
DOI: https://doi.org/10.1007/978-3-030-36189-1_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-36188-4
Online ISBN: 978-3-030-36189-1
eBook Packages: Computer ScienceComputer Science (R0)