Skip to main content

Deep 3D Facial Landmark Detection on Position Maps

  • Conference paper
  • First Online:
Intelligence Science and Big Data Engineering. Visual Data Engineering (IScIDE 2019)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 11935))

Abstract

3D facial landmark detection is a crucial step for many computer vision applications, such as 3D facial expression analysis, 3D face recognition, and 3D reconstruction. Pose variations, expression changes and self-occlusion yet make 3D facial landmark detection a very challenging task. In this paper, we propose a novel 3D Face Landmark Localization Network (3DLLN), which is robust to the above challenges. Different from existing methods, the proposed 3DLLN utilizes the position maps as an intermediate representation, from which 3DLLN detects 3D landmark coordinates. Further, we demonstrate the usage of a deep regression architecture to improve the accuracy and robustness of a large number of landmarks. The proposed scheme is evaluated on two public datasets FRGCv2 and BU_3DFE and achieves superior results to state-of-the-arts.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Berretti, S., Amor, B.B., Daoudi, M., Del Bimbo, A.: 3D facial expression recognition using sift descriptors of automatically detected keypoints. Vis. Comput. 27(11), 1021 (2011)

    Article  Google Scholar 

  2. Blanz, V., Vetter, T., et al.: A morphable model for the synthesis of 3D faces. In: SIGGRAPH, vol. 99, pp. 187–194 (1999)

    Google Scholar 

  3. Booth, J., Zafeiriou, S.: Optimal UV spaces for facial morphable model construction. In: 2014 IEEE International Conference on Image Processing (ICIP), pp. 4672–4676. IEEE (2014)

    Google Scholar 

  4. Bulat, A., Tzimiropoulos, G.: Binarized convolutional landmark localizers for human pose estimation and face alignment with limited resources. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3706–3714 (2017)

    Google Scholar 

  5. Bulat, A., Tzimiropoulos, G.: How far are we from solving the 2D & 3D face alignment problem? (and a dataset of 230,000 3D facial landmarks). In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1021–1030 (2017)

    Google Scholar 

  6. Chang, K.I., Bowyer, K.W., Flynn, P.J.: Multiple nose region matching for 3D face recognition under varying facial expression. IEEE Trans. Pattern Anal. Mach. Intell. 28(10), 1695–1700 (2006)

    Article  Google Scholar 

  7. Cihan Camgoz, N., Struc, V., Gokberk, B., Akarun, L., Alp Kindiroglu, A.: Facial landmark localization in depth images using supervised ridge descent. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 136–141 (2015)

    Google Scholar 

  8. Colombo, A., Cusano, C., Schettini, R.: 3D face detection using curvature analysis. Pattern Recogn. 39(3), 444–455 (2006)

    Article  Google Scholar 

  9. Creusot, C., Pears, N., Austin, J.: A machine-learning approach to keypoint detection and landmarking on 3D meshes. Int. J. Comput. Vis. 102(1–3), 146–179 (2013)

    Article  Google Scholar 

  10. Dibeklioğlu, H., Gökberk, B., Akarun, L.: Nasal region-based 3D face recognition under pose and expression variations. In: Tistarelli, M., Nixon, M.S. (eds.) ICB 2009. LNCS, vol. 5558, pp. 309–318. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-01793-3_32

    Chapter  Google Scholar 

  11. Emambakhsh, M., Evans, A.N., Smith, M.: Using nasal curves matching for expression robust 3D nose recognition. In: 2013 IEEE Sixth International Conference on Biometrics: Theory, Applications and Systems (BTAS), pp. 1–8. IEEE (2013)

    Google Scholar 

  12. Fan, X., Jia, Q., Huyan, K., Gu, X., Luo, Z.: 3D facial landmark localization using texture regression via conformal mapping. Pattern Recogn. Lett. 83, 395–402 (2016)

    Article  Google Scholar 

  13. Feng, Y., Wu, F., Shao, X., Wang, Y., Zhou, X.: Joint 3D face reconstruction and dense alignment with position map regression network. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 534–551 (2018)

    Chapter  Google Scholar 

  14. Feng, Z.H., Hu, G., Kittler, J., Christmas, W., Wu, X.J.: Cascaded collaborative regression for robust facial landmark detection trained using a mixture of synthetic and real images with dynamic weighting. IEEE Trans. Image Process. 24(11), 3425–3440 (2015)

    Article  MathSciNet  Google Scholar 

  15. Grewe, C.M., Zachow, S.: Fully automated and highly accurate dense correspondence for facial surfaces. In: Hua, G., Jégou, H. (eds.) ECCV 2016. LNCS, vol. 9914, pp. 552–568. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-48881-3_38

    Chapter  Google Scholar 

  16. Harris, C.G., Stephens, M., et al.: A combined corner and edge detector. In: Alvey Vision Conference, vol. 15, pp. 10–5244. Citeseer (1988)

    Google Scholar 

  17. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  18. Jahanbin, S., Choi, H., Bovik, A.C.: Passive multimodal 2-D+ 3-D face recognition using gabor features and landmark distances. IEEE Trans. Inf. Forensics Secur. 6(4), 1287–1304 (2011)

    Article  Google Scholar 

  19. Križaj, J., Emeršič, Ž., Dobrišek, S., Peer, P., Štruc, V.: Localization of facial landmarks in depth images using gated multiple ridge descent. In: 2018 IEEE International Work Conference on Bioinspired Intelligence (IWOBI), pp. 1–8. IEEE (2018)

    Google Scholar 

  20. Li, H., Huang, D., Morvan, J.M., Wang, Y., Chen, L.: Towards 3D face recognition in the real: a registration-free approach using fine-grained matching of 3D keypoint descriptors. Int. J. Comput. Vis. 113(2), 128–142 (2015)

    Article  MathSciNet  Google Scholar 

  21. Mian, A., Bennamoun, M., Owens, R.: An efficient multimodal 2D–3D hybrid approach to automatic face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 29(11), 1927–1943 (2007)

    Article  Google Scholar 

  22. Newell, A., Yang, K., Deng, J.: Stacked hourglass networks for human pose estimation. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9912, pp. 483–499. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46484-8_29

    Chapter  Google Scholar 

  23. Passalis, G., Perakis, P., Theoharis, T., Kakadiaris, I.A.: Using facial symmetry to handle pose variations in real-world 3D face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 33(10), 1938–1951 (2011)

    Article  Google Scholar 

  24. Paulsen, R.R., Juhl, K.A., Haspang, T.M., Hansen, T., Ganz, M., Einarsson, G.: Multi-view consensus CNN for 3D facial landmark placement. In: Jawahar, C.V., Li, H., Mori, G., Schindler, K. (eds.) ACCV 2018. LNCS, vol. 11361, pp. 706–719. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20887-5_44

    Chapter  Google Scholar 

  25. Perakis, P., Passalis, G., Theoharis, T., Kakadiaris, I.A.: 3D facial landmark detection under large yaw and expression variations. IEEE Trans. Pattern Anal. Mach. Intell. 35(7), 1552–1564 (2012)

    Article  Google Scholar 

  26. Segundo, M.P., Queirolo, C., Bellon, O.R., Silva, L.: Automatic 3D facial segmentation and landmark detection. In: 14th International Conference on Image Analysis and Processing (ICIAP 2007), pp. 431–436. IEEE (2007)

    Google Scholar 

  27. Segundo, M.P., Silva, L., Bellon, O.R.P., Queirolo, C.C.: Automatic face segmentation and facial landmark detection in range images. IEEE Trans. Syst. Man Cybern. Part B (Cybern.) 40(5), 1319–1330 (2010)

    Article  Google Scholar 

  28. Terada, T., Chen, Y.W., Kimura, R.: 3D facial landmark detection using deep convolutional neural networks. In: 2018 14th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD), pp. 390–393. IEEE (2018)

    Google Scholar 

  29. Zhao, X., Dellandrea, E., Chen, L., Kakadiaris, I.A.: Accurate landmarking of three-dimensional facial data in the presence of facial expressions and occlusions using a three-dimensional statistical facial feature model. IEEE Trans. Syst. Man Cybern. Part B (Cybern.) 41(5), 1417–1428 (2011)

    Article  Google Scholar 

  30. Zhu, X., Liu, X., Lei, Z., Li, S.Z.: Face alignment in full pose range: a 3D total solution. IEEE Trans. Pattern Anal. Mach. Intell. 41(1), 78–92 (2019)

    Article  Google Scholar 

  31. Zulqarnain Gilani, S., Shafait, F., Mian, A.: Shape-based automatic detection of a large number of 3D facial landmarks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4639–4648 (2015)

    Google Scholar 

Download references

Acknowledgement

This work was supported by the National Science Foundation of China, No. 61703077, U1833128, the Fundamental Research Funds for the Central Universities, No. YJ201755, and the Sichuan Science and Technology Major Projects (2018GZDZX0029).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Keren Fu .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Gao, K., Yang, S., Fu, K., Cheng, P. (2019). Deep 3D Facial Landmark Detection on Position Maps. In: Cui, Z., Pan, J., Zhang, S., Xiao, L., Yang, J. (eds) Intelligence Science and Big Data Engineering. Visual Data Engineering. IScIDE 2019. Lecture Notes in Computer Science(), vol 11935. Springer, Cham. https://doi.org/10.1007/978-3-030-36189-1_25

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-36189-1_25

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-36188-4

  • Online ISBN: 978-3-030-36189-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics