Skip to main content
SpringerLink
Log in

3D Morphable Face Models and Their Applications

  • Conference paper
  • First Online:
Book cover Articulated Motion and Deformable Objects (AMDO 2016)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 9756))

Included in the following conference series:

Abstract

3D Morphable Face Models (3DMM) have been used in face recognition for some time now. They can be applied in their own right as a basis for 3D face recognition and analysis involving 3D face data. However their prevalent use over the last decade has been as a versatile tool in 2D face recognition to normalise pose, illumination and expression of 2D face images. A 3DMM has the generative capacity to augment the training and test databases for various 2D face processing related tasks. It can be used to expand the gallery set for pose-invariant face matching. For any 2D face image it can furnish complementary information, in terms of its 3D face shape and texture. It can also aid multiple frame fusion by providing the means of registering a set of 2D images. A key enabling technology for this versatility is 3D face model to 2D face image fitting. In this paper recent developments in 3D face modelling and model fitting will be overviewed, and their merits in the context of diverse applications illustrated on several examples, including pose and illumination invariant face recognition, and 3D face reconstruction from video.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Abiantun, R., Prabhu, U., Savvides, M.: Sparse feature extraction for pose-tolerant face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 36(10), 2061–2073 (2014)

    Article  Google Scholar 

  2. Aldrian, O., Smith, W.A.P.: Inverse rendering of faces with a 3D morphable model. IEEE Trans. Pattern Anal. Mach. Intell. 35(5), 1080–1093 (2013)

    Article  Google Scholar 

  3. Asthana, A., Marks, T.K., Jones, M.J., Tieu, K.H., Rohith, M.: Fully automatic pose-invariant face recognition via 3D pose normalization. In: IEEE International Conference on Computer Vision, (ICCV), pp. 937–944. IEEE (2011)

    Google Scholar 

  4. Blanz, V., Vetter, T.: A morphable model for the synthesis of 3D faces. In: the 26th Annual Conference on Computer Graphics and Interactive Techniques, (SIGGRAPH), pp. 187–194. ACM Press/Addison-Wesley Publishing Co. (1999)

    Google Scholar 

  5. Blanz, V., Vetter, T.: Face recognition based on fitting a 3D morphable model. IEEE Trans. Pattern Anal. Mach. Intell. 25(9), 1063–1074 (2003)

    Article  Google Scholar 

  6. Boom, B.J., Spreeuwers, L.J., Veldhuis, R.N.J.: Subspace-based holistic registration for low-resolution facial images. EURASIP J. Adv. Signal Process. 2010, 1–14 (2010)

    Article  Google Scholar 

  7. Burgos-Artizzu, X., Perona, P., Dollár, P.: Robust face landmark estimation under occlusion. In: IEEE International Conference on Computer Vision, (ICCV), pp. 1513–1520 (2013)

    Google Scholar 

  8. Cao, C., Bradley, D., Zhou, K., Beeler, T.: Real-time high-fidelity facial performance capture. ACM Trans. Graph. 34(4), 46:1–46:9 (2015)

    Article  Google Scholar 

  9. Cao, X., Wei, Y., Wen, F., Sun, J.: Face alignment by explicit shape regression. Int. J. Comput. Vis. 107(2), 177–190 (2014)

    Article  MathSciNet  Google Scholar 

  10. Chan, C., Tahir, M.A., Kittler, J., Pietikäinen, M.: Multiscale local phase quantization for robust component-based face recognition using kernel fusion of multiple descriptors. IEEE Trans. Pattern Anal. Mach. Intell. 35(5), 1164–1177 (2013)

    Article  Google Scholar 

  11. Chu, B., Romdhani, S., Chen, L.: 3D-aided face recognition robust to expression and pose variations. In: IEEE Conference on Computer Vision and Pattern Recognition, (CVPR), pp. 1907–1914 (2014)

    Google Scholar 

  12. Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 23(6), 681–685 (2001)

    Article  Google Scholar 

  13. Cootes, T.F., Taylor, C.J., Cooper, D.H., Graham, J.: Active shape models-their training and application. Comput. Vis. Image Underst. 61(1), 38–59 (1995)

    Article  Google Scholar 

  14. Cristinacce, D., Cootes, T.F.: Feature detection and tracking with constrained local models. In: British Machine Vision Conference, (BMVC), pp. 929–938 (2006)

    Google Scholar 

  15. Dollár, P., Welinder, P., Perona, P.: Cascaded pose regression. In: IEEE Conference on Computer Vision and Pattern Recognition, (CVPR), pp. 1078–1085 (2010)

    Google Scholar 

  16. Egger, B., Schönborn, S., Forster, A., Vetter, T.: Pose normalization for eye gaze estimation and facial attribute description from still images. In: Jiang, X., Hornegger, J., Koch, R. (eds.) GCPR 2014. LNCS, vol. 8753, pp. 317–327. Springer, Heidelberg (2014)

    Google Scholar 

  17. Feng, Z.H., Hu, G., Kittler, J., Christmas, W., Wu, X.J.: Cascaded collaborative regression for robust facial landmark detection trained using a mixture of synthetic and real images with dynamic weighting. IEEE Trans. Image Process. 24(11), 3425–3440 (2015)

    Article  MathSciNet  Google Scholar 

  18. Feng, Z.H., Huber, P., Kittler, J., Christmas, W., Wu, X.J.: Random cascaded-regression copse for robust facial landmark detection. IEEE Signal Process. Lett. 22(1), 76–80 (2015)

    Article  Google Scholar 

  19. Feng, Z.H., Kittler, J., Christmas, W., Wu, X.J., Pfeiffer, S.: Automatic face annotation by multilinear AAM with missing values. In: IEEE International Conference on Pattern Recognition, (ICPR), pp. 2586–2589. IEEE (2012)

    Google Scholar 

  20. Garrido, P., Valgaert, L., Wu, C., Theobalt, C.: Reconstructing detailed dynamic face geometry from monocular video. ACM Trans. Graph. 32(6), 158:1–158:10 (2013)

    Article  Google Scholar 

  21. Gökberk, B., Akarun, L.: Comparative analysis of decision-level fusion algorithms for 3D face recognition. In: IAPR International Conference on Pattern Recognition, (ICPR), pp. 1018–1021 (2006)

    Google Scholar 

  22. Gross, R., Matthews, I., Cohn, J., Kanade, T., Baker, S.: Multi-pie. Image Vis. Comput. 28(5), 807–813 (2010)

    Article  Google Scholar 

  23. Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, Cambridge (2004)

    Book  MATH  Google Scholar 

  24. Hu, G., Chan, C.H., Kittler, J., Christmas, W.: Resolution-aware 3D morphable model. In: British Machine Vision Conference, (BMVC), pp. 1–10 (2012)

    Google Scholar 

  25. Hu, G., Chan, C., Yan, F., Christmas, W.J., Kittler, J.: Robust face recognition by an albedo based 3D morphable model. In: IEEE International Joint Conference on Biometrics, (IJCB), pp. 1–8 (2014)

    Google Scholar 

  26. Hu, G., Mortazavian, P., Kittler, J., Christmas, W.J.: A facial symmetry prior for improved illumination fitting of 3D morphable model. In: International Conference on Biometrics, (ICB), pp. 1–6 (2013)

    Google Scholar 

  27. Huber, P., Feng, Z., Christmas, W., Kittler, J., Rätsch, M.: Fitting 3D morphable models using local features. In: IEEE International Conference on Image Processing, (ICIP) (2015). http://dx.doi.org/10.1109/ICIP.2015.7350989

  28. Huber, P., Hu, G., Tena, R., Mortazavian, P., Koppen, W.P., Christmas, W., Rätsch, M., Kittler, J.: A multiresolution 3D morphable face model and fitting framework. In: International Conference on Computer Vision Theory and Applications (VISAPP) (2016). http://dx.doi.org/10.5220/0005669500790086

  29. Ichim, A.E., Bouaziz, S., Pauly, M.: Dynamic 3D avatar creation from hand-held video input. ACM Trans. Graph. 34(4), 45:1–45:14 (2015)

    Article  Google Scholar 

  30. Jeni, L., Cohn, J., Kanade, T.: Dense 3D face alignment from 2D videos in real-time. In: 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, (FG), vol. 1, pp. 1–8 (2015)

    Google Scholar 

  31. Jobson, D.J., Rahman, Z.U., Woodell, G.A.: Properties and performance of a center/surround retinex. IEEE Trans. Image Process. 6(3), 451–462 (1997)

    Article  Google Scholar 

  32. Kakadiaris, I.A., Passalis, G., Toderici, G., Murtuza, M.N., Lu, Y., Karampatziakis, N., Theoharis, T.: Three-dimensional face recognition in the presence of facial expressions: an annotated deformable model approach. IEEE Trans. Pattern Anal. Mach. Intell. 29(4), 640–649 (2007)

    Article  Google Scholar 

  33. Koppen, W.P., Chan, C., Christmas, W.J., Kittler, J.: An intrinsic coordinate system for 3D face registration. In: IAPR International Conference on Pattern Recognition, (ICPR), pp. 2740–2743 (2012)

    Google Scholar 

  34. Lanitis, A., Taylor, C.J., Cootes, T.F.: Automatic interpretation and coding of face images using flexible models. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 743–756 (1997)

    Article  Google Scholar 

  35. Li, A., Shan, S., Gao, W.: Coupled bias-variance tradeoff for cross-pose face recognition. IEEE Trans. Image Process. 21(1), 305–315 (2012)

    Article  MathSciNet  Google Scholar 

  36. Li, S., Liu, X., Chai, X., Zhang, H., Lao, S., Shan, S.: Morphable displacement field based image matching for face recognition across pose. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part I. LNCS, vol. 7572, pp. 102–115. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  37. Little, A., Hancock, P., DeBruine, L., Jones, B.: Adaptation to antifaces and the perception of correct famous identity in an average face. Frontiers Psychol. 3, 1–9 (2012)

    Article  Google Scholar 

  38. Mortazavian, P., Kittler, J., Christmas, W.: 3D morphable model fitting for low-resolution facial images. In: 5th IAPR International Conference on Biometrics, (ICB), pp. 132–138. IEEE (2012)

    Google Scholar 

  39. Niinuma, K., Han, H., Jain, A.: Automatic multi-view face recognition via 3D model based pose regularization. In: BTAS, pp. 1–8 (2013)

    Google Scholar 

  40. Pan, G., Han, S., Wu, Z., Wang, Y.: 3D face recognition using mapped depth images. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops, (CVPRW), p. 175 (2005)

    Google Scholar 

  41. Paysan, P., Knothe, R., Amberg, B., Romdhani, S., Vetter, T.: A 3D face model for pose and illumination invariant face recognition. In: IEEE International Conference on Advanced Video and Signal based Surveillance (AVSS) (2009)

    Google Scholar 

  42. Pepik, B., Stark, M., Gehler, P., Schiele, B.: Teaching 3D geometry to deformable part models. In: IEEE Conference on Computer Vision and Pattern Recognition, (CVPR), pp. 3362–3369 (2012)

    Google Scholar 

  43. Pishchulin, L., Jain, A., Andriluka, M., Thormahlen, T., Schiele, B.: Articulated people detection and pose estimation: Reshaping the future. In: IEEE Conference on Computer Vision and Pattern Recognition, (CVPR), pp. 3178–3185 (2012)

    Google Scholar 

  44. Prabhu, U., Heo, J., Savvides, M.: Unconstrained pose-invariant face recognition using 3D generic elastic models. IEEE Trans. Pattern Anal. Mach. Intell. 33(10), 1952–1961 (2011)

    Article  Google Scholar 

  45. Ramamoorthi, R., Hanrahan, P.: A signal-processing framework for reflection. ACM Trans. Graph. 23(4), 1004–1042 (2004)

    Article  Google Scholar 

  46. Rätsch, M., Huber, P., Quick, P., Frank, T., Vetter, T.: Wavelet reduced support vector regression for efficient and robust head pose estimation. In: IEEE Ninth Conference on Computer and Robot Vision (CRV), pp. 260–267 (2012). http://dx.doi.org/10.1109/CRV.2012.41

  47. Rodriguez, J.T.: 3D Face Modelling for 2D+3D Face Recognition. Ph.D. thesis, Surrey University, Guildford, UK (2007)

    Google Scholar 

  48. Romdhani, S., Vetter, T.: Efficient, robust and accurate fitting of a 3D morphable model. In: IEEE International Conference on Computer Vision, (ICCV), pp. 59–66. IEEE (2003)

    Google Scholar 

  49. Romdhani, S., Blanz, V., Vetter, T.: Face identification by fitting a 3D morphable model using linear shape and texture error functions. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002, Part IV. LNCS, vol. 2353, pp. 3–19. Springer, Heidelberg (2002)

    Chapter  Google Scholar 

  50. Romdhani, S., Vetter, T.: Estimating 3D shape and texture using pixel intensity, edges, specular highlights, texture constraints and a prior. In: IEEE Conference on Computer Vision and Pattern Recognition, (CVPR), vol. 2, pp. 986–993. IEEE (2005)

    Google Scholar 

  51. Salah, A.A., Akarun, L.: 3D facial feature localization for registration. In: Gunsel, B., Jain, A.K., Tekalp, A.M., Sankur, B. (eds.) MRCS 2006. LNCS, vol. 4105, pp. 338–345. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  52. Sánchez-Escobedo, D., Castelán, M., Smith, W.A.P.: Statistical 3D face shape estimation from occluding contours. Comput. Vis. Image Underst. 142, 111–124 (2016)

    Article  Google Scholar 

  53. Schönborn, S., Forster, A., Egger, B., Vetter, T.: A Monte Carlo strategy to integrate detection and model-based face analysis. In: Weickert, J., Hein, M., Schiele, B. (eds.) GCPR 2013. LNCS, vol. 8142, pp. 101–110. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  54. Shen, J., Zafeiriou, S., Chrysos, G.G., Kossaifi, J., Tzimiropoulos, G., Pantic, M.: The first facial landmark tracking in-the-wild challenge: benchmark and results. In: IEEE International Conference on Computer Vision Workshop, (ICCVW), pp. 1003–1011 (2015)

    Google Scholar 

  55. Sim, T., Baker, S., Bsat, M.: The CMU pose, illumination, and expression database. IEEE Trans. Pattern Anal. Mach. Intell. 25, 1615–1618 (2003)

    Article  Google Scholar 

  56. Tena, J.R., Smith, R.S., Hamouz, M., Kittler, J., Hilton, A., Illingworth, J.: 2D face pose normalisation using a 3D morphable model. In: IEEE Conference on Advanced Video and Signal Based Surveillance (AVSS), pp. 51–56, September 2007

    Google Scholar 

  57. Walker, M., Jiang, F., Vetter, T., Sczesny, S.: Universals and cultural differences in forming personality trait judgments from faces. Soc. Psychol. Pers. Sci. 2, 609–617 (2011)

    Article  Google Scholar 

  58. Walker, M., Vetter, T.: Portraits made to measure: manipulating social judgments about individuals with a statistical face model. J. Vis. 9, 1–13 (2009)

    Article  Google Scholar 

  59. Walker, M., Vetter, T.: Changing the personality of a face: perceived big two and big five personality factors modeled in real photographs. J. Pers. Soc. Psychol. 110, 609–624 (2015). http://dx.doi.org/10.1037/pspp0000064

    Article  Google Scholar 

  60. Wang, Y., Liu, Z., Hua, G., Wen, Z., Zhang, Z., Samaras, D.: Face re-lighting from a single image under harsh lighting conditions. In: IEEE Conference on Computer Vision and Pattern Recognition, (CVPR), pp. 1–8. IEEE (2007)

    Google Scholar 

  61. Xie, X., Zheng, W.S., Lai, J., Yuen, P.C., Suen, C.Y.: Normalization of face illumination based on large-and small-scale features. IEEE Trans. Image Process. 20(7), 1807–1821 (2011)

    Article  MathSciNet  Google Scholar 

  62. Xiong, X., De la Torre, F.: Supervised descent method and its applications to face alignment. In: IEEE Conference on Computer Vision and Pattern Recognition, (CVPR), pp. 532–539 (2013)

    Google Scholar 

  63. Xu, C., Wang, Y., Tan, T., Quan, L.: Automatic 3D face recognition combining global geometric features with local shape variation information. In: IAPR International Conference on Pattern Recognition, (ICPR), pp. 308–313 (2004)

    Google Scholar 

  64. Yan, J., Lei, Z., Yi, D., Li, S.Z.: Learn to combine multiple hypotheses for accurate face alignment. In: IEEE International Conference on Computer Vision Workshops, (ICCVW), pp. 392–396, December 2013

    Google Scholar 

  65. Yi, D., Lei, Z., Li, S.Z.: Towards pose robust face recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, (CVPR), pp. 3539–3545 (2013)

    Google Scholar 

  66. Zhang, L., Samaras, D.: Face recognition from a single training image under arbitrary unknown lighting using spherical harmonics. IEEE Trans. Pattern Anal. Mach. Intell. 28(3), 351–363 (2006)

    Article  Google Scholar 

  67. Zhang, W., Shan, S., Gao, W., Chen, X., Zhang, H.: Local gabor binary pattern histogram sequence (lgbphs): a novel non-statistical model for face representation and recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, (CVPR). vol. 1, pp. 786–791 (2005)

    Google Scholar 

  68. Zhang, X., Gao, Y., Leung, M.K.H.: Recognizing rotated faces from frontal and side views: An approach toward effective use of mugshot databases. IEEE Trans. Inf. Forensics Secur. 3(4), 684–697 (2008)

    Article  Google Scholar 

  69. Zhu, X., Yan, J., Yi, D., Lei, Z., Li, S.Z.: Discriminative 3D morphable model fitting. In: International Conference on Automatic Face and Gesture Recognition (FG) (2015)

    Google Scholar 

  70. Zhu, Z., Luo, P., Wang, X., Tang, X.: Deep learning identity preserving face space. In: International Conference on Computer Vision, (ICCV), vol. 1, p. 2 (2013)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Centre for Vision, Speech and Signal Processing, University of Surrey, Guildford, UK

    Josef Kittler, Patrik Huber, Zhen-Hua Feng & William Christmas

  2. LEAR Team, Inria Grenoble Rhone-Alpes, Grenoble, France

    Guosheng Hu

Authors
  1. Josef Kittler
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Patrik Huber
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhen-Hua Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Guosheng Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. William Christmas
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Josef Kittler .

Editor information

Editors and Affiliations

  1. UIB-Universitat de les Illes Balears, Palma de Mallorca, Spain

    Francisco José Perales

  2. University of Surrey, Guildford, United Kingdom

    Josef Kittler

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Kittler, J., Huber, P., Feng, ZH., Hu, G., Christmas, W. (2016). 3D Morphable Face Models and Their Applications. In: Perales, F., Kittler, J. (eds) Articulated Motion and Deformable Objects. AMDO 2016. Lecture Notes in Computer Science(), vol 9756. Springer, Cham. https://doi.org/10.1007/978-3-319-41778-3_19

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-41778-3_19

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-41777-6

  • Online ISBN: 978-3-319-41778-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation