Skip to main content
SpringerLink
Log in

3D Face Alignment via Cascade 2D Shape Alignment and Constrained Structure from Motion

  • Conference paper
Advanced Concepts for Intelligent Vision Systems (ACIVS 2009)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 5807))

Abstract

In this paper, we consider fitting a 3D wireframe face model to continuous video sequences for the tasks of simultaneous tracking of rigid head motion and non-rigid facial animation. We propose a two-level integrated model for accurate 3D face alignment. At the low level, the 2D shape is accurately extracted via a regularized shape model relied on a cascaded parameter/constraint prediction and optimization. At the high level, those already accurately aligned points from the low level are used to constrain the projected 3D wireframe alignment. Using a steepest descent approach, the algorithm is able to extract simultaneously the parameters related to the face expression and to the 3D posture. Extensive fitting and tracking experiments demonstrate the feasibility, accuracy and effectiveness of the developed methods. A performance evaluation also shows that the proposed methods can outperform the fitting based on an active appearance model search and can tackle many disadvantages associated with such approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Ekman, P., Friesen, W.: Facial Action Coding System. Consulting Psychologists Press, Palo Alto (1977)

    Google Scholar 

  2. Metaxas, D., Terzopoulos, D.: Shape and nonrigid motion estimation through physics-based synthesis. IEEE Transactions on PAMI 15, 580–591 (1993)

    Article  Google Scholar 

  3. Azarbayejani, A., Pentland, A.: Recursive estimation of motion structure and focal length. IEEE Transactions on PAMI 17, 562–575 (1995)

    Article  Google Scholar 

  4. Sarris, N., Grammalidis, N., Strintzis, M.G.: Fap extraction using three-dimensional motion estimation. IEEE Transactions on CSVT 12, 865–876 (2002)

    Google Scholar 

  5. Lowe, D.G.: Fitting parameterized three-dimensional models to images. IEEE PAMI 13, 441–450 (1991)

    Article  Google Scholar 

  6. Xiao, J., Baker, S., Matthews, I., Kanade, T.: Real-time combined 2d+3d active appearance models. In: Proceedings of the IEEE CVPR, vol. 2, pp. 535–542 (2004)

    Google Scholar 

  7. Dornaika, F., Davoine, F.: On appearance based face and facial action tracking. IEEE Transactions on CSVT 16, 1107–1124 (2006)

    Google Scholar 

  8. Pighin, F., Szeliski, R., Salesin, D.H.: Resynthesizing facial animation through 3d model-based tracking. In: Proceedings of the ICCV, pp. 143–150 (1999)

    Google Scholar 

  9. Blanz, V., Vetter, T.: Face recognition based on fitting a 3d morphable model. IEEE PAMI 25 (2003)

    Article  Google Scholar 

  10. Li, H., Roivainen, P., Forchheimer, R.: 3d motion estimation in model-based facial image coding. IEEE Transactions on PAMI 15, 545–555 (1993)

    Article  Google Scholar 

  11. Li, H., Forchheimer, R.: Two-view facial movement estimation. IEEE Transactions on CSVT 4, 276–287 (1994)

    Google Scholar 

  12. Shi, J., Tomasi, C.: Good features to track. In: CVPR, pp. 593–600 (1994)

    Google Scholar 

  13. Cootes, T.F., Taylor, C.J., Cooper, D.H., Graham, J.: Active shape models - their training and application. CVIU 61, 38–59 (1995)

    Google Scholar 

  14. Hou, Y., Ravyse, I., Sahli, H., Zhang, Y., Zhao, R.: Robust shape-based head tracking. In: Blanc-Talon, J., Philips, W., Popescu, D., Scheunders, P. (eds.) ACIVS 2007. LNCS, vol. 4678, pp. 340–351. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  15. Tomasi, C., Kanade, T.: Shape and motion from image streams under orthography: A factorization method. IJCV 9, 137–154 (1992)

    Article  Google Scholar 

  16. Brand, M., Bhotika, R.: Flexible flow for 3d nonrigid tracking and shape recovery. In: Proceedings of CVPR, vol. 1, pp. 315–322 (2001)

    Google Scholar 

  17. Shan, Y., Liu, Z., Zhang, Z.: Model-based bundle adjustment with application to face modeling. In: Proceedings of the ICCV, pp. 644–651 (2001)

    Google Scholar 

  18. Essa, I.A., Pentland, A.P.: Coding, analysis, interpretation, and recognition of facial expressions. IEEE Transactions on PAMI 19, 757–763 (1997)

    Article  Google Scholar 

  19. Blake, A., Isard, M.: Active Contours. Springer, Heidelberg (1998)

    Book  Google Scholar 

  20. Zhou, Y., Gu, L., Zhang, H.: Bayesian tangent shape model: estimating shape and pose parameters via bayesian inference. In: Proceedings of CVPR, pp. 109–116 (2003)

    Google Scholar 

  21. Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. In: Proceedings of ECCV, vol. 2, pp. 484–498 (1998)

    Google Scholar 

  22. Zhou, Y., Zhang, W., Tang, X., Shum, H.: A bayesian mixture model for multi-view face alignment. In: Proceedings of CVPR, vol. 2, pp. 741–746 (2005)

    Google Scholar 

  23. Gu, L., Kanade, T.: 3d alignment of face in a single image. In: Proceedings of CVPR, pp. 1305–1312 (2006)

    Google Scholar 

  24. Gonzalez-Mora, J., De la Torre, F., Murthi, R., Guil, N., Zapata, E.L.: Bilinear active appearance models. In: ICCV, pp. 1–8 (2007)

    Google Scholar 

  25. Dedeoglu, G., Kanade, T., Baker, S.: The asymmetry of image registration and its application to face tracking. IEEE Transactions on PAMI 29, 807–823 (2007)

    Article  Google Scholar 

  26. Liang, L., Xiao, R., Wen, F., Sun, J.: Face alignment via component-based discriminative search. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part II. LNCS, vol. 5303, pp. 72–85. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  27. Gu, L., Kanade, T.: A generative shape regularization model for robust face alignment. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 413–426. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  28. Ahlberg, J.: Candide-3: An updated parametrized face. Technical Report LiTH-ISY-R-2326, Linköping Univ. (2001)

    Google Scholar 

  29. Ravyse, I., Enescu, V., Sahli, H.: Kernel-based head tracker for videophony. In: ICIP 2005, vol. 3, pp. 1068–1071 (2005)

    Google Scholar 

  30. Dornaika, F., Davoine, F.: Simultaneous facial action tracking and expression recognition using a particle filter. In: Proceedings of ICCV, pp. 1733–1738 (2005)

    Google Scholar 

  31. Goodall, C.: Procrustes methods in the statistical analysis of shape. J. R. Statist. Soc. B., 285–339 (1991)

    Google Scholar 

  32. Ross, D., Lim, J., Lin, R.S., Yang, M.H.: Incremental learning for robust visual tracking. International Journal of Computer Vision 77, 125–141 (2008)

    Article  Google Scholar 

  33. Cascia, M.L., Sclaroff, S., Athitsos, V.: Fast, reliable head tracking under varying illumination: An approach based on robust registration of texture-mapped 3d models. IEEE Trans. PAMI 22 (2000)

    Google Scholar 

  34. Kanade, T., Cohn, J., Tian, Y.L.: Comprehensive database for facial expresison analysis. In: Proceedings of AFGR, pp. 46–53 (2000)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department ETRO, Research Group on Audio Visual Signal Processing (AVSP) Vrije Universiteit Brussel, Pleinlaan 2, 1050, Brussel, Belgium

    Yunshu Hou, Ping Fan, Ilse Ravyse & Hichem Sahli

Authors
  1. Yunshu Hou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ping Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ilse Ravyse
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hichem Sahli
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. DGA/D4S/MRIS, CEP/GIP, 16 bis avenue Prieur de la côte d’or., 94114, Arcueil, France

    Jacques Blanc-Talon

  2. Department of Telecommunication and Information Processing, Ghent University, St.-Pietersnieuwstraat 41, 9000, Gent, Belgium

    Wilfried Philips

  3. CSIRO ICT Centre, Epping, Po Box 76, 1710, Sydney, NSW, Australia

    Dan Popescu

  4. University of Antwerp, Universiteitsplein 1; Building N., 2610, Wilrijk, Belgium

    Paul Scheunders

Rights and permissions

Reprints and permissions

Copyright information

© 2009 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Hou, Y., Fan, P., Ravyse, I., Sahli, H. (2009). 3D Face Alignment via Cascade 2D Shape Alignment and Constrained Structure from Motion. In: Blanc-Talon, J., Philips, W., Popescu, D., Scheunders, P. (eds) Advanced Concepts for Intelligent Vision Systems. ACIVS 2009. Lecture Notes in Computer Science, vol 5807. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-04697-1_51

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-04697-1_51

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-04696-4

  • Online ISBN: 978-3-642-04697-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation