Abstract
Facial expression modeling is central to facial expression recognition and expression synthesis for facial animation. Previous works reported that modeling the facial expression with low-dimensional manifold is more appropriate than using a linear subspace. In this paper, we propose a manifold-based 3D face reconstruction approach to estimating the 3D face model and the associated expression deformation from a single face image. In the training phase, we build a nonlinear 3D expression manifold from a large set of 3D facial expression models to represent the facial shape deformations due to facial expressions. Then a Gaussian mixture model in this manifold is learned to represent the distribution of expression deformation. By combining the merits of morphable neutral face model and the low-dimensional expression manifold, we propose a new algorithm to reconstruct the 3D face geometry as well as the 3D shape deformation from a single face image with expression in an energy minimization framework. Experimental results on CMU-PIE image database and FG-Net video database are shown to validate the effectiveness and accuracy of the proposed algorithm.
Chapter PDF
References
Blanz, V., Vetter, T.: A morphable model for the synthesis of 3d-faces. In: SIGGRAPH (1999)
Blanz, V., Vetter, T.: Face recognition based on fitting a 3d morphable model. PAMI 25(9), 1063–1074 (2003)
Bronstein, A., Bronstein, M., Kimmel, R.: Expression invariant 3d face recognition. AVBPA 2688, 62–70 (2003)
Bronstein, A., Bronstein, M., Kimmel, R.: Three dimensional face recognition. IJCV 64(1), 5–30 (2005)
Wang, Y., Pan, G., Wu, Z.: 3d face recognition in the presence of expression: a guidance-based constraint deformation approach. In: CVPR, pp. 1–7 (2007)
Kakadiaris, I.A., Passalis, G., Toderici, G., Murtuza, M.N., Lu, Y., Karampatziakis, N., Theoharis, T.: Three-dimensional face recognition in the presence of facial expressions: an annotated deformable model approach. PAMI (2007)
Wen, Z., Huang, T.: Capturing subtle facial motions in 3d face tracking. In: ICCV (2003)
Zalewski, L., Gong, S.: Synthesis and recognition of facial expressions in virtual 3d views. FGR (2004)
Pighin, F., Hecker, J., Lischinski, D., Szeliski, R., Salesin, D.H.: Synthesizing realistic facial expressions from photographs. In: SIGGRAPH, pp. 75–84 (1998)
Blanz, V., Basso, C., Poggio, T., Vetter, T.: Reanimating faces in images and video. EG (2003)
Zhang, L., Wang, Y., Wang, S., Samaras, D., Zhang, S., Huang, P.: Image-driven re-targeting and relighting of facial expressions. CGI (2005)
Tenenbaum, J., de Silva, V., Langford, J.: A global geometric framework for nonlinear dimensionality reduction. Science (2000)
Roweis, S., Saul, L.: Nonlinear dimensionality reduction by locally linear embedding. Science (2000)
Roweis, S., Saul, L., Hinton, G.: Global coordination of local linear models. Neural Information Processing Systems 14 14, 889–896 (2001)
Chang, Y., Hu, C., Turk, M.: Manifold of facial expression. In: Proc. IEEE intern. Workshop AMFG (2003)
Chang, Y., Ho, C., Turk, M.: Probabilistic expression analysis on manifolds. In: CVPR (2004)
Hu, C., Chang, Y., Feris, R., Turk, M.: Manifold based analysis of facial expression. In: IEEE Workshop on Face Processing in Video (2004)
Wang, Y., Huang, X., Lee, C.S., Zhang, S., Li, Z.: High resolution acquisition, learning and transfer of dynamic 3-d facial expressions. EG (2004)
Basri, R., Jacobs, D.: Lambertian reflectance and linear subspaces. PAMI 25(2), 218–233 (2003)
Yin, L., Wei, X., Sun, Y., Wang, J., Rosato, M.: A 3d facial expression database for facial behavior research. FG, 211–216 (2006)
Zhang, L., Samaras, D.: Face recognition from a single training image under arbitrary unknown lighting using spherical harmonics. PAMI 28(3), 351–363 (2006)
Bilmes, J., Gentle, A.: Tutorial of the em algorithm and its application to parameter estimation for gaussian mixture and hidden markov models. International Computer Science Institute (1998)
Levenberg, K.: A method for the solution of certain non-linear problems in least squares. Quarterly of Applied Mathematics 2(2), 164–168 (1944)
Sim, T., Baker, S., Bsat, M.: The cmu pose, illumination, and expression database. PAMI, 1615–1618 (2003)
FG-Net database, http://www.mmk.ei.tum.de/waf/fgnet/feedtum.html
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Electronic Supplementary Material
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Wang, SF., Lai, SH. (2008). Estimating 3D Face Model and Facial Deformation from a Single Image Based on Expression Manifold Optimization. In: Forsyth, D., Torr, P., Zisserman, A. (eds) Computer Vision – ECCV 2008. ECCV 2008. Lecture Notes in Computer Science, vol 5302. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-88682-2_45
Download citation
DOI: https://doi.org/10.1007/978-3-540-88682-2_45
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-88681-5
Online ISBN: 978-3-540-88682-2
eBook Packages: Computer ScienceComputer Science (R0)