Abstract
Given a person’s neutral face, we can predict his/her unseen expression by machine learning techniques for image processing. Different from the prior expression cloning or image analogy approaches, we try to hallucinate the person’s plausible facial expression with the help of a large face expression database. In the first step, regularization network based nonlinear manifold learning is used to obtain a smooth estimation for unseen facial expression, which is better than the reconstruction results of PCA. In the second step, Markov network is adopted to learn the low-level local facial feature’s relationship between the residual neutral and the expressional face image’s patches in the training set, then belief propagation is employed to infer the expressional residual face image for that person. By integrating the two approaches, we obtain the final results. The experimental results show that the hallucinated facial expression is not only expressive but also close to the ground truth.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Parke FI, Waters K (1996) Computer facial animation. AKPeters, Wellesley Massachusetts
Liu Z, Shan Y, Zhang Z (2001) Expressive expression mapping with ratio images. In: Proceedings of ACM SIGGRAPH, Los Angeles, California, pp 271–276
Noh J, Neumann U (2001) Expression cloning. In: Proceedings of ACM SIGGRAPH, Los Angeles, California, pp 277–288
Zhang Q, Liu Z, Guo B, Shum H (2003) Geometry-driven photorealistic facial expression synthesis. In: Proceedings of the ACM SIGGRAPH/eurographics symposium on computer animation (SCA’03), San Diego, California, pp 177–186
Pyun H, Kim Y, Chae W, Kang HW, Shin SY (2003) An example-based approach for facial expression cloning. In: Proceedings of the ACM SIGGRAPH/eurographics symposium on computer animation (SCA’03), San Diego, California, pp 167–176
Turk M, Pentland A (1991) Eigenfaces for recognition. J Cogn Neurosci 3:(1):71–86
Tenenbaum JB, de Silva V, Langford JC (2000) A global geometric framework for nonlinear dimensionality reduction. Science 290:2319–2323
Wang Q, Xu G, Ai H (2003) Learning object intrinsic structure for robust visual tracking. In: Proceedings of IEEE computer society conference on computer vision and pattern recognition (CVPR’03), (Madison, Wisconsin)
Poggio T, Girosi F (1990) Networks for approximation and learning. Proc IEEE 78(9):1481–1497
Roweis ST, Saul LK (2000) Nonliear dimensionality reduction by local linear embedding. Science 290:2323–2326
Geman S, Geman D (1984) Stochastic relaxation, Gibbs distribution, and the Bayesian restoration of images. IEEE Trans Pattern Anal Mach Intell 6(4):721–741
Li SZ (1995) Markov random field modeling in computer vision. Springer, Berlin Heidelberg New York
Black MJ, Anandan P (1993) A framework for the robust estimation of optical flow. In: Proceedings of the fourth international conference on computer vision (ICCV’93), Berlin, Germany, pp 231–236
Freeman WT, Jones TR, Pasztor EC (2002) Example-based super-resolution. IEEE Comput Graph Appl 22(2):56–65
Greig D, Porteous B, Seheult A (1989) Exact maximum a posteriori estimation for binary images. J R Stat Soc Ser B 51:271–279
Boykov Y, Veksler O, Zabih R (1999) Fast approximate energy minimization via graph cuts. In: Proceedings of the seventh international conference on computer vision (ICCV’99), Kerkyra, Greece, pp 377–384
Kwatra V, Schodl A, Essa I, Bobick A (2003) Graphcut textures: image and video synthesis using graph cuts. In: Proceedings of ACM SIGGRAPH, San Diego, California, pp 277–286
Martinez AM, Benavente R (1998) The AR face database. CVC Technical report no. 24
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Huang, L., Su, C. Facial Expression Synthesis Using Manifold Learning and Belief Propagation. Soft Comput 10, 1193–1200 (2006). https://doi.org/10.1007/s00500-005-0041-7
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-005-0041-7