Abstract
Precise facial feature extraction is essential to the high-level face recognition and expression analysis. This paper presents a novel method for the real-time geometric facial feature extraction from live video. In this paper, the input image is viewed as a weighted graph. The segmentation of the pixels corresponding to the edges of facial components of the mouth, eyes, brows, and nose is implemented by means of random walks on the weighted graph. The graph has an 8-connected lattice structure and the weight value associated with each edge reflects the likelihood that a random walker will cross that edge. The random walks simulate an anisotropic diffusion process that filters out the noise while preserving the facial expression pixels. The seeds for the segmentation are obtained from a color and motion detector. The segmented facial pixels are represented with linked lists in the original geometric form and grouped into different parts corresponding to facial components. For the convenience of implementing high-level vision, the geometric description of facial component pixels is further decomposed into shape and registration information. Shape is defined as the geometric information that is invariant under the registration transformation, such as translation, rotation, and isotropic scale. Statistical shape analysis is carried out to capture global facial features where the Procrustes shape distance measure is adopted. A Bayesian approach is used to incorporate high-level prior knowledge of face structure. Experimental results show that the proposed method is capable of real-time extraction of precise geometric facial features from live video. The feature extraction is robust against the illumination changes, scale variation, head rotations, and hand interference.
Similar content being viewed by others
References
Li S Z, Jain A K, eds. Handbook of Face Recognition. Berlin: Springer-Verlag, 2004
Zhao W, Chellappa R, eds. Face Processing: Advanced Modeling and Methods. Amsterdam: Elsevier, 2006
Yang M H, Kreigman D J, Ahuja N. Detecting faces in images: a survey. IEEE Trans Pattern Anal Mach Intell, 2002, 24(1): 34–58
Lin D H, Tang X O. Recognize high resolution faces: from macrocosm to microcosm. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2006), 2006, 2: 1355–1362
Mumford D, Shah J. Boundary detection by minimizing functions. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 1985), 1985. 22–26
Tu Z W, Zhu S C. Image segmentation by data-driven Markov chain Monte Carlo. IEEE Trans Pattern Anal Mach Intell, 2002, 24(5): 657–673
Zhu Z W, Ji Q. Robust real-time face pose and facial expression recovery. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2006), 2006, 1: 681–688
Barbu A, Zhu S C. Generalizing Swendsen-Wang to sampling arbitrary posterior probabilities. IEEE Trans Pattern Anal Mach Intel, 2005, 27(8): 1239–1253
Biggs N. Algebraic potential theory on graphs. Bull. London Math Soc, 1997, 29: 641–682
Boykov Y, Veksler O, Zabih R. Fast approximate energy minimization via graph cuts. IEEE Trans Pattern Anal Mach Intell, 2001, 23(11): 1222–1239
Grady L. Random walks for image segmentation. IEEE Trans Pattern Anal Mach Intell, 2006, 28(11): 1768–1783
Kolmogorov V, Zabih R. What energy functions can be minimized via graph cuts? IEEE Trans Pattern Anal Machine Intell, 2004, 26(2): 147–159
Perona P, Malik J. Scale-space and edge detection using anisotropic diffusion. IEEE Trans Pattern Anal Mach Intell, 1990, 12(7): 629–639
Shi J, Malik J. Normalized cuts and image segmentation. IEEE Trans Pattern Anal Mach Intell, 2000, 22(8): 888–905
Meila M, Shi J. Learning segmentation by random walks. In: Neural Information Processing Systems Conference (NIPS 2000), 2000. 873–879
Cheng B, Wang Y, Zheng N, et al. MRF model and FRAME model-based unsupervised image segmentation. Sci China Ser F-Inf Sci, 2004, 47(6): 697–705
Dryden I L, Mardia K V. Statistical Shape Analysis. New York: John Wiley & Sons, 1998
Olver P J, Tannenbaum A, eds. Mathematical Methods in Computer Vision. Berlin: Springer, 2003
Srivastava A, Joshi S H, Mio W, et al. Statistical shape analysis: clustering, learning, and testing. IEEE Trans Pattern Anal Mach Intell, 2005, 27(4): 590–562
Kent J T, Mardia K V. Shape, Tangent projections and bilateral symmetry. Biometrika, 2001, 88: 469–485
Grenander U. General Pattern Theory. Oxford: Clarendon Press, 1994
Author information
Authors and Affiliations
Corresponding author
Additional information
Supported by the National Natural Science Foundation of China (Grant No. 60672071), the Ministry of Science and Technology (Grant No. 2005CCA04400) and the Ministry of Education (Grant No. NCET-05-0534)
Rights and permissions
About this article
Cite this article
Zhao, J. Live facial feature extraction. Sci. China Ser. F-Inf. Sci. 51, 489–498 (2008). https://doi.org/10.1007/s11432-008-0049-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11432-008-0049-6