Face alignment using texture-constrained active shape models☆
Introduction
Accurate extraction and alignment of faces from images are required in many computer vision and pattern recognition applications. Active Shape Models (ASM) and Active Appearance Models (AAM), proposed by Cootes et al. [4], are two popular shape and appearance models for object localization. They have been developed and improved for years [5], [6], [7], [9].
In ASM, the local appearance model, which represents the local statistics around each landmark, efficiently finds the ‘best’ candidate point for each landmark in searching the image. The solution space is constrained by the properly trained global shape model. By means of modeling of the local features, ASM obtains nice results in shape localization. AAM [2], [3], [10] combines constraints on both shape and texture in its characterization of face appearance. In the context of this paper, texture means the intensity patch contained in the shape after warping to the mean shape [4]. There are two linear mappings assumed for optimization: from appearance variation to texture variation, and from texture variation to position variation. The shape is extracted by minimizing the texture reconstruction error. According to the different optimization criteria, ASM performs more accurately in shape localization while AAM gives a better match to image texture. On the other hand, ASM tends to be stuck in local minima, dependent on the initialization. AAM is sensitive to the illumination, in particular if the lighting in the test is significantly different from the training. Meanwhile, training an AAM model is time consuming.
In this paper, a novel shape model, called Texture-Constrained Active Shape Model (TC-ASM), is proposed to address the above problems of ASM and AAM. TC-ASM inherits the local appearance model in ASM for the robustness of varying lighting. We borrow the global texture in AAM to TC-ASM, acting as a constraint over shape and providing an optimization criterion for determining the shape parameters. In TC-ASM, the conditional distribution of a shape given its associated texture is modeled as a Gaussian distribution. Thus, the texture corresponding to the shape obtained from the local appearance model, could linearly predict a texture-constrained shape. It converges to a local optimum when the shape from the local appearance model is very close to the texture-constrained shape.
Extensive experiments show that TC-ASM outperforms ASM and AAM in facial shape localization. It is also demonstrated that TC-ASM performs no worse than AAM in texture reconstruction.
This paper is organized as follows. In Section 2, we briefly review the shape and appearance models. The details of TC-ASM are discussed in Section 3. Experiments are presented in Section 4. We conclude this paper in Section 5.
Section snippets
Classical shape and appearance models
Assume a training set of shape-texture pairs to be The shape is a sequence of K points in the image lattice. The texture Tis is the image patch enclosed by Si. Let S̄ be the mean shape of all the training shapes, as illustrated in Fig. 1. S̄ is calculated from an iterative procedure [1] such that all the shapes are aligned to the tangent space of the mean shape After shape warping [4], the texture Tis is warped correspondingly to where L is
Texture constrained active shape model
From above analysis, it is natural to develop a novel model to inherit the merits and reject the demerits of ASM and AAM. We propose a TC-ASM to borrow local appearance models from ASM for landmark localization, and incorporate the global texture constraint over the shape from AAM for more accurate shape parameters estimation. It consists of several types of models: a shape model, a texture model, K local appearance models, and a texture-constrained shape model. The former three types are
Experiments
A data set containing 700 face images from about 300 persons with different illumination conditions and expressions are selected from the AR database [8] in our experiments, each of which is a 512×512, 256 gray-levels image containing the frontal view face about 200×200. Eighty-three landmark points are manually labeled on the face. We randomly select 600 for training and the other 100 for testing.
For comparison, ASM and AAM are trained on the same data sets, in a three-level image pyramid
Conclusion
In this paper we proposed a novel shape model, TC-ASM, for face shape localization. TC-ASM efficiently incorporates the local information around each landmark and the global texture information for alignment. It is more robust to initialization, more accurate in shape localization and less sensitive to illumination, when compared with conventional methods. For future work, the generalization of the shape prediction from the texture can be evaluated on a larger data set.
References (10)
- et al.
Active appearance models
Proceedings of the European Conference on Computer Vision
(1998) - et al.
Active appearance models
IEEE Transactions on Pattern Analysis and Machine Intelligence
(2001) - et al.
On representing edge structure for model matching
Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
(2001) - T. Cootes, C. Taylor, Statistical models of appearance for computer vision, http://www.isbe.man.ac.uk/~bim/refs.html,...
- et al.
Proceedings of the Mathematical Methods in Biomedical Image Analysis
(2001)
Cited by (0)
- ☆
The work was performed at Microsoft Research Asia.