Face verification of age separated images under the influence of internal and external factors☆
Highlights
► The verification accuracy reduces as the age gap between the image pair increases for both humans and machines. ► Discriminative cues aid humans in verification of image pairs with large age gaps and for machines for all age gaps. ► Machines perform better than humans when the discriminative cues are provided as well as with minimal training. ► Non-linear discriminant functions perform better than linear ones in case of individual and generalized ethnicity models. ► Pose and Expression affect performance despite training with such variations, while facial hair and eyeglass act as cues.
Introduction
The human face has been an important modality in biometrics, and face recognition has been an active research area for the past several decades. Face recognition is important due to the breadth of applications such as crowd surveillance, security systems, border control, access control to buildings and other secured areas, identifying missing children, law enforcement, verification of duplicate enrollments, etc. Face verification is a challenging task due to the facial appearance changes, which are mainly caused by age progression besides other internal and external factors. The appearance changes of a face are attributed to shape (e.g. weight loss/gain) and texture changes (e.g. wrinkles, scar, etc.), as age progresses. Besides biological factors, factors such as ethnicity, habits, etc., and external factors such as eyeglasses, facial hair, pose and expression changes, etc. often contribute to the physical changes of the face. A detailed survey of contributions from both psychologists and computer scientists can be found in [1], [2].
Current verification systems face challenges due to an inadequate set of images available for a subject across age. This is evident in the case of applications such as identification of missing children, verification of duplicate enrollments, etc. Hence, a verification system should take into account the variations caused by age in order to provide better verification performance on age separated images.
Face verification across age has not been explored much in the past in spite of its importance in real-world applications. A detailed survey of the effects of aging on face verification tasks can be found in [3], [4]. Earlier approaches [5], [6], [7], [8], [9], [10] perform recognition by transforming one image to have the same age as the other, or by transforming both the images to reduce the aging effects. Ramanathan and Chellappa [11] proposed an aging model to perform face verification of images under the age 18. Park and Jain [12] also proposed a 3D aging model to perform recognition across age. The authors use a 3D aging model to perform age transformation of the images.
One of the shortcomings of the above mentioned approaches is that the information about the age of the probe image is required in order to perform the age transformation. This information is usually not available in real-world applications. Also, the accuracy in age transformation relies on the accuracy of the aging model. Such inaccuracies may result in inappropriate age transformations causing instabilities to these approaches. Hence, we propose a discriminative approach to perform face verification across age progression.
Discriminative approaches proposed in the past [13], [14], [15], [16], [17] follow a non-generative approach to perform face verification across age progression. Ramanathan and Chellappa [13] proposed a discriminative approach for face verification across age progression. The authors adapted the probabilistic eigenspace framework and a Bayesian model to learn the differences between intra-personal pairs and extra-personal pairs. Ling et al. [14] also used a discriminative approach for face verification of age separated images. The authors proposed a face representation called gradient orientation pyramid, in which the image gradients are computed hierarchically to represent a face image. SVM based classification framework is then used for classification of the image pairs. Zhang et al. [15] and Wang et al. [17] used variants of LBP for extracting facial features and used them in their classification framework to classify image pairs of the same age. Kumar et al. [18] proposed attribute and simile classifiers which utilize the information from visual cues and perform face categorization in order to perform face verification. Li et al. [19] proposed a Q-stack model to perform face verification across age and head pose variations. Our work differs from the above mentioned approaches in the face representation and the classification framework.
We propose a discriminative approach for the task of face verification of age separated images. A discriminative feature based face representation coupled with a classification framework is proposed. The proposed framework has been applied to two aging databases, which include both internal and external variations in the face images. A detailed analysis on the performance of the proposed approach in comparison with other state-of-the-art approaches under these variations has been discussed in Section 5.
The rest of the paper is organized as follows. The problem formulation and our contributions are discussed in Section 2. The face verification framework is discussed in Section 3. Then, we introduce the hierarchical face representation in Section 4 and also provide a detailed analysis on the hierarchical face representation for the extraction of age invariant patterns. The experimental setup, datasets used, etc. are discussed in Section 5. Experiments to study the effects of internal and external factors are discussed in 6 Effects of internal factors, 7 Effect of external factors, respectively. A detailed analysis on the performance of humans as well as the proposed approach is discussed in detail in Section 8. A statistical analysis on the human verification performance is also provided. Finally, Section 9 presents the conclusions and further discussions on this work.
Section snippets
Face verification framework
Face recognition involves identifying the identity of an individual in the given probe image by comparing it with a gallery of individuals. But, the task of face verification involves identifying whether two images from an image pair belong to the same person or not. This method of verification is suitable for applications such as access control, border control, verification of photo-ID documents, etc. where the validation is performed by verifying the new photo with the old one. Earlier
Classification framework
As in [20], [21], [22], [14], we model the face verification task as a two-class classification problem. Face verification is a multi-class problem, which can be converted to a two-class problem by classifying the image pairs as intra-personal and extra-personal. Given two images Ii and Ij, the task is reduced to classifying this image pair as either intra-personal or extra-personal. A feature vector is obtained by mapping the image pair into a feature space, and is given as follows.
Hierarchical face description
Each face is described by constructing an image pyramid from the face image and computing LBP descriptors from each level of the pyramid. The final LBP descriptor is obtained by concatenating the LBP descriptors at each level of the pyramid.
The original LBP operator proposed by Ojala et al. [29] is a simple but very efficient and powerful operator for texture description. The operator labels the pixels of an image by thresholding the n × n neighborhood of each pixel with the value of the center
Datasets
The face verification experiments are conducted on the FG-NET [23] and the MORPH [24] aging databases. Both FG-NET and MORPH are publicly available databases, which include images across ethnicity (mostly Caucasians and African Americans), age, gender, pose, illumination, expression, occlusions, facial hair, etc.
FG-NET includes 1002 images from 82 subjects with an age range of 0–69 years and an average of 12 images per subject. FG-NET includes real-world images from limited number of subjects
Effects of aging
We perform 5-fold cross validation face verification experiments on the FG-NET and the MORPH datasets. The effects of aging (both children and adults) and the effects of age gap between the images in the image pair are analyzed through these experiments. For the face verification experiment, we generated 5800 and 78,735 intra-personal pairs from the FG-NET and the MORPH datasets, respectively. An equal number of extra-personal pairs are randomly generated from images of different subjects. The
Effect of external factors
Besides age related variations, the presence of external factors on the face image of a subject can affect the verification performance. In order to evaluate the effects of these external factors, we conducted face verification experiments on four subsets of the FG-NET database. Each subset includes images with pose, expression, eyeglasses, and facial hair variations, respectively. Two kinds of experiments were conducted in which the classifier is trained with (Experiment1) and without (
Human vs. machine learning evaluation of face verification
Humans are backed with knowledge base allowing them to use and interpret multiple information in recognizing faces. Human perception studies suggest that humans utilize visual cues more than discriminative cues to recognize face images [42]. Hence, the use of such cues in recognizing/verifying face images by humans can provide an insight on the use of these cues in the face verification task. In order to evaluate the performance of humans, with and without the presence of discriminative cue
Conclusion and discussion
In this paper, we studied the problem of face verification with age variations using discriminative methods. Face image is holistically represented using the hierarchical local binary pattern feature descriptor. The LBP provides an effective representation across minimal age variations, illumination, and minimal pose variations, which makes it a suitable descriptor for description of images across age. The spatial information is incorporated by combining the LBP at each level of the Gaussian
References (43)
- et al.
Computational methods for modeling facial aging: a survey
J. Vis. Lang. Comput.
(2009) - et al.
Support vector machines for face authentication
Image Vis. Comput.
(2002) - et al.
A comparative study of texture measures with classification based on feature distributions
Pattern Recognit.
(1996) - et al.
Face recognition: a literature survey
ACM Comput. Surv.
(2003) A survey of the effects of aging on biometric identity verification
Int. J. Biometrics
(2010)- J. Zeng, H. Ling, J. Latecki, S. Figzhugh, G.-D. Guo, Analysis of facial images across age progression by humans, ISRN...
- et al.
Learning long term face aging patterns from partially dense aging databases
- et al.
A compositional and dynamic model for face aging
TPAMI
(2009) - et al.
A multi-resolution dynamic model for face aging simulation
Comput. Vis. Pattern Recognit.
(2007) - et al.
Toward automatic simulation of aging effects on face images
TPAMI
(2002)
Prototyping and transforming facial texture for perception research
Comput. Graph. Appl.
Modeling age progression in young faces
Comput. Vis. Pattern Recognit.
Face recognition with temporal invariance: a 3d aging model
Face verification across age progression
IEEE Trans. Image Process.
Face verification across age progression using discriminative methods
IEEE Trans. Inf. Forensics Secur.
Boosting local binary pattern (LBP)-based face recognition
Cross-age face recognition on a very large database: the performance versus age intervals and improvement using soft biometric traits
Boosted multi-task learning for face verification with applications to web image and video search
Comput. Vis. Pattern Recognit.
Describable visual attributes for face verification and image search
TPAMI
Combination of age and head pose for adult face verification
Cited by (11)
Finger Vein Recognition Using Principle Component Analysis and Adaptive k-Nearest Centroid Neighbour Classifier
2021, International Journal of Integrated EngineeringMachine Learning to Identify Gender via Hair Elements
2019, Proceedings of the International Joint Conference on Neural NetworksThe impact of age and threshold variation on facial recognition algorithm performance using images of children
2018, Proceedings - 2018 International Conference on Biometrics, ICB 2018Invariant feature extraction for facial recognition: A survey of the state-of-The-Art
2018, 2018 Conference on Information Communications Technology and Society, ICTAS 2018 - ProceedingsAge-invariant face recognition based on local K-means ensemble
2016, ICIC Express Letters
- ☆
This paper has been recommended for acceptance by Rama Chellappa.