Elsevier

Image and Vision Computing

Volume 30, Issue 12, December 2012, Pages 1052-1061
Image and Vision Computing

Face verification of age separated images under the influence of internal and external factors

https://doi.org/10.1016/j.imavis.2012.10.003Get rights and content

Abstract

In this paper we study the task of face verification of age-separated images with the presence of various internal and external factors. We propose a hierarchical local binary pattern (HLBP) feature descriptor for robust face representation across age. The effective representation by HLBP across minimal age, illumination, and expression variations combined with its hierarchical computation provides a discriminative representation of the face image. The proposed face descriptor is combined with an AdaBoost classification framework to model the face verification task as a two-class problem. Experimental results on the FG-NET and MORPH aging datasets indicate that the performance of the proposed framework is robust with respect to images of both adults and children. A detailed empirical analysis on the effects of internal (age gap, gender, and ethnicity) and external (pose, expressions, facial hair, and glasses) factors in the face verification performance is also studied. The results indicate that the verification accuracy reduces as the age gap between the image pair increases. A quantitative comparison on the effects of gender on verification performance by both humans and the proposed machine learning approach is provided. The analysis indicate that the cues aid humans in verifying image pairs with large age gaps, while it aids machines for all age gaps. However, the cues mislead humans in the case of images of children and extra-personal pairs with large age gaps. Our analyses indicate that the pose and expression variations affect the performance, despite training with such variations, while facial hair and glasses act as discriminative cues. A study on the effects of ethnicity indicate that non-linear algorithms have insignificant effect in performance with the use of both generalized and individual ethnicity models when compared with linear algorithms.

Highlights

► The verification accuracy reduces as the age gap between the image pair increases for both humans and machines. ► Discriminative cues aid humans in verification of image pairs with large age gaps and for machines for all age gaps. ► Machines perform better than humans when the discriminative cues are provided as well as with minimal training. ► Non-linear discriminant functions perform better than linear ones in case of individual and generalized ethnicity models. ► Pose and Expression affect performance despite training with such variations, while facial hair and eyeglass act as cues.

Introduction

The human face has been an important modality in biometrics, and face recognition has been an active research area for the past several decades. Face recognition is important due to the breadth of applications such as crowd surveillance, security systems, border control, access control to buildings and other secured areas, identifying missing children, law enforcement, verification of duplicate enrollments, etc. Face verification is a challenging task due to the facial appearance changes, which are mainly caused by age progression besides other internal and external factors. The appearance changes of a face are attributed to shape (e.g. weight loss/gain) and texture changes (e.g. wrinkles, scar, etc.), as age progresses. Besides biological factors, factors such as ethnicity, habits, etc., and external factors such as eyeglasses, facial hair, pose and expression changes, etc. often contribute to the physical changes of the face. A detailed survey of contributions from both psychologists and computer scientists can be found in [1], [2].

Current verification systems face challenges due to an inadequate set of images available for a subject across age. This is evident in the case of applications such as identification of missing children, verification of duplicate enrollments, etc. Hence, a verification system should take into account the variations caused by age in order to provide better verification performance on age separated images.

Face verification across age has not been explored much in the past in spite of its importance in real-world applications. A detailed survey of the effects of aging on face verification tasks can be found in [3], [4]. Earlier approaches [5], [6], [7], [8], [9], [10] perform recognition by transforming one image to have the same age as the other, or by transforming both the images to reduce the aging effects. Ramanathan and Chellappa [11] proposed an aging model to perform face verification of images under the age 18. Park and Jain [12] also proposed a 3D aging model to perform recognition across age. The authors use a 3D aging model to perform age transformation of the images.

One of the shortcomings of the above mentioned approaches is that the information about the age of the probe image is required in order to perform the age transformation. This information is usually not available in real-world applications. Also, the accuracy in age transformation relies on the accuracy of the aging model. Such inaccuracies may result in inappropriate age transformations causing instabilities to these approaches. Hence, we propose a discriminative approach to perform face verification across age progression.

Discriminative approaches proposed in the past [13], [14], [15], [16], [17] follow a non-generative approach to perform face verification across age progression. Ramanathan and Chellappa [13] proposed a discriminative approach for face verification across age progression. The authors adapted the probabilistic eigenspace framework and a Bayesian model to learn the differences between intra-personal pairs and extra-personal pairs. Ling et al. [14] also used a discriminative approach for face verification of age separated images. The authors proposed a face representation called gradient orientation pyramid, in which the image gradients are computed hierarchically to represent a face image. SVM based classification framework is then used for classification of the image pairs. Zhang et al. [15] and Wang et al. [17] used variants of LBP for extracting facial features and used them in their classification framework to classify image pairs of the same age. Kumar et al. [18] proposed attribute and simile classifiers which utilize the information from visual cues and perform face categorization in order to perform face verification. Li et al. [19] proposed a Q-stack model to perform face verification across age and head pose variations. Our work differs from the above mentioned approaches in the face representation and the classification framework.

We propose a discriminative approach for the task of face verification of age separated images. A discriminative feature based face representation coupled with a classification framework is proposed. The proposed framework has been applied to two aging databases, which include both internal and external variations in the face images. A detailed analysis on the performance of the proposed approach in comparison with other state-of-the-art approaches under these variations has been discussed in Section 5.

The rest of the paper is organized as follows. The problem formulation and our contributions are discussed in Section 2. The face verification framework is discussed in Section 3. Then, we introduce the hierarchical face representation in Section 4 and also provide a detailed analysis on the hierarchical face representation for the extraction of age invariant patterns. The experimental setup, datasets used, etc. are discussed in Section 5. Experiments to study the effects of internal and external factors are discussed in 6 Effects of internal factors, 7 Effect of external factors, respectively. A detailed analysis on the performance of humans as well as the proposed approach is discussed in detail in Section 8. A statistical analysis on the human verification performance is also provided. Finally, Section 9 presents the conclusions and further discussions on this work.

Section snippets

Face verification framework

Face recognition involves identifying the identity of an individual in the given probe image by comparing it with a gallery of individuals. But, the task of face verification involves identifying whether two images from an image pair belong to the same person or not. This method of verification is suitable for applications such as access control, border control, verification of photo-ID documents, etc. where the validation is performed by verifying the new photo with the old one. Earlier

Classification framework

As in [20], [21], [22], [14], we model the face verification task as a two-class classification problem. Face verification is a multi-class problem, which can be converted to a two-class problem by classifying the image pairs as intra-personal and extra-personal. Given two images Ii and Ij, the task is reduced to classifying this image pair as either intra-personal or extra-personal. A feature vector is obtained by mapping the image pair into a feature space, and is given as follows.x=SIiIj,

Hierarchical face description

Each face is described by constructing an image pyramid from the face image and computing LBP descriptors from each level of the pyramid. The final LBP descriptor is obtained by concatenating the LBP descriptors at each level of the pyramid.

The original LBP operator proposed by Ojala et al. [29] is a simple but very efficient and powerful operator for texture description. The operator labels the pixels of an image by thresholding the n × n neighborhood of each pixel with the value of the center

Datasets

The face verification experiments are conducted on the FG-NET [23] and the MORPH [24] aging databases. Both FG-NET and MORPH are publicly available databases, which include images across ethnicity (mostly Caucasians and African Americans), age, gender, pose, illumination, expression, occlusions, facial hair, etc.

FG-NET includes 1002 images from 82 subjects with an age range of 0–69 years and an average of 12 images per subject. FG-NET includes real-world images from limited number of subjects

Effects of aging

We perform 5-fold cross validation face verification experiments on the FG-NET and the MORPH datasets. The effects of aging (both children and adults) and the effects of age gap between the images in the image pair are analyzed through these experiments. For the face verification experiment, we generated 5800 and 78,735 intra-personal pairs from the FG-NET and the MORPH datasets, respectively. An equal number of extra-personal pairs are randomly generated from images of different subjects. The

Effect of external factors

Besides age related variations, the presence of external factors on the face image of a subject can affect the verification performance. In order to evaluate the effects of these external factors, we conducted face verification experiments on four subsets of the FG-NET database. Each subset includes images with pose, expression, eyeglasses, and facial hair variations, respectively. Two kinds of experiments were conducted in which the classifier is trained with (Experiment1) and without (

Human vs. machine learning evaluation of face verification

Humans are backed with knowledge base allowing them to use and interpret multiple information in recognizing faces. Human perception studies suggest that humans utilize visual cues more than discriminative cues to recognize face images [42]. Hence, the use of such cues in recognizing/verifying face images by humans can provide an insight on the use of these cues in the face verification task. In order to evaluate the performance of humans, with and without the presence of discriminative cue

Conclusion and discussion

In this paper, we studied the problem of face verification with age variations using discriminative methods. Face image is holistically represented using the hierarchical local binary pattern feature descriptor. The LBP provides an effective representation across minimal age variations, illumination, and minimal pose variations, which makes it a suitable descriptor for description of images across age. The spatial information is incorporated by combining the LBP at each level of the Gaussian

References (43)

  • N. Ramanathan et al.

    Computational methods for modeling facial aging: a survey

    J. Vis. Lang. Comput.

    (2009)
  • K. Jonsson et al.

    Support vector machines for face authentication

    Image Vis. Comput.

    (2002)
  • T. Ojala et al.

    A comparative study of texture measures with classification based on feature distributions

    Pattern Recognit.

    (1996)
  • W. Zhao et al.

    Face recognition: a literature survey

    ACM Comput. Surv.

    (2003)
  • A. Lanitis

    A survey of the effects of aging on biometric identity verification

    Int. J. Biometrics

    (2010)
  • J. Zeng, H. Ling, J. Latecki, S. Figzhugh, G.-D. Guo, Analysis of facial images across age progression by humans, ISRN...
  • J. Suo et al.

    Learning long term face aging patterns from partially dense aging databases

  • J. Suo et al.

    A compositional and dynamic model for face aging

    TPAMI

    (2009)
  • J. Suo et al.

    A multi-resolution dynamic model for face aging simulation

    Comput. Vis. Pattern Recognit.

    (2007)
  • A. Lanitis et al.

    Toward automatic simulation of aging effects on face images

    TPAMI

    (2002)
  • M. Tiddeman et al.

    Prototyping and transforming facial texture for perception research

    Comput. Graph. Appl.

    (2001)
  • R. Singh et al.
  • N. Ramanathan et al.

    Modeling age progression in young faces

    Comput. Vis. Pattern Recognit.

    (2006)
  • U. Park et al.

    Face recognition with temporal invariance: a 3d aging model

  • N. Ramanathan et al.

    Face verification across age progression

    IEEE Trans. Image Process.

    (2006)
  • H. Ling et al.

    Face verification across age progression using discriminative methods

    IEEE Trans. Inf. Forensics Secur.

    (2010)
  • G. Zhang et al.

    Boosting local binary pattern (LBP)-based face recognition

  • G.-D. Guo et al.

    Cross-age face recognition on a very large database: the performance versus age intervals and improvement using soft biometric traits

  • X. Wang et al.

    Boosted multi-task learning for face verification with applications to web image and video search

    Comput. Vis. Pattern Recognit.

    (2009)
  • N. Kumar et al.

    Describable visual attributes for face verification and image search

    TPAMI

    (2011)
  • W. Li et al.

    Combination of age and head pose for adult face verification

  • This paper has been recommended for acceptance by Rama Chellappa.

    View full text