Face biometric quality assessment via light CNN

Highlights

• We learnt a robust degradation classifier from noisy labels by employing a light CNN model.

• We considered the recognition confidence in the estimation of face biometric quality.

• The proposed BQA algorithm can be used to improve the reliability of face recognition systems.

Abstract

In this paper, we proposed a novel biometric quality assessment (BQA) method for face images and explored its applications in face recognition. Here, we considered five categories of common homogeneous distortion in video suvillance applications, i.e. low-resolution, blurring, additive Gaussian white noise, salt and pepper noise, and Poisson noise. In the BQA model, we first learnt a classifier to simultaneously predict the categories and degrees of the degradation in a face image. Because the quality labels are often ambiguous and inaccurate, we used a light convolutional neural network with the Max-Feature-Map units to make the BQA model robust to noisy labels. Afterwards, we calculated the biometric quality score by pooling such predictions based on the recognition confidence of each degradation class. Finally, we proposed one promising strategy for developing reliable face recognition systems based on this BQA method. Thorough experiments have been conducted on the CASIA, FLW, and YouTube databases. The results demonstrate the effectiveness of the proposed BQA method.

Introduction

Face recognition system plays a significant role in the public security area. Unfortunately, in practical video surveillance scenarios, face images may suffer from various degradations during the process of capture, compression, and transformations [7]. For example, face images might be of low-resolution, blur, or noisy. Such degradations lead to great challenges in the face recognition system and tremendously decrease the face recognition performance. It is critical to deploy reliable face recognition systems for video surveillance applications [6].

For this purpose, one possible way is using biometric quality assessment (BQA) [4] techniques. The goal of BQA is estimating the ability of an image to function as a biometric. For face recognition systems, BQA algorithms can automatically predict whether a face image is proper for person identification or verification. Besides, BQA algorithms can be used to monitor the video surveillance cameras, to optimize the face image processing algorithms [15], to select “good” images from videos [30], [35], and to design proper face recognition algorithms [26], etc. All these utilizations can improve both the effectiveness and robustness of a face recognition system.

Initially, international researchers and organizations devote great efforts to propose standards of acquiring proper face images for identification and verification [25]. These standards contain a number of instructions for judging the quality of face image based on various factors, e.g. brightness, facial pose, emotion, occlusion, etc. Accordingly, researchers define the biometric quality as the degree one face image departures from the standard frontal face image. They typically use hand-crafted features, e.g. the Local Binary Pattern (LBP) and Scale Invariant Feature Transform (SIFT), to represent a face image and evaluate the biometric quality based on these instructions [3], [12], [21], [38].

Another pipeline of BQA is to evaluate the visual quality of a face image, following the idea of image quality assessment (IQA) [9], [11], [16]. Such methods are adept at estimating visual quality degradations, such as noise, compression artifacts, and blurring [17], [26]. For example, Gunasekaret al. [15] release a face quality assessment database and use the statistics of Discrete Cosine Transform (DCT) coefficients to characterize quality degradations. However, face images are rather specific in term of content. It is critical to explore specific features for developing reliable face BQA models.

Lately, Beveridge et al. [2], Phillips et al. [24] find that, given a face image, its biometric quality is not only dependent on its own visual quality but also highly related to both the test image and the query image. As a result, researchers propose to denote the biometric quality as the matching between the test image and a reference image [1], [21]. In the face recognition system, the neutral frontal face image always achieves better recognition performance than those with variations, it is thus regarded as the reference.

Recently, deep learning techniques, especially Convolutional Neural Networks (CNNs), have lead to great success in solving various image processing problems [10]. Researchers are therefore motivated to learn BQA models using CNNs. For example, Vignesh et al. [30] use the matching score between a given face image and the reference image as the quality index. Specially, they use both the local binary pattern (LBP) and histogram of gradient (HOG) for feature extraction, and use the mutual subspace method (MSM) to calculate the matching score. Finally, they adopt such matching scores as the quality labels and learn a quality assessment model via an eight-layer CNN. More recently, Pan et al. [22] adopt the Deep Face network [23] to extract the feature representation, and use the Probabilistic Linear Discriminant Analysis (PLDA) [5] to estimate the distance between a given image and the reference image. Finally, they learn a quality assessment model via the VGG-16 network [27]. However, the reliability of the generated quality ratings is curious. Besides, the size of quality-labeled face images is relatively small.

In this paper, we proposed a novel BQA method for face images and explored its applications in face recognition. Specially, we considered five common homogeneous distortion categories in video surveillance applications, i.e. low-resolution, blurring, additive Gaussian white noise, salt and pepper noise, and Poisson noise. In the BQA model, we first learnt a classifier to simultaneously predict the categories and degrees of the degradation in a face image. Specially, we used the light convolutional neural network (CNN) with the Max-Feature-Map units, because the quality labels are often ambiguous and inaccurate. Afterwards, we calculated the biometric quality score by pooling such predictions based on the recognition confidence of each degradation class. Here, we evaluated the recognition confidence of one degradation class by statistically measuring its impact on the recognition performance. Finally, we proposed one promising strategy for developing reliable face recognition systems based on this BQA method. Thorough experiments were conducted on the CASIA, FLW, and YouTube databases. The results demonstrate the effectiveness of the proposed BQA method.

Our contributions are briefly four-folds:

• We learnt a robust BQA model from noisy degradation labels by employing the light CNN model with Max-Feature-Map;

• We statistically and numerically considered the relationships between the quality degradation and the face recognition performance in the estimation of face image quality;

• We proposed one potential use of BQA for improving the reliability of face recognition systems; and

• The proposed BQA algorithms is highly consistent with human perception.

The rest of the paper is organized as follows. Section 2 details the framework of the proposed BQA method. Afterwards, the experiment settings and results are presented in Sections 3. Finally, Section 4 concludes this paper with directions for future work.