Learning deep compact similarity metric for kinship verification from face images

Highlights

• A new DNN is proposed to facilitate fusion of deep embeddings for parent-child data.

• A deep metric learning algorithm is derived to learn a compact kin similarity metric.

• Evaluations show the efficacy of our kinship metric with high verification accuracy.

Abstract

Recent advances in kinship verification have shown that learning an appropriate kinship similarity metric on human faces plays a critical role in this problem. However, most of existing distance metric learning (DML) based solutions rely on linearity assumption of the kinship metric model, and the domain knowledge of large cross-generation discrepancy (e.g., large age span and gender difference between parent and child images) has not been considered in metric learning, leading to degraded performance for genetic similarity measure on human faces. To address these limitations, we propose in this work a new kinship metric learning (KML) method with a coupled deep neural network (DNN) model. KML explicitly models the cross-generation discrepancy inherent on parent-child pairs, and learns a coupled deep similarity metric such that the image pairs with kinship relation are pulled close, while those without kinship relation (but with high appearance similarity) are pushed as far away as possible. Moreover, by imposing the intra-connection diversity and inter-connection consistency over the coupled DNN, we introduce the property of hierarchical compactness into the coupled network to facilitate deep metric learning with limited amount of kinship training data. Empirically, we evaluate our algorithm on several kinship benchmarks against the state-of-the-art DML alternatives, and the results demonstrate the superiority of our method.

Introduction

Recent evidence in psychology has indicated that face appearance is a reliable and critical cue for measure of the genetic similarity between the parent and their children [1], [2], [3]. Motivated by this, researchers from biometrics and computer vision societies have developed some computational models for kinship verification via face images [4], [5], [6], [7]. The objective of this verification problem is to determine whether there exists a kin relationship between a given pair of face images. Potential applications based on such verification technique ranges from social media mining to children adoptions and missing children searching.

While encouraging results have been demonstrated over the past a few years, kinship verification using face images still remains open. On one hand, face images are often captured in wild conditions, and varying illumination, poses and expressions in such scenarios make the verification problem quite challenging. On the other hand, kinship verification aims to investigate the kin relationship between two different visual entities (e.g., father and daughter), and thus the inherent appearance gap of intra-class in kinship verification is generally far larger than that in traditional face recognition [8], [9], [10], [11], [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], [22].

Recent advances in kinship verification have indicated that learning an appropriate similarity metric on human faces plays a critical role in kinship verification. Distance metric learning (DML) methods [23], [24], [25], [26] have been investigated in kinship verification [7], [27], [28] for the purpose of achieving an optimal distance metric rather than a pre-specified one for more robust kin-faces matching. Despite the success of DML-based approaches, existing solutions to kinship verification still suffer from two critical limitations:

(1) They are often proposed to learn a linear distance metric for input space, which is less powerful to capture the nonlinear manifold where the genetic traits inherent on human face lie. Moreover, in existing DML-based solutions the parent-child face images share a common linear transformation for visual matching, and hence the domain prior of large distribution gap between the parent and child has not been taken into account, leading to inaccurate measure of inherent kin similarity on human faces.

(2) While learning a nonlinear distance metric based on the deep neural networks (DNNs) [20], [29] is a straightforward solution to this problem, supervised metric learning with DNN typically requires a large number of labeled training samples, which is extremely expensive to collected in practical kinship verification due to the privacy concerns and involved time and human costs.

To address these issues, we propose in this paper a new kinship metric learning (KML) method for kinship verification from face images with a well-designed DNN architecture. The main contributions of this work are summarized as follows:

(1) We design a coupled DNN, named KinNet, for kinship verification from face images. KinNet explicitly models the cross-generation discrepancy inherent on parent-child pairs, and facilitates deep metric learning with limited amount of labeled kinship data. Particularly, by imposing the diversity regularization and cross-generation consistency regularization on the coupled connections, we introduce the property of hierarchical compactness into the coupled network to improve generalization performance of the kinship metric model.

(2) We develop a new deep metric learning algorithm with the proposed KinNet architecture to learn a deep compact cross-generation similarity metric. The learned similarity metric possesses some desirable properties that help address the limitations of most existing DML-based solutions to kinship verification.

From the information fusion point of view, the parent-child faces input to KML can be regarded as the two-view kin data for kinship verification, and hence KML can be considered as a multi-view metric learning in the deep learning framework. Essentially, our KML implicitly learns to fuse a pair of deep embeddings for robust similarity measure of the parent-child pairs.

On the other hand, by latent variable modeling, an ensemble of latent factors in weight matrices of the KinNet are enforced to be as diverse from one another as possible, such that the learned deep embeddings are compact enough to reduce information redundancy in metric learning. From the ensemble learning point of view, our KML implicitly learns to fuse a set of diverse latent factors in deep metric learning.

(3) We empirically evaluate our method on several benchmark datasets, and the results show that our proposed KML significantly boosts the current state-of-the-art level of kinship verification.

The remainder of this paper is organized as follows. We first briefly review the related work in Section 2 , and Section 3 details the kinship metric learning method with the proposed KinNet architecture. Experimental settings, results and discussions are presented in Section 4 , and Section 5 concludes the paper.