Video-based kinship verification using distance metric learning
Introduction
Kinship verification from human faces in a relatively new problem in biometrics in recent years. The key motivation of this research topic is from the research observations in results in psychology and cognitive sciences [1], [2], [3], [4] where human faces convey an important cue for kin similarity measure because children usually look like their parents. Verifying human kinship relationship has several potential applications such as image annotation, family album organization, social media mining, and missing children searching. Over the past few years, a number of kinship verification methods have been proposed in the literature, which aims to present effective computational models to verify human kinship relations via facial image analysis [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], [15], [16], [17]. While these methods have achieved some encouraging performance [5], [6], [7], [8], [9], [10], [11], [12], [13], [15], [18], it is still challenging to develop discriminative and robust kinship verification approached for real-world applications, especially when face images are captured in unconstrained environments where large variations of pose, illumination, expression, and background occurs.
Most existing kinship verification methods determine human kinship relationship from still face images. Due to the large variations of human faces, a single still image may not be discriminative enough to verify human kin relationship. Compared to a single image, a face video provides more information to describe the appearance of human face. It can capture the face of the person of interest from different poses, expressions, and illuminations. Moreover, face videos can be much easier captured in real applications because there are extensive surveillance cameras installed in public areas. Hence, it is desirable to employ face videos to determine the kin relations of persons. However, it is also challenging to exploit discriminative information of face videos because intra-class variations are usually larger within a face video than a single sill image.
In this paper, we investigate the problem of video-based kinship verification via human face analysis. Specifically, we make the two contributions to video-based kinship verification. On one hand, we present a new video face dataset called Kinship Face Videos in the Wild (KFVW) which were captured in wild conditions for the video-based kinship verification study, as well as the standard benchmark. On the other hand, we employ our benchmark to evaluate and compare the performance of several state-of-the-art metric learning based kinship verification methods. Experimental results are presented to demonstrate the efficacy of our proposed dataset and the effectiveness of existing metric learning methods for video-based kinship verification. Lastly, we also test human ability on kinship verification from facial videos and experimental results show that metric learning based computational methods are not as good as that of human observers.
The rest of this paper is organized as follows. In Section 2, we briefly review some related work, and Section 3 introduces the Kinship Face Videos in the Wild (KFVW) dataset. Section 4 presents some popular metric learning methods which have been widely used in kinship verification. Section 5 presents the experimental results and analysis. Finally, Section 6 concludes this paper.
Section snippets
Related work
In this section, we briefly review the related topics to our work: (1) kinship verification, (2) metric learning, and (3) video-based face analysis.
The kinship face videos in the wild dataset
In past few years, several facial datasets have been released to advance the kinship verification problem, e.g., CornellKin [5], UB KinFace [7], IIITD Kinship [18], Family101 [12], KinFaceW-I [13], KinFaceW-II [13], TSKinFace [66], etc. Table 2 provides a summary of existing facial datasets for kinship verification. However, these datasets only consist of still face images, in which each subject usually has a single face image. Due to the large variations of human faces, a single still image
Video-based kinship verification using metric learning
Metric learning involves seeking a suitable distance metric from a training set of data points. Following the evaluation and settings used in Ref. [67], we employ several distance metric learning methods as baseline methods for the video-based kinship verification problem. These metric learning methods include information theoretic metric learning (ITML) [68], side-information based linear discriminant analysis (SILD) [69], KISS metric learning (KISSME) [28], and cosine similarity metric
Experiments
In this section, we evaluated several state-of-the-art metric learning methods for video-based kinship verification on the KFVW dataset, and provided some baseline results on this dataset.
Conclusion
In this paper, we have studied the problem of video-based kinship verification. To our best knowledge, this problem has not been formally addressed in the literature. We have first presented a new video face dataset called Kinship Face Videos in the Wild (KFVW) which were captured in wild conditions for the video-based kinship verification study, as well as the standard benchmark. Then, we have evaluated and compared the performance of several state-of-the-art metric learning based kinship
Acknowledgments
This work was supported in part by the National Natural Science Foundation of China under Grant 61603048, the Beijing Natural Science Foundation under Grant 4174101, and the Fundamental Research Funds for the Central Universities.
Haibin Yan received the B.E. and M.E. degrees from the Xi’an University of Technology, Xi’an, China, in 2004 and 2007, and the Ph.D. degree from the National University of Singapore, Singapore, in 2013, all in mechanical engineering. Now, she is an Assistant Professor in the School of Automation, Beijing University of Posts and Telecommunications, Beijing, China. From October 2013 to July 2015, she was a research fellow at the Department of Mechanical Engineering, National University of
References (73)
- et al.
Kin recognition signals in adult faces
Vision Res.
(2009) - et al.
The FG 2015 kinship verification in the wild evaluation
IEEE International Conference and Workshops on Automatic Face and Gesture Recognition
(2015) - et al.
Gait recognition for human identification based on ica and fuzzy svm through multiple views fusion
Pattern Recognit. Lett.
(2007) - et al.
Where are kin recognition signals in the human face?
J. Vision
(2006) - et al.
Cross-cultural perceptions of facial resemblance between kin
J. Vision
(2009) - et al.
Human ability to detect kinship in strangers’ faces: effects of the degree of relatedness
Proc. R. Soc. B
(2009) - et al.
Towards computational models of kinship verification
IEEE International Conference on Image Processing
(2010) - et al.
Kinship verification from facial images under uncontrolled conditions
ACM International Conference on Multimedia
(2011) - et al.
Understanding kin relationships in a photo
IEEE Trans. Multimedia
(2012) - et al.
Toward kinship verification using visual attributes
International Conference on Pattern Recognition
(2012)
Kinship measurement on salient facial features
IEEE Trans. Instrum. Meas.
Gabor-based gradient orientation pyramid for kinship verification under uncontrolled environments
ACM International Conference on Multimedia
Can faces verify blood-relations?
IEEE International Conference on Biometrics: Theory, Applications and Systems
Kinship classification by modeling facial feature heredity
IEEE International Conference on Image Processing
Neighborhood repulsed metric learning for kinship verification
IEEE Trans. Pattern Anal. Mach. Intell.
Graph-based kinship recognition
International Conference on Pattern Recognition
Discriminative multimetric learning for kinship verification
IEEE Trans. Inf. Forensics Secur.
Kinship verification in the wild: the first kinship verification competition
IEEE International Joint Conference on Biometrics
Self-similarity representation of weber faces for kinship classification
IEEE International Conference on Biometrics: Theory, Applications, and Systems
Like father, like son: facial expression dynamics for kinship verification
IEEE International Conference on Computer Vision
Prototype-based discriminative feature learning for kinship verification
IEEE Trans. Cybern.
Kinship verification through transfer learning
International Joint Conference on Artificial Intelligence
Genealogical face recognition based on ub kinface database
IEEE Conference on Computer Vision and Pattern Recognition Workshops
Large margin multi-metric learning for face and kinship verification in the wild
Asian Conference on Computer Vision
Improved face representation by nonuniform multilevel selection of Gabor convolution features
IEEE Trans. Syst. Man Cybern. Part B
Extended src: undersampled face recognition via intraclass variant dictionary
IEEE Trans. Pattern Anal. Mach. Intell.
Is that you? Metric learning approaches for face identification
IEEE International Conference on Computer Vision
Regularized locality preserving projections and its extensions for face recognition
IEEE Trans. Syst. Man Cybern. Part B
Large scale metric learning from equivalence constraints
IEEE Conference on Computer Vision and Pattern Recognition
Human activity recognition with metric learning
European Conference on Computer Vision
Learning distance metric for regression by semidefinite programming with application to human age estimation
ACM International Conference on Multimedia
Person re-identification by probabilistic relative distance comparison
IEEE Conference on Computer Vision and Pattern Recognition
Pcca: a new approach for distance learning from sparse pairwise constraints
IEEE Conference on Computer Vision and Pattern Recognition
Discriminative deep metric learning for face verification in the wild
IEEE Conference on Computer Vision and Pattern Recognition
Human identity and gender recognition from gait sequences with arbitrary walking directions
IEEE Trans. Inf. Forensics Secur.
Discriminative multimanifold analysis for face recognition from a single training sample per person
IEEE Trans. Pattern Anal. Mach. Intell.
Cited by (0)
Haibin Yan received the B.E. and M.E. degrees from the Xi’an University of Technology, Xi’an, China, in 2004 and 2007, and the Ph.D. degree from the National University of Singapore, Singapore, in 2013, all in mechanical engineering. Now, she is an Assistant Professor in the School of Automation, Beijing University of Posts and Telecommunications, Beijing, China. From October 2013 to July 2015, she was a research fellow at the Department of Mechanical Engineering, National University of Singapore, Singapore. Her research interests include service robotics and computer vision.
Junlin Hu received the B.E. degree from the Xian University of Technology, Xian, China, in 2008, and the M.E. degree from the Beijing Normal University, Beijing, China, in 2012. He is currently pursuing the Ph.D. degree with the School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore. His current research interests include computer vision, pattern recognition, and biometrics.