skip to main content
10.1145/3511176.3511177acmotherconferencesArticle/Chapter ViewAbstractPublication PagesicvipConference Proceedingsconference-collections
research-article

Negative Sample Mining based Deep Feature Learning for Kinship Verification

Published: 12 March 2022 Publication History

Abstract

Kinship verification is an important and challenging task. The aim of it is to find out whether there is a kin relationship between one pair of facial images. However, there are many challenges for kinship verification. One of the most important challenge is the problem of data imbalance. Specifically, if there exists N positive kinship pairs, we can get N(N-1) negative pairs. Obviously, the number of negative kinship pairs is much larger than the number of positive kinship pairs. How to make full use of positive and negative samples for training is a valuable problem. But most of the existing methods just randomly select the same number of negative sample pairs as the positive sample pairs. What is more, different negative pairs contain different discrimination information. Therefore, finding discriminative sample is also an important problem. In this paper, we propose a simple and effective method, named Negative sample Mining based Deep Feature Learning (NMDFL) to solve the above problems. Specifically, different with most existing methods, we sample N positive kinship pairs and T*N negative pairs, where T>1. Then, they are sent to the train net, and their impacts on network updating are adjusted in proportion to the number of positive and negative samples. At the same time, the corresponding weight of each negative sample is dynamically adjusted according to the training results of each time. Experimental results on the KinFaceW-I and KinFaceW-II datasets proves the effectiveness of our method.

References

[1]
L. Ye, L. Zhang, L. Nie, Y. Yan, and D. Rosenblum†, “Fortune teller: Predicting your career path,” pp. 201–207, 2016.
[2]
Y. Guo, H. Dibeklioglu, and L. Maaten, “Graph-based kinship recognition,” in 2014 22nd International Conference on Pattern Recognition (ICPR), 2014.
[3]
J. Hu, J. Lu, J. Yuan, and Y. P. Tan, “Large margin multi-metric learning for face and kinship verification in the wild,” Springer, Cham, 2014.
[4]
X. Zhou, J. Lu, J. Hu, and Y. Shang, “Gabor-based gradient orientation pyramid for kinship verification under uncontrolled environments,” ACM, 2012.
[5]
H. Yan, J. Lu, W. Deng, and X. Zhou, “Discriminative multimetric learning for kinship verification,” IEEE Transactions on Information Forensics and Security, vol. 9, no. 7, pp. 1169–1178, 2014.
[6]
R. Fang, K. D. Tang, N. Snavely, and T. Chen, “Towards computational models of kinship verification,” in Proceedings of the International Conference on Image Processing, ICIP 2010, September 26-29, Hong Kong, China, 2010.
[7]
M. Shao, S. Xia, and Y. Fu, “Genealogical face recognition based on ub kinface database,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2011, pp. 60–65.
[8]
Q. Liu, A. Puthenputhussery, and C. Liu, “Inheritable fisher vector feature for kinship verification,” in IEEE International Conference on Biometrics Theory, Applications and Systems (BTAS), 2015, pp. 1–6.
[9]
A. Tidjani, A. Taleb-Ahmed, D. Samai, and K. E. Aiadi, “Deep learning features for robust facial kinship verification,” IET Image Process., vol. 12, no. 12, pp. 2336–2345, 2018.
[10]
Q. Duan, L. Zhang, and W. Jia, “Adv-kin: An adversarial convolutional network for kinship verification,” in Biometric Recognition - 12th Chinese Conference (CCBR), vol. 10568, 2017, pp. 48–57.
[11]
N. Kohli, M. Vatsa, R. Singh, A. Noore, and A. Majumdar, “Hierarchical representation learning for kinship verification,” IEEE Transactions on Image Processing, vol. 26, no. 1, pp. 289–302, 2017.
[12]
K. Zhang, Y. Huang, C. Song, H. Wu, and L. Wang, “Kinship verification with deep convolutional neural networks,” in British Machine Vision Conference (BMVC), 2015, pp. 148.1–148.12.
[13]
H. Yan and C. Song, “Multi-scale deep relational reasoning for facial kinship verification,” Pattern Recognit., vol. 110, p. 107541, 2021.
[14]
H. Yan, J. Lu, and X. Zhou, “Prototype-based discriminative feature learning for kinship verification,” IEEE Transactions on Cybernetics, vol. 45, no. 11, pp. 2535–2545, 2015.
[15]
H. Yan, “Learning discriminative compact binary face descriptor for kinship verification,” Pattern Recognition Letters, vol. 117, pp. 146–152, 2019.
[16]
J. Lu, X. Zhou, Y. Tan, Y. Shang, and J. Zhou, “Neighborhood repulsed metric learning for kinship verification,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 36, no. 2, pp. 331–345, 2014.
[17]
H. Yan, J. Lu, W. Deng, and X. Zhou, “Discriminative multimetric learning for kinship verification,” IEEE Transactions on Information Forensics and Security, vol. 9, no. 7, pp. 1169–1178, 2014.
[18]
M. Wang, Z. Li, X. Shu, J. Wang, and J. Tang, “Deep kinship verification,” in IEEE International Workshop on Multimedia Signal Processing (MMSP), 2015, pp. 1–6
[19]
A. Dehghan, E. G. Ortiz, R. Villegas, and M. Shah, “Who do I look like? determining parent-offspring resemblance via gated autoencoders,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014, pp. 1757–1764.
[20]
J. Liang, Q. Hu, C. Dang, and W. Zuo, “Weighted graph embedding- based metric learning for kinship verification,” IEEE Transactions on Image Processing, vol. 28, no. 3, pp. 1149–1162, 2019.
[21]
A. Sharma, A. Kumar, H. D. III, and D. W. Jacobs, “Generalized multiview analysis: A discriminative latent space,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2012, pp. 2160– 2167.
[22]
L. Li, X. Feng, X. Wu, Z. Xia, and A. Hadid, “Kinship verification from faces via similarity metric based convolutional neural network,” in Image Analysis and Recognition International Conference (ICIAR), vol. 9730, 2016, pp. 539–548.
[23]
M. Dawson, A. Zisserman, and C. Nella°ker, “From same photo: Cheating on visual kinship challenges,” in Asian Conference on Computer Vision (ACCV), vol. 11363, 2018, pp. 654–668.
[24]
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
[25]
A. Sellam and A. Hamid, “Linear feature learning for kinship verification in the wild,” in International Conference on Applied Smart Systems (ICASS), 2018.
[26]
J. Lu, J. Hu, and Y. Tan, “Discriminative deep metric learning for face and kinship verification,” IEEE Transactions on Image Processing, vol. 26, no. 9, pp. 4269–4282, 2017.
[27]
L. Zhang, Q. Duan, D. Zhang, W. Jia, and X. Wang, “Advkin: Adversarial convolutional network for kinship verification,” IEEE Transactions on Cybernetics, pp. 1–14, 2020.
[28]
Diego Lelis and Dibio L. Borges, "Facial Kinship Verification with Large Age Variation Using Deep Linear Metric Learning," Journal of Image and Graphics, vol. 7, no. 2, pp. 50-58, 2019.
[29]
Samuel Lukas, Aditya Rama Mitra, Ririn Ikana Desanti, and Dion Krisnadi, "Implementing Discrete Wavelet and Discrete Cosine Transform with Radial Basis Function Neural Network in Facial Image Recognition," Journal of Image and Graphics, vol. 4, no. 1, pp. 6-10, 2016.
[30]
Humayra B. Ali and David M W Powers, "Fusion Based FastICA Method: Facial Expression Recognition," Journal of Image and Graphics, vol. 2, no. 1, pp. 1-7, 2014.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
ICVIP '21: Proceedings of the 2021 5th International Conference on Video and Image Processing
December 2021
219 pages
ISBN:9781450385893
DOI:10.1145/3511176
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 March 2022

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Deep Feature Learning
  2. Image-based Kinship Verification
  3. Mining Negative Samples
  4. Unbalanced Data

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Funding Sources

Conference

ICVIP 2021

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 24
    Total Downloads
  • Downloads (Last 12 months)5
  • Downloads (Last 6 weeks)0
Reflects downloads up to 06 Jan 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media