Skip to main content
SpringerLink
Log in

Learning multi-view deep and shallow features through new discriminative subspace for bi-subject and tri-subject kinship verification

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

This paper presents the combination of deep and shallow features (multi-view features) using the proposed metric learning (SILD+WCCN/LR) approach for kinship verification. Our approach based on an automatic and more efficient two-step learning into deep/shallow information. First, five layers for deep features and five shallow features (i.e. texture and shape), representing more precisely facial features involved in kinship relations (Father-Son, Father-Daughter, Mother-Son, and Mother-Daughter) are used to train the proposed Side-Information based Linear Discriminant Analysis integrating Within Class Covariance Normalization (SILD+WCCN) method. Then, each of the features projected through the discriminative subspace of the proposed SILD+WCCN metric learning method. Finally, a Logistic Regression (LR) method is used to fuse the six scores of the projected features. To show the effectiveness of our SILD+WCNN method, we do some experiments on LFW database. In term of evaluation, the proposed automatic Facial Kinship Verification (FKV) is compared with existing ones to show its effectiveness, using two challenging kinship databases. The experimental results showed the superiority of our FKV against existing ones and reached verification rates of 86.20% and 88.59% for bi-subject matching on the KinFaceW-II and TSKinFace databases, respectively. Verification rates for tri-subject matching of 90.94% and 91.23% on the available TSKinFace database for Father-Mother-Son and Father-Mother-Daughter, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Alvergne A, Oda R, Faurie C, Matsumoto-Oda A, Durand V, Raymond M (2009) Cross-cultural perceptions of facial resemblance between kin. J Vis 9(6):23. https://doi.org/10.1167/9.6.23

    Article  Google Scholar 

  2. Barkan O, Weill J, Wolf L, Aronowitz H (2013) Fast high dimensional vector multiplication face recognition. In: 2013 IEEE international conference on computer vision, pp 1960–1967. https://doi.org/10.1109/ICCV.2013.246

  3. Chang CC, Lin CJ (2011) Libsvm: a library for support vector machines. ACM Trans Intell Syst Technol (TIST) 2(3):27. https://www.csie.ntu.edu.tw/cjlin/libsvm/

    Google Scholar 

  4. Dal Martello MF, Maloney LT (2006) Where are kin recognition signals in the human face? J Vis 6(12):2. https://doi.org/10.1167/6.12.2

    Article  Google Scholar 

  5. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05), vol 1, pp 886–893. https://doi.org/10.1109/CVPR.2005.177

  6. Davis JV, Kulis B, Jain P, Sra S, Dhillon IS (2007) Information-theoretic metric learning. In: Proceedings of the 24th international conference on machine learning, ICML ’07. ACM, New York, pp 209–216. https://doi.org/10.1145/1273496.1273523. http://doi.acm.org/10.1145/1273496.1273523

  7. DeBruine LM, Smith FG, Jones BC, Roberts SC, Petrie M, Spector TD (2009) Kin recognition signals in adult faces. Vis Res 49 (1):38–43. https://doi.org/10.1016/j.visres.2008.09.025. http://www.sciencedirect.com/science/article/pii/S0042698908004707

    Article  Google Scholar 

  8. Dehak N, Kenny PJ, Dehak R, Dumouchel P, Ouellet P (2011) Front-end factor analysis for speaker verification. IEEE Trans Audio Speech Lang Process 19(4):788–798. https://doi.org/10.1109/TASL.2010.2064307

    Article  Google Scholar 

  9. Goldberger J, Hinton GE, Roweis ST, Salakhutdinov RR (2005) Neighbourhood components analysis. In: Saul LK, Weiss Y, Bottou L (eds) Advances in neural information processing systems 17. MIT Press, pp 513–520. http://papers.nips.cc/paper/2566-neighbourhood-components-analysis.pdf

  10. Guillaumin M, Verbeek J, Schmid C (2009) Is that you? metric learning approaches for face identification. In: 2009 IEEE 12th international conference on computer vision, pp 498–505. https://doi.org/10.1109/ICCV.2009.5459197

  11. Harrell FE Jr (2015) Regression modeling strategies: with applications to linear models, logistic and ordinal regression, and survival analysis. Springer, Berlin

    Book  Google Scholar 

  12. Hastie T, Tibshirani R, Friedman J (2001) The elements of statistical learning. Springer series in statistics. Springer, New York

    MATH  Google Scholar 

  13. Hu J, Lu J, Tan Y (2018) Sharable and individual multi-view metric learning. IEEE Trans Pattern Anal Mach Intell 40(9):2281–2288. https://doi.org/10.1109/TPAMI.2017.2749576

    Article  Google Scholar 

  14. Huang GB, Mattar M, Berg T, Learned-Miller E (2008) Labeled faces in the wild: a database for studying face recognition in unconstrained environments. In: Workshop on faces in ’real-life’ images: detection, alignment, and recognition. Erik Learned-Miller and Andras Ferencz and Frédéric Jurie, Marseille, France. https://hal.inria.fr/inria-00321923

  15. Iman RL, Conover WJ (1982) A distribution-free approach to inducing rank correlation among input variables. Commun Stat Simul Comput 11(3):311–334. https://doi.org/10.1080/03610918208812265

    Article  MATH  Google Scholar 

  16. Kaminski G, Dridi S, Graff C, Gentaz E (2009) Human ability to detect kinship in strangers’ faces: effects of the degree of relatedness. Proc R Soc Lond B Biol Sci 276(1670):3193–3200. https://doi.org/10.1098/rspb.2009.0677. http://rspb.royalsocietypublishing.org/content/276/1670/3193

    Article  Google Scholar 

  17. Kaminski G, Ravary F, Graff C, Gentaz E (2010) Firstborns’ disadvantage in kinship detection. Psychol Sci 21(12):1746–1750. https://doi.org/10.1177/0956797610388045. PMID: 21051523

    Article  Google Scholar 

  18. Kan M, Xu D, Shan S, Li W, Chen X (2013) Learning prototype hyperplanes for face verification in the wild. IEEE Trans Image Process 22:3310–3316

    Article  Google Scholar 

  19. Kannala J, Rahtu E (2012) Bsif: binarized statistical image features. In: Proceedings of the 21st international conference on pattern recognition (ICPR2012), pp 1363–1366

  20. Liu C (2014) Discriminant analysis and similarity measure. Pattern Recogn 47(1):359–367. https://doi.org/10.1016/j.patcog.2013.06.023. http://www.sciencedirect.com/science/article/pii/S0031320313002756

    Article  Google Scholar 

  21. Lu J, Hu J, Tan YP (2017) Discriminative deep metric learning for face and kinship verification. IEEE Trans Image Process 26(9):4269–4282. https://doi.org/10.1109/TIP.2017.2717505

    Article  MathSciNet  MATH  Google Scholar 

  22. Lu J, Zhou X, Tan YP, Shang Y, Zhou J (2014) Neighborhood repulsed metric learning for kinship verification. IEEE Trans Pattern Anal Mach Intell 36(2):331–345. https://doi.org/10.1109/TPAMI.2013.134

    Article  Google Scholar 

  23. Ma J, Zhao Y, Ahalt S (2002) Osu svm classifier matlab toolbox (ver 3.00). Pulsed Neural Networks. http://svm.sourceforge.net/docs/3.00/api/

  24. Kan M, Shan S, Xu D, Chen X (2011) Side-information based linear discriminant analysis for face recognition. In: Proceedings of the BMVC, pp 125.1–125.0. https://doi.org/10.5244/C.25.125

  25. Nguyen HV, Bai L (2011) Cosine similarity metric learning for face verification. Springer, Berlin, pp 709–720. https://doi.org/10.1007/978-3-642-19309-5_55

    Google Scholar 

  26. Nosaka R, Ohkawa Y, Fukui K (2012) Feature extraction based on co-occurrence of adjacent local binary patterns. Springer, Berlin, pp 82–91. https://doi.org/10.1007/978-3-642-25346-1_8

    Google Scholar 

  27. Ojala T, Pietikainen M, Maenpaa T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24(7):971–987. https://doi.org/10.1109/TPAMI.2002.1017623

    Article  MATH  Google Scholar 

  28. Ojansivu V, Heikkilä J (2008) Blur insensitive texture classification using local phase quantization. Springer, Berlin, pp 236–243. https://doi.org/10.1007/978-3-540-69905-7_27

    Google Scholar 

  29. Owusu E, Zhan Y, Mao QR (2014) An svm-adaboost facial expression recognition system. Appl Intell 40(3):536–545. https://doi.org/10.1007/s10489-013-0478-9

    Article  Google Scholar 

  30. Pang Y, Wang S, Yuan Y (2014) Learning regularized lda by clustering. IEEE Trans Neural Netw Learn Syst 25(12):2191–2201. https://doi.org/10.1109/TNNLS.2014.2306844

    Article  Google Scholar 

  31. Parkhi OM, Vedaldi A, Zisserman A (2015) Deep face recognition. In: British machine vision conference

  32. Qin X, Tan X, Chen S (2015) Tri-subject kinship verification: understanding the core of a family. IEEE Trans Multimedia 17(10):1855–1867. https://doi.org/10.1109/TMM.2015.2461462

    Article  Google Scholar 

  33. Song HO, Xiang Y, Jegelka S, Savarese S (2016) Deep metric learning via lifted structured feature embedding. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 4004–4012. https://doi.org/10.1109/CVPR.2016.434

  34. Swets DL, Weng JJ (1996) Using discriminant eigenfeatures for image retrieval. IEEE Trans Pattern Anal Mach Intell 18(8):831–836. https://doi.org/10.1109/34.531802

    Article  Google Scholar 

  35. Weinberger KQ, Saul LK (2009) Distance metric learning for large margin nearest neighbor classification. J Mach Learn Res 10:207–244. http://dl.acm.org/citation.cfm?id=1577069.1577078

    MATH  Google Scholar 

  36. Wu P, Hoi SC, Xia H, Zhao P, Wang D, Miao C (2013) Online multimodal deep similarity learning with application to image retrieval. In: Proceedings of the 21st ACM international conference on multimedia, MM’13. ACM, New York, pp 153–162. https://doi.org/10.1145/2502081.2502112. http://doi.acm.org/10.1145/2502081.2502112

  37. Wu X, Boutellaa E, López MB, Feng X, Hadid A (2016) On the usefulness of color for kinship verification from face images. In: 2016 IEEE international workshop on information forensics and security (WIFS), pp 1–6. https://doi.org/10.1109/WIFS.2016.7823901

  38. Xing EP, Ng AY, Jordan MI, Russell S (2002) Distance metric learning, with application to clustering with side-information. In: Proceedings of the 15th international conference on neural information processing systems, NIPS’02. MIT Press, Cambridge, pp 521–528. http://dl.acm.org/citation.cfm?id=2968618.2968683

  39. Yan H (2017) Kinship verification using neighborhood repulsed correlation metric learning. Image Vis Comput 60:91–97. https://doi.org/10.1016/j.imavis.2016.08.009. http://www.sciencedirect.com/science/article/pii/S0262885616301330. Regularization Techniques for High-Dimensional Data Analysis

    Article  Google Scholar 

  40. Yan H, Lu J, Deng W, Zhou X (2014) Discriminative multimetric learning for kinship verification. IEEE Trans Inf Forensics Secur 9(7):1169–1178. https://doi.org/10.1109/TIFS.2014.2327757

    Article  Google Scholar 

  41. Yan H, Lu J, Zhou X (2015) Prototype-based discriminative feature learning for kinship verification. IEEE Trans Cybern 45(11):2535–2545. https://doi.org/10.1109/TCYB.2014.2376934

    Article  Google Scholar 

  42. Yu H, Chung CY, Wong KP, Lee HW, Zhang JH (2009) Probabilistic load flow evaluation with hybrid latin hypercube sampling and cholesky decomposition. IEEE Trans Power Syst 24(2):661–667. https://doi.org/10.1109/TPWRS.2009.2016589

    Article  Google Scholar 

  43. Zhang T, Fang B, Tang YY, Shang Z, Xu B (2010) Generalized discriminant analysis: A matrix exponential approach. IEEE Trans Syst Man Cybern Part B Cybern 40(1):186–197. https://doi.org/10.1109/TSMCB.2009.2024759

    Article  Google Scholar 

  44. Zhao YG, Song Z, Zheng F, Shao L (2018) Learning a multiple kernel similarity metric for kinship verification. Inf Sci 430-431:247–260. https://doi.org/10.1016/j.ins.2017.11.048. http://www.sciencedirect.com/science/article/pii/S0020025516310416

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Laboratory of LESIA, University of Biskra, Biskra, Algeria

    Oualid Laiadi

  2. IEMN DOAE UMR CNRS 8520 Laboratory, Polytechnic University of Hauts-de-France, Valenciennes, France

    Oualid Laiadi & Abdelmalik Taleb-Ahmed

  3. University of Biskra, Biskra, Algeria

    Abdelmalik Ouamane

  4. Laboratory of LGEB, University of Biskra, Biskra, Algeria

    Abdelhamid Benakcha

  5. Center for Machine Vision and Signal Analysis, University of Oulu, Oulu, Finland

    Abdenour Hadid

Authors
  1. Oualid Laiadi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Abdelmalik Ouamane
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Abdelhamid Benakcha
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Abdelmalik Taleb-Ahmed
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Abdenour Hadid
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Oualid Laiadi.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Laiadi, O., Ouamane, A., Benakcha, A. et al. Learning multi-view deep and shallow features through new discriminative subspace for bi-subject and tri-subject kinship verification. Appl Intell 49, 3894–3908 (2019). https://doi.org/10.1007/s10489-019-01489-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-019-01489-2

Keywords

Search

Navigation