Skip to main content

Advertisement

Springer Nature Link
Log in

A pyramid multi-level face descriptor: application to kinship verification

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Texture descriptors such as Local Binary Pattern (LBP), Local Phase Quantization (LPQ), and Histogram of Oriented Gradients (HOG) have been widely used for face image analysis. This work introduces a novel framework for image-based kinship verification able to efficiently combine local and global facial information extracted from diverse descriptors. The proposed scheme relies on two main points: (1) we model the face images using a Pyramid Multi-level (PML) representation where local descriptors are extracted from several blocks at different resolution scales; (2) we compute the covariance (second-order statistics) between diverse local features characterizing each individual block in the PML representation. This gives rise to a face descriptor with two interesting properties: (i) thanks to the PML representation, scales and face parts are explicitly encoded in the final descriptor without having to detect the facial landmarks; (ii) the covariance descriptor encodes spatial features of any type allowing the integration of several state-of-the-art texture and color features. Experiments conducted on three public kinship databases show that the proposed descriptor can outperform many state-of-the-art kinship verification algorithms and descriptors including those that are based on deep Convolutional Neural Nets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Ahonen T, Hadid A, Pietikainen M (2006) Face description with local binary patterns: application to face recognition. IEEE Trans Pattern Anal Mach Intell 28 (12):2037–2041

    Article  Google Scholar 

  2. Alirezazadeh P, Fathi A, Abdali-Mohammadi F (2015) A genetic algorithm-based feature selection for kinship verification. IEEE Signal Process Lett 22(12):2459–2463

    Article  Google Scholar 

  3. Bekhouche S, Ouafi A, Benlamoudi A, Taleb-Ahmed A, Hadid A (2015) Automatic age estimation and gender classification in the wild. In: Proceeding of the international conference on automatic control, telecommunications and signals ICATS’15

  4. Bekhouche S, Ouafi A, Dornaika F, Taleb-Ahmed A, Hadid A (2017) Pyramid multi-level features for facial demographic estimation. Expert Syst Appl 80 (Supplement C):297–310

    Article  Google Scholar 

  5. Bianconi F, Bello R, Napoletano P, Maria FD (2017) Improved opponent colour local binary patterns for colour texture classification. In: Workshop computational color imaging workshop, CCIW

    Google Scholar 

  6. Cao Z, Yin Q, Tang X, Sun J (2010) Face recognition with learning-based descriptor. In: 2010 IEEE Computer society conference on computer vision and pattern recognition, pages 2707–2714

  7. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection IEEE computer society conference on computer vision and pattern recognition, 2005. CVPR 2005. vol 1. IEEE, pp 886–893

  8. Davarpanah SH, Khalid F, Nurliyana Abdullah L, Golchin M (2016) A texture descriptor: background local binary pattern (bglbp). Multimedia Tools Appl 75 (11):6549–6568

    Article  Google Scholar 

  9. DeBruine L, Jones B, Little A, Perrett D (2008) Social perception of facial resemblance in humans. Arch Sex Behav 37(1):64–77

    Article  Google Scholar 

  10. DeBruine LM, Smith FG, Jones BC, Roberts SC, Petrie M, Spector TD (2009) Kin recognition signals in adult faces. Vis Res 49(1):38–43

    Article  Google Scholar 

  11. Dehghan A, Ortiz EG, Villegas R, Shah M (2014) Who do i look like? Determining parent-offspring resemblance via gated autoencoders. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 1757–1764

  12. Dornaika F, Moujahid A, Merabet YE, Ruichek Y (2016) Building detection from orthophotos using a machine learning approach: an empirical study on image segmentation and descriptors. Expert Syst Appl 58:130–142

    Article  Google Scholar 

  13. Fang R, Tang KD, Snavely N, Chen T (2010) Towards computational models of kinship verification. In: 2010 17th IEEE international conference on image processing (ICIP), pp 1577–1580

  14. Fang Y, Chen YYS, Wang H, Shu C (2016) Sparse similarity metric learning for kinship verification. In: Visual communications and image processing (VCIP), 2016. IEEE, pp 1–4

  15. Guan D, Yuan W, Lee Y-K, Najeebullah K, Rasel MK (2014) A review of ensemble learning based feature selection. IETE Tech Rev 31(3):190–198

    Article  Google Scholar 

  16. Guo Y, Dibeklioglu H, van der Maaten L (2014) Graph-based kinship recognition. In: 2014 22nd international conference on pattern recognition (ICPR), IEEE, pp 4287–4292

  17. Hu J, Lu J, Yuan J, Tan Y-P (2014) Large margin multi-metric learning for face and kinship verification in the wild. In: ACCV (3), pp 252–267

    Google Scholar 

  18. Hussain SU, Napoléon T, Jurie F (2012) Face recognition using local quantized patterns. In: British machive vision conference Guildford, United Kingdom, p 11

  19. Kabbai L, Azaza A, Abdellaoui M, Douik A (2015) Image matching based on lbp and sift descriptor. In: 2015 IEEE 12th international multi-conference on systems, signals and devices (SSD15), pp 1–6

  20. Kannala J, Rahtu E (2012) Bsif: Binarized statistical image features. In: 2012 21st international conference on pattern recognition (ICPR), pp 1363–1366

  21. (2014). KinFaceW. http://www.kinfacew.com/index.html

  22. Kohli N, Vatsa M, Singh R, Noore A, Majumdar A (2017) Hierarchical representation learning for kinship verification. IEEE Trans Image Process 26(1):289–302

    Article  MathSciNet  Google Scholar 

  23. Kumar CR (2017) Harmonic rule for measuring the facial similarities among relatives. Trans Mach Learn Artif Intell 4(6):29

    Google Scholar 

  24. Lan R, Zhou Y (2016) Quaternion-michelson descriptor for color image classification. IEEE Trans Image Process 25(11):5281–5292

    Article  MathSciNet  Google Scholar 

  25. Lan R, Zhou Y, Tang YY (2016) Quaternionic local ranking binary pattern: a local descriptor of color images. IEEE Trans Image Process 25(2):566–579

    Article  MathSciNet  Google Scholar 

  26. Lan R, Zhou Y, Tang YY (2017) Quaternionic weber local descriptor of color images. IEEE Trans Circuits Syst Video Technol 27(2):261–274

    Article  Google Scholar 

  27. Lan R, Zhou Y, Tang YY, Chen CLP (2014) Person reidentification using quaternionic local binary pattern. In: 2014 IEEE international conference on multimedia and expo (ICME), pp 1–6

  28. Li L, Feng X, Wu X, Xia Z, Hadid A (2016) Kinship verification from faces via similarity metric based convolutional neural network. In: International conference image analysis and recognition. Springer, pp 539–548

  29. Liu Q, Puthenputhussery A, Liu C (2016) A novel inheritable color space with application to kinship verification. In: 2016 IEEE winter conference on applications of computer vision (WACV). IEEE, pp 1–9

  30. Liu Y, Nie L, Liu L, Rosenblum DS (2016) From action to activity: sensor-based activity recognition. Neurocomputing 181:108–115. Big Data Driven Intelligent Transportation Systems

    Article  Google Scholar 

  31. Liu Y, Zhang L, Nie L, Yan Y, Rosenblum DS (2016) Fortune teller: predicting your career path. In: 13th AAAI conference on artificial intelligence

  32. López MB, Boutellaa E, Hadid A (2016) Comments on the “kinship face in the wild” data sets. IEEE Trans Pattern Anal Mach Intell 38(11):2342–2344

    Article  Google Scholar 

  33. Lu J, Hu J, Liong VE, Zhou X, Bottino A, Islam IU, Vieira TF, Qin X, Tan X, Chen S, Mahpod S, Keller Y, Zheng L, Idrissi K, Garcia C, Duffner S, Baskurt A, Castrillón-Santana M, Lorenzo-Navarro J (2015) The fg 2015 kinship verification in the wild evaluation. In: 2015 11th IEEE international conference and workshops on automatic face and gesture recognition (FG), vol 1, pp 1–7

  34. Lu J, Liong VE, Zhou X, Zhou J (2015) Learning compact binary face descriptor for face recognition. IEEE Trans Pattern Anal Mach Intell 37(10):2041–2056

    Article  Google Scholar 

  35. Lu J, Zhou X, Tan Y-P, Shang Y, Zhou J (2014) Neighborhood repulsed metric learning for kinship verification. IEEE Trans Pattern Anal Mach Intell 36(2):331–345

    Article  Google Scholar 

  36. Ma Y, Wu X, Yu G, Xu Y, Wang Y (2016) Pedestrian detection and tracking from low-resolution unmanned aerial vehicle thermal imagery. Sensors 16(4)

    Article  Google Scholar 

  37. Nguyen DT, Cho SR, Shin KY, Bang JW, Park KR (2014) Comparative study of human age estimation with or without preclassification of gender and facial expression. Sci World J 2014:15

    Google Scholar 

  38. Ojala T, Pietikäinen M, Maenpaa T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24:971–987

    Article  Google Scholar 

  39. Ojala T, Pietikainen M, Maenpaa T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24(7):971–987

    Article  Google Scholar 

  40. Patel B, Maheshwari R, Raman B (2017) Evaluation of periocular features for kinship verification in the wild. Comput Vis Image Underst 160(Supplement C):24–35

    Article  Google Scholar 

  41. Puthenputhussery A, Liu Q, Liu C (2016) Sift flow based genetic fisher vector feature for kinship verification. In: 2016 IEEE international conference on image processing (ICIP), IEEE, pp 2921–2925

  42. Qin X, Tan X, Chen S (2016) Mixed bi-subject kinship verification via multi-view multi-task learning. Neurocomputing 214:350–357

    Article  Google Scholar 

  43. Shao M, Xia S, Fu Y (2011) Genealogical face recognition based on ub kinface database. In: CVPR 2011 WORKSHOPS, pp 60–65

  44. Silva C, Bouwmans T, Frélicot C (2015) An extended center-symmetric local binary pattern for background modeling and subtraction in videos. In: Proceedings of the 10th international conference on computer vision theory and applications - volume 1: VISAPP, (VISIGRAPP 2015), pp 395–402

  45. Szeliski R (2011) Computer vision algorithms and applications. Springer, New York

    MATH  Google Scholar 

  46. Takala V, Ahonen T, Pietik?inen M (2005) Block-based methods for image retrieval using local binary patterns. In: Image analysis, SCIA, volume LNCS, 3540

    Chapter  Google Scholar 

  47. Tan X, Triggs B (2010) Enhanced local texture feature sets for face recognition under difficult lighting conditions. IEEE Trans Image Processing 19:1635–1650

    Article  MathSciNet  Google Scholar 

  48. Tuzel O, Porikli F, Meer P (2006) A fast descriptor for detection and classification. In: European conf. on computer vision, pp 589–600

  49. Vinay K, Shreyas B (2006) Face recognition using gabor wavelets. In: 14th asilomar conference on signals, systems and computers, pp 593–597

  50. Wang W, Chen W, Xu D (2011) Pyramid-based multi-scale lbp features for face recognition. In: 2011 international conference on multimedia and signal processing (CMSP), vol 1, pp 151–155

  51. Wolf L, Hassner T, Taigman Y (2008) Descriptor based methods in the wild. In: Faces in real-life images workshop in ECCV

  52. Xia S, Shao M, Fu Y (2011) Kinship verification through transfer learning. In: Proceedings of the twenty-second international joint conference on artificial intelligence - volume three, IJCAI’11. AAAI Press, pp 2539–2544

  53. Xia S, Shao M, Luo J, Fu Y (2012) Understanding kin relationships in a photo. IEEE Trans Multimedia 14(4):1046–1056

    Article  Google Scholar 

  54. Xia S, Shao M, Luo J, Fu Y (2012) Understanding kin relationships in a photo. IEEE Trans Multimedia 14(4):1046–1056

    Article  Google Scholar 

  55. Yan H (2017) Kinship verification using neighborhood repulsed correlation metric learning. Image Vis Comput 60:91–97

    Article  Google Scholar 

  56. Yan H, Lu J, Deng W, Zhou X (2014) Discriminative multimetric learning for kinship verification. IEEE Trans Inf Forensics Secur 9(7):1169–1178

    Article  Google Scholar 

  57. Yan H, Lu J, Deng W, Zhou X (2014) Discriminative multimetric learning for kinship verification. IEEE Trans Inf Forensics Secur 9(7):1169–1178

    Article  Google Scholar 

  58. Yan H, Lu J, Zhou X (2015) Prototype-based discriminative feature learning for kinship verification. IEEE Trans Cybern 45(11):2535–2545

    Article  Google Scholar 

  59. Yang B, Chen S (2013) A comparative study on local binary pattern (LBP) based face recognition LBP histogram versus LBP image. Neurocomputing 120:365–379

    Article  Google Scholar 

  60. Zhang K, Huang Y, Song C, Wu H, Wang L (2015) Kinship verification with deep convolutional neural networks. In: Proceedings of the British machine vision conference (BMVC). BMVA Press, pp 148.1–148.12

  61. Zhou X, Hu J, Lu J, Shang Y, Guan Y (2011) Kinship verification from facial images under uncontrolled conditions. In: Proceedings of the 19th ACM international conference on multimedia. ACM, pp 953–956

  62. Zhou X, Shang Y, Yan H, Guo G (2016) Ensemble similarity learning for kinship verification from facial images in the wild. Information Fusion 32:40–48

    Article  Google Scholar 

  63. Zhou X, Yan H, Shang Y (2016) Kinship verification from facial images by scalable similarity fusion. Neurocomputing 197:136–142

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of the Basque Country UPV/EHU, San Sebastian, Spain

    A. Moujahid & F. Dornaika

  2. IKERBASQUE, Basque Foundation for Science, Bilbao, Spain

    F. Dornaika

Authors
  1. A. Moujahid
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. F. Dornaika
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to F. Dornaika.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Moujahid, A., Dornaika, F. A pyramid multi-level face descriptor: application to kinship verification. Multimed Tools Appl 78, 9335–9354 (2019). https://doi.org/10.1007/s11042-018-6517-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-6517-0

Keywords