Skip to main content
SpringerLink
Log in

Weighted fusion joint bayesian metric with patch-based facial region-specific features for face identification

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

This paper presents a new automated face identification method. The novelty of our method consists of two parts: (1) facial region-specific sparse feature learning and (2) Weighted Fusion Joint Bayesian (WFJB) metric learning for classification. In the former part, a face is partitioned into a number of local regions and a set of small image patches is then extracted from each of the local regions. Subsequently, local texture descriptors extracted from patch images are applied to dictionary learning for creating sparse representations of individual local regions. The low-dimensional features of sparse representations from all the local regions are then pooled to generate our proposed Patch-based Local Spare Feature (PLSF) set, which is discriminant and complementary for face identification. In our WFJB model, the similarity between gallery (pre-enrolled) and probe (test) faces is measured using a weighted sum of probabilistic similarity scores, each computed for a particular feature element within a PLSF set. The weights are determined in an automatic and adaptive way via class-wise discriminant analysis on PLSF sets. Extensive and comparative experiments have been conducted on five public face databases. Results show that combination of our proposed PLSF set and WFJB metric is a feasible solution for improving face identification on challenging face images with severe variation in illumination, viewpoint, and expression. In addition, our method achieves better or comparable state-of-the-art results on the standard LFW identification protocols.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. From our previous work [9, 10], FR systems generally falls into two tasks: identification and verification of a face identity. In identification, FR system identifies an unknown face in an image, while in verification a system confirms the claimed identity of a face presented to it. It should be emphasized that the focus of our work is on the development of FR algorithm for identification.

  2. The source code of FDDL model is publicly available at http://www4.comp.polyu.edu.hk/~cslzhang/papers.htm.

  3. Implementation code available at http://www.cs.utexas.edu/users/pjain/itml/

  4. Implementation code available at http://www.cs.cornell.edu/~kilian/code/lmnn/lmnn.html

  5. Implementation code available at http://www.cs.cmu.edu/~liuy/distlearn.htm

References

  1. Ahonen T, Hadid A, Pietikainen M (2006) Face description with local binary patterns: Application to face recognition. IEEE Trans Pattern Anal Mach Intell 28(12):2037–2041

    Article  MATH  Google Scholar 

  2. Best-Rowden L, Han H, Otto C, Klare B, Jain AK (2014) Unconstrained face recognition: Identifying a person of interest from a media collection. IEEE Trans Information Forensics and Security 9(12):2144–2157

    Article  Google Scholar 

  3. Brown M, Hua G, Winder S (2011) Discriminative learning of local image descriptors. IEEE Trans Pattern Anal Mach Intell 33(1):43–57

    Article  Google Scholar 

  4. Cao X, Wipf D, Wen F, Duan G, Sun J (2013) A practical transfer learning algorithm for face verification. In: Proc. Int’l Conf. Computer Vision (ICCV), pp. 3208–3215

  5. Chai Z, Sun Z, Mendez-Vazquez H, He R, Tan T (2014) Gabor Ordinal Measures for Face Recognition. IEEE Trans Information Forensics and Security 9(1):14–26

    Article  Google Scholar 

  6. Chan CH, Kittler J (2010) Sparse Representation of (Multiscale) Histograms for Face Recognition robust to Registration and Illumination Problems. In: Proc. IEEE 17th Int’l Conf. Image Processing (ICIP), pp. 2441–2444

  7. Chen D, Cao X, Wang L, Wen F, Sun J (2012) Bayesian face revisited: A joint formulation. European Conf. Computer Vision (ECCV), pp. 566–579

  8. Chen D, Cao X, Wen F, Sun J (2013) Blessing of dimensionality: High-dimensional feature and its efficient compression for face verification. In: Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 3025–3032

  9. Choi JY, Plataniotis KN, Ro YM (2012) Face Feature Weighted Fusion Based Fuzzy Membership Degree for Video Face Recognition. IEEE Trans on Systems Man and Cybernetics- Part B 42(4):1270–1282

    Article  Google Scholar 

  10. Choi JY, Ro YM, Plataniotis KN (2012) Color Local Texture Features for Color Face Recognition. IEEE Trans Image Process 21(3):1366–1380

    Article  MathSciNet  MATH  Google Scholar 

  11. Davis J, Kulis B, Jain P, Sra S, Dhillon I (2007) Information theoretic metric learning. Proc. 24th Int’l Conf. Machine Learning (ICML), pp. 209–216

  12. Ding C, Choi J, Tao D, Davis LS (2016) Multi-directional multi-level dual-cross patterns for robust face recognition. IEEE Trans Pattern Anal Mach Intell 38(3):518–531

    Article  Google Scholar 

  13. Ding C, Tao D (2016) A comprehensive survey on pose-invariant face recognition. ACM Trans Intelligent Systems and Technology (TIST) 7(3):1–40

    Article  Google Scholar 

  14. Ding C, Xu C, Tao D (2015) Multi-task pose-invariant face recognition. IEEE Trans Image Process 24(3):980–993

    Article  MathSciNet  MATH  Google Scholar 

  15. Fukunaga K (1992) Introduction to Statistical Pattern Recognition. Academic, New York

    MATH  Google Scholar 

  16. Geng C, Jiang X (2009) Face recognition using SIFT features. Proc. IEEE Int’l Conf. Image Processing (ICIP), pp. 3313–3316

  17. Guillaumin M, Verbeek J, Schmid C (2009) Is that you? Metric learning approaches for face identification. Proc. 12th IEEE Int’l Conf. Computer Vision (ICCV), pp. 498–505

  18. Huang GB, Lee H, Learned-Miller E (2012) Learning hierarchical representations for face verification with convolutional deep belief networks. In: Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 2518–2525

  19. Huang GB, Ramesh M, Berg T, Learned-Miller E (2007) Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments, vol 1. Univ. of Massachusetts, Amherst, pp 7–49

    Google Scholar 

  20. Jain AK, Ross A, Prabhakar S (2004) An Introduction to Biometric Recognition. IEEE Trans Circuits and Systems for Video Technology 14:4–20

    Article  Google Scholar 

  21. Jiang X, Mandal B, Kot A (2008) Eigenfeature regularization and extraction in face recognition. IEEE Trans Pattern Anal Mach Intell 30(3):383–394

    Article  Google Scholar 

  22. Lei Z, Li SZ, Chu R, Zhu X (2007) Face Recognition with Local Gabor Textons. Proc Int’l Conf Advances in Biometrics (ICB) 4642:49–57

    Article  Google Scholar 

  23. Liu C, Wechsler H (2002) Gabor feature based classification using the enhanced fisher liner discriminant model for face recognition. IEEE Trans Image Process 11(4):467–476

    Article  Google Scholar 

  24. Lu J, Tan YP (2010) Cost-sensitive subspace learning for face recognition. In: IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 2661–2666

  25. Martinez AM, Benavente R (1998) The AR face database. CVC Tech. Report, 24

  26. Maturana D, Mery D, Soto A (2010) Face Recognition with Decision Tree-Based Local Binary Patterns. Proc 10th Asian Conf Computer Vision (ACCV) 6495:618–629

    Google Scholar 

  27. Maturana D, Mery D, Soto A (2011) Learning discriminative local binary patterns for face recognition. In: Proc. IEEE Int’l Conf. Automatic Face and Gesture Recognition and Workshops (FG), pp. 470–475

  28. Meng X, Shan S, Chen X, Gao W (2006) Local Visual Primitives (LVP) for Face Modelling and Recognition. Proc 18th Int’l Conf Pattern Recognition (ICPR) 2:536–539

    Article  Google Scholar 

  29. Messer K, Mastas J, Kittler J, Luettin J, Maitre G (1999) XM2VTSDB: The extended M2VTS database. In: Proc. IEEE Int’l Conf. Audio and Video-based Biometric Person Authentication (AVBPA), vol. 964, pp. 72–77

  30. Mignon A, Jurie F (2012) Pcca: A new approach for distance learning from sparse pairwise constraints. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 2666–2672

  31. Nguyen HV, Bai L (2010) Cosine similarity metric learning for face verification. Asian Conf. Computer Vision (ACCV), pp. 709–720

  32. Parkhi OM, Vedaldi A, Zisserman A (2015) Deep Face Recognition. British Machine Vision 1(3):6

    Google Scholar 

  33. Parkhi OM, Vedaldi A, Zisserman A (2015) Deep Face Recognition. British Machine Vision Conference (BMVC) 1, 6(3)

  34. Phillips PJ, Moon H, Rizvi SA, Rauss PJ (2000) The FERET Evaluation Methodology for Face Recognition Algorithms. IEEE Trans Pattern Anal Mach Intell 22(10):1090–1104

    Article  Google Scholar 

  35. Ramirez I, Sprechmann P, Sapiro G (2010) Classification and clustering via dictionary learning with structured incoherence and shared features. In: IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 3501–3508

  36. Sim T, Baker S, Bsat M (2003) The CMU Pose Illumination, and Expression Database. IEEE Trans Pattern Anal Mach Intell 25(12):1615–1618

    Article  Google Scholar 

  37. Sun Y, Wang X, Tang X (2014) Deep Learning Face Representation from Predicting 10,000 Classes. In: Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 1891–1898

  38. Sun Y, Wang X, Tang X (2015) Deeply learned face representations are sparse, selective, and robust. In: Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 2892–2900

  39. Taigman Y, Yang M, Ranzato M, Wolf L (2014) DeepFace: closing the gap to human-level performance in face verification. Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 1701–1708

  40. Taigman Y, Yang M, Ranzato M, Wolf L (2015) Web-scale training for face identification. In: Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 2746–2754

  41. Vu NS, Caplier A (2012) Enhanced Patterns of Oriented Edge Magnitudes for Face Recognition and Image Matching. IEEE Trans Image Processing 21(3):1352–1365

    Article  MathSciNet  MATH  Google Scholar 

  42. Wang X, Tang X (2004) A unified framework for subspace face recognition. IEEE Trans Pattern Anal Mach Intell 26(9):1222–1228

    Article  Google Scholar 

  43. Weinberger K, Saul L (2009) Distance Metric Learning for Large Margin Nearest Neighbor Classification. The Journal of Machine Learning Research 10:207–244

    MATH  Google Scholar 

  44. Wright J, Ganesh A, Yang A, Ma Y (2009) Robust face recognition via sparse representation. IEEE Trans Pattern Anal and Mach Intell 31(2):210–227

    Article  Google Scholar 

  45. Wright J, Ma Y, Mairal J, Sapiro G, Huang T, Yan S (2010) Sparse representation for computer vision and pattern recognition. Proc IEEE Special Issue on Applications of Compressive Sensing & Sparse Representation 98(6):1031–1044

    Google Scholar 

  46. Wu X, He R, Sun Z (2015) A lightened cnn for deep face representation. IEEE Conf. Computer Vision and Pattern Recognition (CVPR)

  47. Xie S, Shan S, Chen X, Meng X, Gao W (2009) Learned Local Gabor Patterns for Face Representation and Recognition. Signal Process 89(12):2333–2344

    Article  MATH  Google Scholar 

  48. Yang A, Ganesh A, Sastry S, Ma Y (2010) Fast L1-Minimization Algorithms and an Application in Robust Face Recognition: A Review. Proc IEEE 17th Int’l Conf Image Processing (ICIP):1849–1852

  49. Yang M, Zhang L (2010) Gabor Feature based Sparse Representation for Face Recognition with Gabor Occlusion Dictionary. European Conf Computer Vision (ECCV) 6316:448–461

    Google Scholar 

  50. Yang M, Zhang L, Feng X, Zhang D (2011) Fisher discrimination dictionary learning for sparse representation. IEEE Int’l Conference on Computer Vision:543–550

  51. Zhang Q, Li BX (2010) Discriminative K-SVD for dictionary learning in face recognition. In: IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 2691–2698

  52. Zhang B, Shan S, Chen X, Gao W (2007) Histogram of Gabor Phase Patterns (HGPP): A Novel Object Representation Approach for Face Recognition. IEEE Trans Image Processing 16(1):57–68

    Article  MathSciNet  Google Scholar 

  53. Zhang W, Shan S, Gao W, Zhang H (2005) Local Gabor Binary Pattern Histogram Sequence (lgbphs): A Novel Non-Statistical Model for Face Representation and Recognition. Proc 10th IEEE Int’l Conf Computer Vision (ICCV) 1:786–791

    Article  Google Scholar 

  54. Zhou Z, Wagner A, Mobahi H, Wright J, Ma Y (2009) Face Recognition with Contiguous Occlusion using Markov Random Fields. In: Proc. IEEE Int’l Conf. Computer Vision (ICCV), pp. 1050–1057

Download references

Acknowledgement

This research was supported by Hankuk University of Foreign Studies Research Fund. This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (No. 2018R1D1A1A09082615).

Author information

Authors and Affiliations

  1. Pattern Recognition and Machine Intelligence Laboratory, Division of Computer & Electronic Systems Engineering, Hankuk University of Foreign Studies, 81, Oedae-ro, Mohyeon-myeon, Cheoin-gu, Yongin-si, Gyeonggi-do, 17305, Republic of Korea

    Jae Young Choi

Authors
  1. Jae Young Choi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jae Young Choi.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Choi, J.Y. Weighted fusion joint bayesian metric with patch-based facial region-specific features for face identification. Multimed Tools Appl 78, 24883–24902 (2019). https://doi.org/10.1007/s11042-018-6950-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-6950-0

Keywords

Search

Navigation