Skip to main content
SpringerLink
Log in

Generative adversarial networks for 2D-based CNN pose-invariant face recognition

  • Regular Paper
  • Published:
International Journal of Multimedia Information Retrieval Aims and scope Submit manuscript

Abstract

The computer vision community considers the pose-invariant face recognition (PIFR) as one of the most challenging applications. Many works were devoted to enhancing face recognition performance when facing profile samples. They mainly focused on 2D- and 3D-based frontalization techniques trying to synthesize frontal views from profile ones. In the same context, we propose in this paper a new 2D PIFR technique based on Generative Adversarial Network image translation. The used GAN is Pix2Pix paired architecture covering many generator and discriminator models that will be comprehensively evaluated on a new benchmark proposed in this paper referred to as Combined-PIFR database, which is composed of four datasets that provide profiles images and their corresponding frontal ones. The paired architecture we are using is based on computing the L1 distance between the generated image and the ground truth one (pairs). Therefore, both generator and discriminator architectures are paired ones. The Combined-PIFR database is partitioned respecting person-independent constraints to evaluate our proposed framework’s frontalization and classification sub-systems fairly. Thanks to the GAN-based frontalization, the recorded results demonstrate an important improvement of 33.57% compared to the baseline.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Ahonen T, Hadid A, Pietikainen M (2006) Face description with local binary patterns: application to face recognition. IEEE Trans Pattern Anal Mach Intell 28(12):2037–2041

    Article  MATH  Google Scholar 

  2. Calvo MG, Lundqvist D (2008) Facial expressions of emotion (kdef): identification under different display-duration conditions. Behav Res Methods 40(1):109–115

    Article  Google Scholar 

  3. Fischer M, Ekenel HK, Stiefelhagen R (2012) Analysis of partial least squares for pose-invariant face recognition. In: 2012 IEEE Fifth international conference on biometrics: theory, applications and systems (BTAS), pp. 331–338

  4. Gang SM, Lee JJ (2019) Depth map extraction from the single image using pix2pix model. J Korea Multim Soc 22(5):547–557

    Google Scholar 

  5. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. Adv Neural Infor Process Syst 2:2672–2680

    Google Scholar 

  6. Gross R, Matthews I, Cohn J, Kanade T, Baker S (2010) Multi-pie. Image Vis Comput 28(5):807–813

    Article  Google Scholar 

  7. Hassner T, Harel S, Paz E, Enbar R (2015) Effective face frontalization in unconstrained images. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4295–4304

  8. He K, Zhang X, Ren S, Sun J.(2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778

  9. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4700–4708

  10. Huang R, Zhang S, Li T, He R (2017) Beyond face rotation: global and local perception gan for photorealistic and identity preserving frontal view synthesis. In: Proceedings of the IEEE international conference on computer vision, pp. 2439–2448

  11. Kan M, Shan S, Chang H, Chen X (2014) Stacked progressive auto-encoders (SPAE) for face recognition across poses. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1883–1890

  12. Karras T, Laine S, Aila T (2019) A style-based generator architecture for generative adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4401–4410

  13. Kim TK, Kittler J (2006) Design and fusion of pose-invariant face-identification experts. IEEE Trans Circuits Syst Video Technol 16(9):1096–1106

    Article  Google Scholar 

  14. King DE (2009) Dlib-ml: A machine learning toolkit. J Mach Learn Res 10:1755–1758

    Google Scholar 

  15. Langner O, Dotsch R, Bijlstra G, Wigboldus DH, Hawk ST, Van Knippenberg A (2010) Presentation and validation of the Radboud faces database. Cogn Emot 24(8):1377–1388

    Article  Google Scholar 

  16. Li A, Shan S, Gao W (2011) Coupled bias-variance tradeoff for cross-pose face recognition. IEEE Trans Image Process 21(1):305–315

    MathSciNet  MATH  Google Scholar 

  17. Li D, Zhou H, Lam KM (2015) High-resolution face verification using pore-scale facial features. IEEE Trans Image Process 24(8):2317–2327

    Article  MathSciNet  MATH  Google Scholar 

  18. Li P, Wu X, Hu Y, He R, Sun Z.(2019) M2fpa: A multi-yaw multi-pitch high-quality dataset and benchmark for facial pose analysis. In: Proceedings of the IEEE International conference on computer vision, pp. 10, 043–10,051

  19. Luo P, Wang X, Tang X (2012) Hierarchical face parsing via deep learning. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, IEEE, pp. 2480–2487

  20. Park U, Jain AK (2010) Face matching and retrieval using soft biometrics. IEEE Trans Inf Forensics Secur 5(3):406–415

    Article  Google Scholar 

  21. Phillips PJ, Moon H, Rizvi SA, Rauss PJ (2000) The Feret evaluation methodology for face-recognition algorithms. IEEE Trans Pattern Anal Mach Intell 22(10):1090–1104

    Article  Google Scholar 

  22. Prince SJ, Elder JH, Warrell J, Felisberti FM (2008) Tied factor analysis for face recognition across large pose differences. IEEE Trans Pattern Anal Mach Intell 30(6):970–984

    Article  Google Scholar 

  23. Ronneberger O, Fischer P, Brox, T (2015) U-net: Convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention, Springer, pp. 234–241

  24. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M, Berg AC, Fei-Fei L (2015) ImageNet large scale visual recognition challenge. Int J Comput Vision (IJCV) 115(3):211–252. https://doi.org/10.1007/s11263-015-0816-y

    Article  MathSciNet  Google Scholar 

  25. Sato M, Hotta K, Imanishi A, Matsuda M, Terai K (2018) Segmentation of cell membrane and nucleus by improving pix2pix. In: BIOSIGNALS, pp. 216–220

  26. Tang H, Xu D, Sebe N, Yan Y (2019) Attention-guided generative adversarial networks for unsupervised image-to-image translation. In: 2019 International Joint Conference on Neural Networks (IJCNN), IEEE, pp. 1–8

  27. Thomaz CE, Giraldi GA (2010) A new ranking method for principal components analysis and its application to face image analysis. Image Vis Comput 28(6):902–913

    Article  Google Scholar 

  28. Wu X, He R, Sun Z, Tan T (2018) A light CNN for deep face representation with noisy labels. IEEE Trans Inf Forensics Secur 13(11):2884–2896

    Article  Google Scholar 

  29. Xu R, Zhou Z, Zhang W, Yu Y (2017) Face transfer with generative adversarial network. arXiv preprint arXiv:1710.06090

  30. Yin Y, Jiang S, Robinson JP, Fu Y (2020) Dual-attention gan for large-pose face frontalization. arXiv preprint arXiv:2002.07227

  31. Zhu JY, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE international conference on computer vision, pp. 2223–2232

  32. Zhu X, Lei Z, Yan J, Yi D, Li SZ (2015) High-fidelity pose and expression normalization for face recognition in the wild. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 787–796

  33. Zhu Z, Luo P, Wang X, Tang X (2014) Recover canonical-view faces in the wild with deep neural networks. arXiv preprint arXiv:1404.3543

  34. Zou H, Zhang H, Li X, Liu J, He Z (2018) Generation textured contact lenses iris images based on 4dcycle-gan. In: 2018 24th international conference on pattern recognition (ICPR), IEEE, pp. 3561–3566

Download references

Acknowledgements

The authors gratefully acknowledge the funding received from CNSRT-Maroc (Centre National de la Recherche Scientifique et Technique) and the French government (Eiffel scholarship).

Author information

Authors and Affiliations

  1. Laboratoire LASTID, Département de Physique, Faculté des Sciences, Université Ibn Tofail, BP 133, 14000, Kenitra, Morocco

    M. Kas, Y. El-merabet & R. Messoussi

  2. CIAD UMR 7533, University Bourgogne Franche-Comté, UTBM, 25200, Montbéliard, France

    M. Kas & Y. Ruichek

Authors
  1. M. Kas
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Y. El-merabet
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Y. Ruichek
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. R. Messoussi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to M. Kas.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kas, M., El-merabet, Y., Ruichek, Y. et al. Generative adversarial networks for 2D-based CNN pose-invariant face recognition. Int J Multimed Info Retr 11, 639–651 (2022). https://doi.org/10.1007/s13735-022-00249-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13735-022-00249-2

Keywords

Search

Navigation