Abstract
Face recognition is an important topic in the field of computer vision and has been a vital biometric technique for identity authentication. It is widely used in areas such as public security, military, and daily life. However, face recognition is inherently a challenging problem due to variations in poses, facial expressions, age, and occlusion. In this work, we propose a generative adversarial network (GAN) architecture that disentangles identity and pose variations to learn generative and discriminative representations for pose-invariant face recognition. We use an iterative warping scheme that achieves better results than with the use of a single generator. The features from the encoder are considered pose-invariant features for face recognition, and evaluations on databases demonstrate the usefulness of this approach over prior methods. For example, we report 97.0% (+ 12.7%) and 90.5% (+ 8.4%) accuracy on the Feret and Caspeal datasets compared to 78.2% achieved by the best method without warping. In particular, there are two notable novelties. First, the disentangled architecture GAN (D-GAN) performs frontal face synthesis via an encoder-decoder structure in the generator with the pose variations provided to the decoder and discriminator. Second, we utilize the generator encoder as a spatial transformer network that seeks realistic image synthesis in the geometric warp parameter space.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ashraf, A. B., Lucey, S., & Chen, T. (2008). Learning patch correspondences for improved viewpoint invariant face recognition. CVPR (pp. 1–8).
Asthana, A., Gedeon, T., Goecke, R., & Sanderson, C. (2009). Learning-based face synthesis for pose-robust recognition from single image. In British machine vision conference, BMVC (pp. 1–10).
Chai, X., Shan, S., Chen, X., & Gao, W. (2007). Locally linear regression for pose-invariant face recognition. IEEE Transactions on Image Processing, 16(7), 1716–1725.
Choi, S.-I., Lee, Y., & Lee, M. (2019). Face Recognition in SSPP problem using face relighting based on coupled bilinear model. Sensors, 19(1), 43.
Ding, C., & Tao, D. (2016). A comprehensive survey on pose-invariant face recognition. ACM Transactions on Intelligent Systems and Technology (TIST), 7(3), 37.
Gao, W., Cao, B., Shan, S., Chen, X., Zhou, D., Zhang, X., et al. (2008). The CAS-PEAL large-scale Chinese face database and baseline evaluations. IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, 38(1), 149–161.
Gao, Y., & Lee, H. J. (2015). Cross-pose face recognition based on multiple virtual views and alignment error. Pattern Recognition Letters, 65, 170–176.
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2014). Generative adversarial nets (pp. 2672–2680).
Hassner, T., Harel, S., Paz, E., & Enbar, R. (2015). Effective face frontalization in unconstrained images (pp. 4295–4304).
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017). Image-to-image translation with conditional adversarial networks.
Jaderberg, M., Simonyan, K., & Zisserman, A. (2015). Spatial transformer networks (pp. 2017–2025).
Ke, Y., & Sukthankar, R. (2014). PCA-SIFT: A more distinctive representation for local image descriptors. CVPR.
Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. Preprint arXiv:1412.6980.
Lin, C.-H., & Lucey, S. (2016). Inverse compositional spatial transformer networks. Preprint arXiv:1612.03897.
Lin, C.-H., Yumer, E., Wang, O., Shechtman, E., & Lucey, S. (2018). ST-GAN: Spatial transformer generative adversarial networks for image compositing (pp. 9455–9464).
Liu, X., & Chen, T. (2005). Pose-robust face recognition using geometry assisted probabilistic modeling (pp. 502–509).
Liu, X., Rittscher, J., & Chen, T. (2006). Optimal pose for face recognition (pp. 1439–1446).
Liu, Z., Zhang, L., Pu, J., Liu, G., & Liu, S. (2019). Using the original and symmetrical face test samples to perform two-step collaborative representation for face recognition. International Journal of Pattern Recognition and Artificial Intelligence, 33(02), 1956001.
Masi, I., Rawls, S., Medioni, G., & Natarajan, P. (2016). Pose-aware face recognition in the wild (pp. 4838–4846).
Pang, M., Cheung, Y.-M., Wang, B., & Liu, R. (2019). Robust heterogeneous discriminative analysis for face recognition with single sample per person. Pattern Recognition.
Parkhi, O. M., Vedaldi, A., & Zisserman, A. (2015). Deep face recognition (p. 6).
Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., & Efros, A. A. (2016). Context encoders: Feature learning by inpainting (pp. 2536–2544).
Phillips, P. J., Moon, H., Rizvi, S. A., & Rauss, P. J. (2000). The FERET evaluation methodology for face-recognition algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(10), 1090–1104.
Radford, A., Metz, L., & Chintala, S. (2015). Unsupervised representation learning with deep convolutional generative adversarial networks. Preprint arXiv:1511.06434.
Sagonas, C., Panagakis, Y., Zafeiriou, S., & Pantic, M. (2015). Robust statistical face frontalization (pp. 3871–3879).
Schroff, F., Kalenichenko, D., & Philbin, J. (2015). Facenet: A unified embedding for face recognition and clustering (pp. 815–823).
Sengupta, S., Chen, J.-C., Castillo, C., Patel, V. M., Chellappa, R., & Jacobs, D. W. (2016). Frontal to profile face verification in the wild (pp. 1–9).
Sharma, A., & Jacobs, D. W. (2011). Bypassing synthesis: PLS for face recognition with pose, low-resolution and sketch. CVPR (pp. 593–600).
Shi, S., Ji, R., & Zhang, F. (2018). Multi-pose facial correction based on Gaussian process with combined kernel function. In Ninth International Conference on Graphic and Image Processing (ICGIP) (p. 106150X).
Tran, L., Yin, X., & Liu, X. (2017). Disentangled representation learning gan for pose-invariant face recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1415–1424).
Wang, M., & Deng, W. (2018). Deep face recognition: A survey. Preprint arXiv:1804.06655.
Yi, D., Lei, Z., Liao, S., & Li, S. Z. (2014). Learning face representation from scratch. Preprint arXiv:1411.7923.
Yim, J., Jung, H., Yoo, B., Choi, C., Park, D., & Kim, J. (2015). Rotating your face using multi-task deep neural network (pp. 676–684).
Yin, X., & Liu, X. (2017). Multi-task convolutional neural network for pose-invariant face recognition. IEEE Transactions on Image Processing.
Zhao, W., Chellappa, R., Phillips, P. J., & Rosenfeld, A. (2003). Face recognition: A literature survey. ACM Computing Surveys (CSUR), 35(4), 399–458.
Zhong, F., Yan, S., Liu, L., & Liu, K. (2018). An effective face recognition framework with subspace learning based on local texture patterns. In 2018 5th international conference on systems and informatics (ICSAI). IEEE (pp. 266–271).
Zhu, X., Lei, Z., Yan, J., Yi, D., & Li, S. Z. (2015). High-fidelity pose and expression normalization for face recognition in the wild (pp. 787–796).
Zhu, Z., Luo, P., Wang, X., & Tang, X. (2014). Multi-view perceptron: a deep model for learning face identity and view representations (pp. 217–225).
Acknowledgements
This study has been supported in part by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (GR2019R1D1A3A03103736).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Chikontwe, P., Gao, Y. & Lee, H.J. Transformation guided representation GAN for pose invariant face recognition. Multidim Syst Sign Process 32, 633–649 (2021). https://doi.org/10.1007/s11045-020-00752-x
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11045-020-00752-x