Skip to main content
SpringerLink
Log in

Pose-and-illumination-invariant face representation via a triplet-loss trained deep reconstruction model

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Face recognition under variable pose and illumination is a challenging problem in computer vision tasks. In this paper, we solve this problem by proposing a new residual based deep face reconstruction neural network to extract discriminative pose-and-illumination-invariant (PII) features. Our deep model can change arbitrary pose and illumination face images to the frontal view with standard illumination. We propose a new triplet-loss training method instead of Euclidean loss to optimize our model, which has two advantages: a) The training triplets can be easily augmented by freely choosing combinations of labeled face images, in this way, overfitting can be avoided; b) The triplet-loss training makes the PII features more discriminative even when training samples have similar appearance. By using our PII features, we achieve 83.8% average recognition accuracy on MultiPIE face dataset which is competitive to the state-of-the-art face recognition methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Ahonen T, Hadid A, Pietikainen M (2006) Face description with local binary patterns: application to face recognition. IEEE Trans Pattern Anal Mach Intell 28:2037–2041

    Article  MATH  Google Scholar 

  2. Asthana A, Marks TK, Jones MJ, Tieu KH, Rohith M (2011) Fully automatic pose-invariant face recognition via 3d pose normalization. In: International conference on computer vision (ICCV), IEEE

  3. Belhumeur P, Hespanha J, Kriegman D (1997) Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19(7):711–720

    Article  Google Scholar 

  4. Castillo CD, Jacobs DW (2011) Wide-baseline stereo for face recognition with large pose variation. In: Conference on computer vision and pattern recognition (CVPR), IEEE

  5. Chen D, Cao X, Wen F, Sun J (2013) Blessing of dimensionality: high-dimensional feature and its efficient compression for face verification. In: Conference on computer vision and pattern recognition (CVPR), IEEE

  6. Gross R, Matthews I, Cohn J, Kanade T, Baker S (2008) Multi-pie. In: International Conference on Automatic Face and Gesture Recognition

  7. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Conference on computer vision and pattern recognition (CVPR), IEEE

  8. Huang G B, Lee H, Learned-Miller E (2012) Learning hierarchical representations for face verification with convolutional deep belief networks. In: Conference on computer vision and pattern recognition (CVPR), IEEE

  9. Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International conference on Machine learning (ICML)

  10. Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, Darrell T (2014) Caffe: convolutional architecture for fast feature embedding. In: International conference on multimedia. ACM, 2014

  11. Kan M, Shan S, Chang H, Chen X (2014) Stacked progressive auto-encoders (spae) for face recognition across poses. In: Conference on computer vision and pattern recognition (CVPR), IEEE

  12. Krizhevsky A, Sutskever I, Hinton G (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems (NIPS)

  13. Li S, Liu X, Chai X, Zhang H, Lao S, Shan S (2012) Morphable displacement field based image matching for face recognition across pose. In: European conference on computer vision (ECCV), Springer

  14. Li A, Shan S, Gao W (2012) Coupled bias–variance tradeoff for cross-pose face recognition. IEEE Trans Image Process 21(1):305–315

    Article  MathSciNet  Google Scholar 

  15. Liu C, Wechsler H (2002) Gabor feature based classification using the enhanced fisher linear discriminant model for face recognition. Trans Image Process 11:467–476

    Article  Google Scholar 

  16. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Conference on computer vision and pattern recognition (CVPR), IEEE

  17. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60:91–110

    Article  Google Scholar 

  18. Maaten LVD, Hinton G (2008) Visualizing data using t-sne. J Mach Learn Res 9(2579–2605):85

    MATH  Google Scholar 

  19. Mathias M., Benenson R., Pedersoli M., Van Gool L (2014) Face detection without bells and whistles. In: European conference on computer vision (ECCV), Springer

  20. Parkhi OM, Vedaldi A, Zisserman A (2015) Deep face recognition. In: British Machine Vision Conference (BMVC)

  21. Schroff F, Treibitz T, Kriegman D, Belongie S (2011) Pose, illumination and expression invariant pairwise face-similarity measure via doppelg¨anger list comparison. In: International conference on computer vision (ICCV), IEEE

  22. Schroff F, Kalenichenko D, Philbin J (2015) Facenet: a unified embedding for face recognition and clustering. In Conference on computer vision and pattern recognition (CVPR), IEEE

  23. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556

  24. Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations

  25. Srivastava N, Hinton GE, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958

    MathSciNet  MATH  Google Scholar 

  26. Sun Y, Wang X, Tang X (2013) Deep convolutional network cascade for facial point detection. In: Conference on computer vision and pattern recognition (CVPR), IEEE

  27. Sun Y, Wang X, Tang X (2014) Deep learning face representation from predicting 10, 000 classes. In Conference on computer vision and pattern recognition (CVPR), IEEE

  28. Sun Y, Wang X, Tang X (2015) Deeply learned face representations are sparse, selective, and robust. In: Conference on computer vision and pattern recognition (CVPR), IEEE

  29. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Conference on computer vision and pattern recognition (CVPR), IEEE

  30. Taigman Y, Yang M, Ranzato MA, Wolf L (2014) Deep-face: closing the gap to human-level performance in face verification. In: Conference on computer vision and pattern recognition (CVPR), IEEE

  31. Taigman Y, Yang M, Ranzato MA, Wolf L (2015) Web-scale training for face identification. In: Conference on computer vision and pattern recognition (CVPR), IEEE

  32. Tang X, Wang X (2004) Face sketch recognition. IEEE Trans Circuits and Syst Video Technol 14(1):50–57

    Article  Google Scholar 

  33. Wen Y, Zhang K, Li Z, Qiao Yu (2016) A discriminative feature learning approach for deep face recognition. In: European conference on computer vision (ECCV), Springer

  34. Yim J, Jung H, Yoo B, Choi C, Park D, Kim J (2015) Rotating your face using multi-task deep neural network. In: conference on computer vision and pattern recognition (CVPR), IEEE

  35. Zhang W, Wang X, Tang X (2011) Coupled information-theoretic encoding for face photo-sketch recognition. In: conference on computer vision and pattern recognition (CVPR), IEEE

  36. Zhu Z, Luo P, Wang X, and Tang X (2013) Deep learning identity-preserving face space. In: International conference on computer vision (ICCV), IEEE

  37. Zhu Z, Luo P, Wang X, Tang X (2014) Multi-view perceptron: a deep model for learning face identity and view representations. In: Advances in neural information processing systems (NIPS)

Download references

Acknowledgements

This work was supported in part by the National Key Research and Development Program of China under grant No.2016YFB1000903, and NSFC No.61573268.

Author information

Authors and Affiliations

  1. Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, Xi’an, People’s Republic of China

    Xingyu Chen, Xuguang Lan, Guoqiang Liang, Jianyi Liu & Nanning Zheng

Authors
  1. Xingyu Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xuguang Lan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guoqiang Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jianyi Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Nanning Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Xingyu Chen or Xuguang Lan.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, X., Lan, X., Liang, G. et al. Pose-and-illumination-invariant face representation via a triplet-loss trained deep reconstruction model. Multimed Tools Appl 76, 22043–22058 (2017). https://doi.org/10.1007/s11042-017-4782-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-4782-y

Keywords

Search

Navigation