Skip to main content
SpringerLink
Log in

RGB-D Face Recognition: A Comparative Study of Representative Fusion Schemes

  • Conference paper
  • First Online:
Biometric Recognition (CCBR 2018)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 10996))

Included in the following conference series:

Abstract

RGB-D face recognition (FR) has drawn increasing attention in recent years with the advances of new RGB-D sensing technologies, and the decrease in sensor price. While a number of multi-modality fusion methods are available in face recognition, there is not known conclusion how the RGB and depth should be fused. We provide a comparative study of four representative fusion schemes in RGB-D face recognition, covering signal-level, feature-level, score-level fusions, and a hybrid fusion we designed for RGB-D face recognition. The proposed method achieves state-of-the-art performance on two large RGB-D datasets. A number of insights are provided based on the experimental evaluations.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    We also tried AlexNet [18], GoogLeNet [17], and VGG-16 [19], but the best performance of the three model for RGB and depth fusion is \(96.8\%\), which is lower than our ResNet-80 (\(98.7\%\)). So we only report the results using our ResNet-80.

  2. 2.

    https://github.com/seetaface/SeetaFaceEngine.

References

  1. Goswami, G., Bharadwaj, S., Vatsa, M., Singh, R.: On RGB-D face recognition using kinect. In: Proceedings of BTAS, pp. 1–6 (2013)

    Google Scholar 

  2. Goswami, G., Vatsa, M., Singh, R.: RGB-D face recognition with texture and attribute features. IEEE Trans. Inf. Forensics Secur. 9(10), 1629–1640 (2014)

    Article  Google Scholar 

  3. Lee, Y., Chen, J., Tseng, C., Lai, S.: Accurate and robust face recognition from RGB-D images with a deep learning approach. In: Proceedings of BMVC, pp. 123.1–123.14 (2016)

    Google Scholar 

  4. Wang, Z., Lu, J., Lin, R., Feng, J., Zhou, J.: Correlated and individual multi-modal deep learning for RGB-D object recognition, in arXiv:1604.01655 (2016)

  5. Song, X., Jiang, S., Herranz, L.: Combining models from multiple sources for RGB-D scene recognition. In: Proceedings of IJCAI, pp. 4523–4529 (2017)

    Google Scholar 

  6. Eitel, A., Springenberg, J., Spinello, L., Riedmiller, M., Burgard, W.: Multimodal deep learning for robust RGB-D object recognition. In: Proceedings of IROS, pp. 681–687 (2015)

    Google Scholar 

  7. Ren, L., Lu, J., Feng, J., Zhou, J.: Multi-modal uniform deep learning for RGB-D person re-identificaiton. Pattern Recogn. 72(12), 446–457 (2017)

    Article  Google Scholar 

  8. Socher, R., Huval, B., Bath, B.: Convolutional-recursive deep learning for 3D object classification. In: Proceedings of NIPS, pp. 665–673 (2012)

    Google Scholar 

  9. Zhu, H., Weibel, J., Lu, S.: Discriminative multi-modal feature fusion for RGBD indoor scene recognition. In: Proceedings of CVPR, pp. 2969–2976 (2016)

    Google Scholar 

  10. Zhang, H., Han, H., Cui, J., Shan, S., Chen, X.: RGB-D face recognition via deep complementary and common feature learning. In: Proceedings of FG, pp. 1–8 (2018)

    Google Scholar 

  11. Zhang, J., Huang, D., Wang, Y., Sun, J.: Lock3DFace: a large-scale database of low-cost kinect 3D faces. In: Proceedings of ICB, pp. 1–8 (2016)

    Google Scholar 

  12. Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: Proceedings of ICCV, pp. 839–846 (1998)

    Google Scholar 

  13. Jain, A.K., Nandakumar, K., Ross, A.: Score normalization in multimodal biometric systems. Pattern Recogn. 38(12), 2270–2285 (2005)

    Article  Google Scholar 

  14. Guo, Y., Zhang, L., Hu, Y., He, X., Gao, J.: MS-Celeb-1M: a dataset and benchmark for large-scale face recognition. In: Proceedings of ECCV (2016)

    Google Scholar 

  15. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition, in arXiv:1512.03385 (2015)

  16. Min, R., Kose, N., Dugelay, J.: KinectFaceDB: a kinect database for face recognition. IEEE Trans. SMC Syst. 44(11), 1534–1548 (2014)

    Google Scholar 

  17. Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Proceedings of ICML, pp. 448–456 (2015)

    Google Scholar 

  18. Krizhevsky, A., Sutskever, I., Hinton, G.: ImageNet classification with deep convolutional neural networks. In: Proceedings of NIPS, pp. 1097–1105 (2012)

    Google Scholar 

  19. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition, in arXiv:1409.1556 (2015)

  20. Han, H., Jain, A.K.: 3D face texture modeling from uncalibrated frontal and profile images. In: Proceedings of BTAS, pp. 223–230 (2012)

    Google Scholar 

  21. Cui, J., Zhang, H., Han, H., Shan, S., Chen, X.: Improving 2D face recognition via discriminative face depth estimation. In: Proceedings of ICB, pp. 1–8 (2018)

    Google Scholar 

Download references

Acknowledgement

This research was supported in part by the Natural Science Foundation of China (grants 61732004, and 61672496), External Cooperation Program of Chinese Academy of Sciences (CAS) (grant GJHZ1843), and Youth Innovation Promotion Association CAS (2018135).

Author information

Authors and Affiliations

  1. Key Laboratory of Intelligent Information Processing of Chinese Academy of Sciences (CAS), Institute of Computing Technology, CAS, Beijing, 100190, China

    Jiyun Cui, Hu Han, Shiguang Shan & Xilin Chen

  2. University of Chinese Academy of Sciences, Beijing, 100049, China

    Jiyun Cui, Shiguang Shan & Xilin Chen

Authors
  1. Jiyun Cui
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hu Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shiguang Shan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xilin Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hu Han .

Editor information

Editors and Affiliations

  1. Tsinghua University, Beijing, China

    Jie Zhou

  2. Beihang University, Beijing, China

    Yunhong Wang

  3. Chinese Academy of Sciences, Beijing, China

    Zhenan Sun

  4. Xinjiang University, Urumqi, China

    Zhenhong Jia

  5. Tsinghua University, Beijing, China

    Jianjiang Feng

  6. Chinese Academy of Sciences, Beijing, China

    Shiguang Shan

  7. Xinjiang University, Urumqi, China

    Kurban Ubul

  8. Tsinghua University, Shenzhen, China

    Zhenhua Guo

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Cui, J., Han, H., Shan, S., Chen, X. (2018). RGB-D Face Recognition: A Comparative Study of Representative Fusion Schemes. In: Zhou, J., et al. Biometric Recognition. CCBR 2018. Lecture Notes in Computer Science(), vol 10996. Springer, Cham. https://doi.org/10.1007/978-3-319-97909-0_39

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-97909-0_39

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-97908-3

  • Online ISBN: 978-3-319-97909-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation