Abstract
2D intensity images and 3D shape models are both useful for face recognition, but in different ways. While algorithms have long been developed using 2D or 3D data, recently has seen work on combining both into multi-modal face biometrics to achieve higher performance. However, the fusion of the two modalities has mostly been at the decision level, based on scores obtained from independent 2D and 3D matchers.
In this paper, we propose a systematic framework for fusing 2D and 3D face recognition at both feature and decision levels, by exploring synergies of the two modalities at these levels. The novelties are the following. First, we propose to use Local Binary Pattern (LBP) features to represent 3D faces and present a statistical learning procedure for feature selection and classifier learning. This leads to a matching engine for 3D face recognition. Second, we propose a statistical learning approach for fusing 2D and 3D based face recognition at both feature and decision levels. Experiments show that the fusion at both levels yields significantly better performance than fusion at the decision level.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Turk, M.A., Pentland, A.P.: Eigenfaces for recognition. Journal of Cognitive Neuroscience 3, 71–86 (1991)
Belhumeur, P.N., Hespanha, J.P., Kriegman, D.J.: Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence 19, 711–720 (1997)
Bartlett, M.S., Lades, H.M., Sejnowski, T.J.: Independent component representations for face recognition. In: Proceedings of the SPIE, Conference on Human Vision and Electronic Imaging III, vol. 3299, pp. 528–539 (1998)
Lades, M., Vorbruggen, J., Buhmann, J., Lange, J., von der Malsburg, C., Wurtz, R.P., Konen, W.: Distortion invariant object recognition in the dynamic link architecture. IEEE Transactions on Computers 42, 300–311 (1993)
Wiskott, L., Fellous, J., Kruger, N., van der Malsburg, C.: Face recognition by elastic bunch graph matching. IEEE Transactions on Pattern Analysis and Machine Intelligence 19, 775–779 (1997)
Ojala, T., Pietikainen, M., Harwood, D.: A comparative study of texture measures with classification based on feature distributions. Pattern Recognition 29, 51–59 (1996)
Ahonen, T., Hadid, A., Pietikainen, M.: Face recognition with local binary patterns. In: Proceedings of the European Conference on Computer Vision, Prague, Czech, pp. 469–481 (2004)
Hadid, A., Pietikinen, M., Ahonen, T.: A discriminative feature space for detecting and recognizing faces. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 797–804 (2004)
Jones, M., Viola, P.: Face recognition using boosted local features. Tech. Report TR2003-025, MERL (2003)
Zhang, G., Huang, X., Li, S.Z., Wang, Y.: Boosting local binary pattern (LBP)-based face recognition. In: Li, S.Z., Lai, J.-H., Tan, T., Feng, G.-C., Wang, Y. (eds.) SINOBIOMETRICS 2004. LNCS, vol. 3338, pp. 180–187. Springer, Heidelberg (2004)
Kanade, T.: Picture Processing by Computer Complex and Recognition of Human Faces. PhD thesis, Kyoto University (1973)
Cartoux, J.Y., LaPreste, J.T., Richetin, M.: Face authentication or recognition by profile extraction from range images. In: Proceedings of the Workshop on Interpretation of 3D Scenes (1989)
Besl, P.J., Jain, R.C.: Intrinsic and extrinsic surface characteristics. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, California, pp. 226–233 (1985)
Bowyer, K.W., Chang, K., Flynn, P.: A survey of 3D and multi-modal 3d+2d face recognition. In: Proceedings of International Conference Pattern Recognition, pp. 358–361 (2004)
Lu, X., Jain, A.K.: Integrating range and texture information for 3d face recognition. In: Proc. 7th IEEE Workshop on Applications of Computer Vision (WACV 2005), Breckenridge, CO. (2005)
Tsalakanidou, F., Malassiotis, S., Strintzis, M.G.: Face localization and authentication using color and depth images 14, 152–168 (2005)
Chang, K.I., Bowyer, K.W., Flynn, P.J.: An evaluation of multi-modal 2D+3D face biometrics. IEEE Transactions on Pattern Analysis and Machine Intelligence (2005) (to appear)
Freund, Y., Schapire, R.: A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences 55, 119–139 (1997)
Schapire, R.: A brief introduction to boosting. In: Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence (1999)
Viola, P., Jones, M.: Robust real time object detection. In: IEEE ICCV Workshop on Statistical and Computational Theories of Vision, Vancouver, Canada (2001)
Moghaddam, B., Nastar, C., Pentland, A.: A Bayesain similarity measure for direct image matching. Media Lab Tech. Report No.393, MIT (1996)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Li, S.Z., Zhao, C., Ao, M., Lei, Z. (2005). Learning to Fuse 3D+2D Based Face Recognition at Both Feature and Decision Levels. In: Zhao, W., Gong, S., Tang, X. (eds) Analysis and Modelling of Faces and Gestures. AMFG 2005. Lecture Notes in Computer Science, vol 3723. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11564386_5
Download citation
DOI: https://doi.org/10.1007/11564386_5
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-29229-6
Online ISBN: 978-3-540-32074-6
eBook Packages: Computer ScienceComputer Science (R0)