Abstract
Recent face recognition algorithm can achieve high accuracy when the tested face samples are frontal. However, when the face pose changes largely, the performance of existing methods drop drastically. Efforts on pose-robust face recognition are highly desirable, especially when each face class has only one frontal training sample. In this study, we propose a 2D face fitting-assisted 3D face reconstruction algorithm that aims at recognizing faces of different poses when each face class has only one frontal training sample. For each frontal training sample, a 3D face is reconstructed by optimizing the parameters of 3D morphable model (3DMM). By rotating the reconstructed 3D face to different views, pose virtual face images are generated to enlarge the training set of face recognition. Different from the conventional 3D face reconstruction methods, the proposed algorithm utilizes automatic 2D face fitting to assist 3D face reconstruction. We automatically locate 88 sparse points of the frontal face by 2D face-fitting algorithm. Such 2D face-fitting algorithm is so-called Random Forest Embedded Active Shape Model, which embeds random forest learning into the framework of Active Shape Model. Results of 2D face fitting are added to the 3D face reconstruction objective function as shape constraints. The optimization objective energy function takes not only image intensity, but also 2D fitting results into account. Shape and texture parameters of 3DMM are thus estimated by fitting the 3DMM to the 2D frontal face sample, which is a non-linear optimization problem. We experiment the proposed method on the publicly available CMUPIE database, which includes faces viewed from 11 different poses, and the results show that the proposed method is effective and the face recognition results toward pose variants are promising.














Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Bellhumer PN, Hespanha J, Kriegman D (1997) Eigenfaces vs. fisherfaces: recognition using class-specific linear projection. IEEE Trans Pattern Anal Mach Intell Spec Issue Face Recognit 17(7):711–720
Blanz V, Vetter T (1999) A morphable model for the synthesis of 3D faces. In: Proceedings of SIGGRAPH, pp 187–194
Blanz V, Vetter T (2003) Face recognition based on fitting a 3D morphable model. IEEE Trans Pattern Anal Mach Intell 25(9):1063–1074
Blanz V, Grother P, Phillips PJ, Vetter T (2005) Face recognition based on frontal views generated from nonfrontal images. IEEE Conf Comput Vis Pattern Recognit 2:454–461
Breuer P, Kim KI, Kienzle W, Scholkopf B, Blanz V (2008) Automatic 3D face reconstruction from single images or video. In: Automatic face and gesture recognition (FG) IEEE international conference, pp 1–8
Brunelli R, Poggio T (1993) Face recognition: features versus templates. IEEE Trans Pattern Anal Mach Intell 15(10):1042–1052
Cappelli R, Maio D, Maltoni D (2002) Subspace classification for face recognition. In: Proceeding workshop on biometric authentication, pp 133–141
Chai XJ, Shan SG, Qing LY, Chen XL, Gao W (2006) Pose and illumination invariant face recognition based on 3D face reconstruction. J Softw 17(3):525–534
Cootes TF, Taylor C, Cooper D, Graham J (1995) Active shape models: their training and their applications. Comput Vis Image Underst 61(1):38–59
Cootes TF, Edwards G, Taylor C (2001) Active Appearance Models. IEEE Trans Pattern Anal Mach Intell 23(6):681–685
Du S, Ward R (2006) Face recognition under pose variations. J Frankl Inst 343:596–613
Franco A, Maio D, Maltoni D (2008) 2D face recognition based on supervised subspace learning from 3D models. Pattern Recognit 41:3822–3833
Gonzalez-Jimenez D, Alba-Castro JL (2007) Face recognition through point distribution models and facial symmetry. IEEE Trans Inf Forensics Secur 2(3):413–429
Guo GD, Li SZ, Chan KL (2000) Face recognition by support vector machines. In: Proceedings of automatic face and gesture recognition, pp 196–201
Gurbuz S, Inoue N (2007) Real-time head pose estimation using reconstructed 3d face data from stereo image pair. Acoustics Speech and Signal Processing (ICASSP), IEEE international conference, vol 2, pp 785–788
Hu YX, Jiang DL, Yan SC, Zhang L, Zhang HJ (2004) Automatic 3D reconstruction for face recognition. In: Proceedings of IEEE automatic face and gesture recognition, pp 843–848
Huang C, Ai HZ, Li Y, Lao SH (2007) High performance rotation invariant multiview face detection. IEEE Trans Pattern Anal Mach Intell 29(4):671–686
Lepetit V, Fua P (2006) Keypoint recognition using randomized trees. IEEE Trans Pattern Anal Mach Intell, pp 1465–1479
Ma Y, Ding XQ, Wang ZE, Wang N (2004) Robust precise eye location under probabilistic framework. IEEE international conference on automatic face and gesture recognition, pp 339–344
Michael J, Jones T (1998) Multidimensional morphable models: a framework for representing and matching object classes. Int J Comput Vis 2(29):107–131
Multimedia, Intelligent Software Technology Beijing Municipal Key Laboratory BJUT (2005) The BJUT-3D large-scale chinese face database. Technical report
Phong BT (1975) Illumination for computer-generated images. Commun ACM 18(6):311–317
Suen CY, Langaroudi AZ, Feng CH, Mao YX (2007) A survey of techniques for face reconstruction. Systems, Man and Cybernetics (ISIC), IEEE international conference, pp 3554–3560
Sung WP, Heo JG, Savvides M (2008) 3D Face econstruction from a single 2D face image. Computer vision and pattern recognition workshops (CVPR workshops), IEEE computer society conference, pp 1–8
Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. Proc Comput Vis Pattern Recognit 1:511–518
Wang ZE, Ding XQ, Fang C (2008) Pose-adaptive lda-based face recognition. In: Proceedings of international conference on pattern recognition
Wiskott L, Fellous JM, Malsburg VD (1997) Face recognition by elastic bunch graph matching. IEEE Trans Pattern Anal Mach Intell 19:775–779
Zhao M, Chua TS, Sim T (2006) Morphable face reconstruction with multiple images. Automatic face and gesture recognition (FGR), International conference, pp 597–602
Zheng Y, Chang JL, Zheng ZG, Wang ZF (2007) 3D face reconstruction from stereo: a model-based approach. Image processing (ICIP), IEEE international conference, vol 3, pp 65–68
Acknowledgments
This work was supported by the National Basic Research Program of China (973 program) under Grant No. 2007CB311004, the National High Technology Research and Development Program of China (863 program) under Grant Nos. 2006AA01Z115 and 2006AA11Z213 and the National Natural Science Foundation of China under Grant Nos. 60772049, 60972094, and 60872086.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Wang, L., Ding, L., Ding, X. et al. 2D face fitting-assisted 3D face reconstruction for pose-robust face recognition. Soft Comput 15, 417–428 (2011). https://doi.org/10.1007/s00500-009-0523-0
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-009-0523-0