Abstract
3D face shape provides a pose and illumination invariant description of human faces. In this paper, we propose a novel component based method to recover the full 3D face shape from a set of sparse feature points. We use a local linear fitting (LLF) scheme so that reconstruction of each subregion depends on both its own vertices and adjacent subregions. This method results in a separate set of shape coefficients each emphasizing the quality of one subregion and improves the model expressiveness. Experiments show that the LLF strategy significantly reduces the model residual error, and thus reduces the sparse reconstruction error under pose variations. Moreover, the problem of estimating pose parameters is revisited, and we use a joint optimization method to improve the reconstruction quality under unknown pose. We evaluate the sensitivity of our method to the selection of feature points. Simulation results show that our method is more robust than prevailing methods.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Blanz, V., Vetter, T.: A morphable model for the synthesis of 3D faces. In: ACM SIGGRAPH’99, pp. 187–194 (1999)
Zhao, W.Y., Chellappa, R.: Symmetric shape-from-shading using self-ratio image. Int. J. Comput. Vis. 45, 55–75 (2001)
Patel, A., Smith, W.A.P.: Driving 3D morphable models using shading cues. Pattern Recognit. 45, 1993–2004 (2012)
Amberg, B., Blake, A., Fitzgibbon, A., Romdhani, S., Vetter, T.: Reconstructing high quality face-surfaces using model based stereo. In: Proc. Eleventh IEEE International Conference on Computer Vision (ICCV 2007), pp. 1–8 (2007)
Levine, M.D., Yu, Y.: State-of-the-art of 3D facial reconstruction methods for face recognition based on a single 2D training image per person. Pattern Recognit. Lett. 30, 908–913 (2009)
Paysan, P., Knothe, R., Amberg, B., Romdhani, S., Vetter, T.: A 3D face model for pose and illumination invariant face recognition. In: Proc. IEEE International Conference on Advanced Video and Signal Based Surveillance, pp. 296–301 (2009)
Romdhani, S., Vetter, T.: Estimating 3D shape and texture using pixel intensity, edges, specular highlights, texture constraints and a prior. In: Proc. 2005 IEEE International Conference on Computer Vision and Pattern Recognition (CVPR 2005), pp. 986–993 (2005)
Moghaddam, B., Lee, J., Pfister, H., Machiraju, R.: Model-based 3D face capture with shape-from-silhouettes. In: Proc. 2003 IEEE Workshop on Analysis and Modeling of Face and Gesture (AMFG 2003), pp. 20–27 (2003)
Hwang, B.-W., Blanz, V., Vetter, T., Lee, S.-W.: Face reconstruction from a small number of feature points. In: Proc. International Conference on Pattern Recognition 2000 (ICPR’00), pp. 838–841 (2000)
Blanz, V., Vetter, T.: Reconstructing the complete 3D shape of faces from partial information. It Ti, Inf.tech. Tech. Inform. 44(6), 295–302 (2002)
Blanz, V., Mehl, V., Vetter, T., Seidel, H.-P.: A statistical method for robust 3D surface reconstruction from sparse data. In: Proc. Second International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT 2004), pp. 293–300 (2004)
Jiang, D., Hu, Y., Yan, S., Zhang, L., Zhang, H., Gao, W.: Efficient 3D reconstruction for face recognition. Pattern Recognit. 38(6), 787–798 (2005)
Knothe, R., Romdhani, S., Vetter, T.: Combining PCA and LFA for surface reconstruction from a sparse set of control. In: Proc. Seventh IEEE International Conference on Automatic Face and Gesture Recognition (FGR 2006), pp. 637–644 (2006)
Patel, A., Smith, W.A.P.: 3D morphable face models revisited. In: Proc. 2009 IEEE International Conference on Computer Vision and Pattern Recognition (CVPR 2009), pp. 1327–1334 (2009)
Aldrian, O., Smith, W.A.P.: Learning the nature of generalization errors in a 3D morphable model. In: Proc. International Conference on Pattern Recognition 2010 (ICPR 2010), pp. 4557–4560 (2010)
Aldrian, O., Smith, W.A.P.: A linear approach of 3D face shape and texture recovery using a 3D morphable model. In: Proc. British Machine Vision Conference 2010 (BMVC 2010), pp. 75.1–75.10 (2010)
Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. Springer, New York (2009)
Ding, L., Ding, X.Q., Fang, C.: Continuous pose normalization for pose-robust face recognition. IEEE Signal Process. Lett. 19(11), 721–724 (2012)
Press, W.H., Teukolsky, S.A., Vetterling, W.T., et al.: Numerical Recipes in C: The Art of Scientific Computing. Cambridge University Press, Oxford (1992)
Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Oxford (2000)
Romdhani, S., Canterakis, N., Vetter, T.: Selective vs. global recovery of rigid and non-rigid motion. http://informatik.unibas.ch/personen/romdhani_sami/tr_sel_sfm.pdf (2003). Accessed 10 Jan 2012
Hamilton, W.R.: On quaternions, or on a new system of imaginaries in algebra. Philos. Mag. 25(3), 489–495 (1844)
Farkas, L.: Anthropometry of the Head and Face. Raven Press, New York (1994)
Logie, R., Baddeley, A., Woodhead, M.: Face recognition, pose and ecological validity. Appl. Cogn. Psychol. 1, 53–69 (1987)
Acknowledgements
This work is supported by the National Natural Science Foundation of China under Grant No. 60972094. The authors would like to thank Dr. Thomas Vetter for providing the BFM Database and Oswald Aldrian for detailed discussions on his paper. The authors also would like to thank the anonymous reviewers for their helpful suggestions for improving the quality of this paper.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Ding, L., Ding, X. & Fang, C. 3D face sparse reconstruction based on local linear fitting. Vis Comput 30, 189–200 (2014). https://doi.org/10.1007/s00371-013-0795-3
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00371-013-0795-3