Skip to main content
SpringerLink
Log in

Facial Shape-from-shading and Recognition Using Principal Geodesic Analysis and Robust Statistics

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

The aim in this paper is to use principal geodesic analysis to model the statistical variations for sets of facial needle maps. We commence by showing how to represent the distribution of surface normals using the exponential map. Shape deformations are described using principal geodesic analysis on the exponential map. Using ideas from robust statistics we show how this deformable model may be fitted to facial images in which there is significant self-shadowing. Moreover, we demonstrate that the resulting shape-from-shading algorithm can be used to recover accurate facial shape and albedo from real world images. In particular, the algorithm can effectively fill-in the facial surface when more than 30% of its area is subject to self-shadowing. To investigate the utility of the shape parameters delivered by the method, we conduct experiments with illumination insensitive face recognition. We present a novel recognition strategy in which similarity is measured in the space of the principal geodesic parameters. We also use the recovered shape information to generate illumination normalized prototype images on which recognition can be performed. Finally we show that, from a single input image, we are able to generate the basis images employed by a number of well known illumination-insensitive recognition algorithms. We also demonstrate that the principal geodesics provide an efficient parameterization of the space of harmonic basis images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • USF HumanID. 3D Face Database, Courtesy of Sudeep. Sarkar, University of South Florida, Tampa, FL.

  • Atick, J. J., Griffin, P. A., & Redlich, A. N. (1996). Statistical approach to SFS: Reconstruction of 3D face surfaces from single 2D images. Neural Computation, 8(6), 1321–1340.

    Article  Google Scholar 

  • Basri, R., & Jacobs, D. W. (2003). Lambertian reflectance and linear subspaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(2), 218–233.

    Article  Google Scholar 

  • Belhumeur, P.N., Kriegman, D.J., & Yuille, A.L. (1999). The bas-relief ambiguity. International Journal of Computer Vision, 35(1), 33–44.

    Article  Google Scholar 

  • Blanz, V., Basso, C., Poggio, T., & Vetter, T. (2003). Reanimating faces in images and video. In Proceedings of EUROGRAPHICS (pp. 641–650).

  • Blanz, V., & Vetter, T. (1999). A morphable model for the synthesis of 3D faces. In Proceedings of SIGGRAPH (pp. 187–194).

  • Blanz, V., & Vetter, T. (2003). Face recognition based on fitting a 3D morphable model. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(9), 1063–1074.

    Article  Google Scholar 

  • Bronstein, A., Bronstein, M., & Kimmel, R. (2005). Three-dimensional face recognition. International Journal of Computer Vision, 64(1), 5–30.

    Article  Google Scholar 

  • Castelán, M., & Hancock, E. R. (2006). Acquiring height data from a single image of a face using local shape indicators. Computer Vision and Image Understanding, 103(1), 64–79.

    Article  Google Scholar 

  • Dovgard, R., & Basri, R. (2004). Statistical symmetric shape from shading for 3D structure recovery of faces. In Proceedings of ECCV (Vol. 2, pp. 99–113).

  • Dupuis, P., & Oliensis, J. (1994). An optimal control formulation and related numerical methods for a problem in shape reconstruction. Annals of Applied Probability, 4(2), 287–346.

    MathSciNet  MATH  Google Scholar 

  • Finlayson, G. D., Drew, M. S., & Lu, C. (2004). Intrinsic images by entropy minimization. In Proceedings of ECCV (pp. 582–595).

  • Fisher, N. I. (1985). Spherical medians. Journal of the Royal Statistical Society, Series B, 47(2), 342–348.

    MathSciNet  MATH  Google Scholar 

  • Fletcher, P. T., Joshi, S., Lu, C., & Pizer, S. M. (2004). Principal geodesic analysis for the study of nonlinear statistics of shape. IEEE Transactions on Medical Imaging, 23(8), 995–1005.

    Article  Google Scholar 

  • Frankot, R. T., & Chellappa, R. (1988). A method for enforcing integrability in shape from shading algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence, 10(4), 439–451.

    Article  MATH  Google Scholar 

  • Fua, P., & Leclerc, Y. G. (1994). Using 3-dimensional meshes to combine image-based and geometry-based constraints. In Proceedings of ECCV (pp. 281–291).

  • Georghiades, A. (2003). Recovering 3-d shape and reflectance from a small number of photographs. In Eurographics symposium on rendering (pp. 230–240).

  • Georghiades, A. S., Belhumeur, P. N., & Kriegman, D. J. (2001). From few to many: Illumination cone models for face recognition under variable lighting and pose. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(6), 643–660.

    Article  Google Scholar 

  • Giles, P. T. (2001). Remote sensing and cast shadows in mountainous terrain. Photogrammetric Engineering & Remote Sensing, 67(7), 833–840.

    Google Scholar 

  • Gregory, R. L. (1997). Knowledge in perception and illusion. Philosophical Transactions of the Royal Society of London, Series B, 352, 1121–1128.

    Article  Google Scholar 

  • Hill, H., & Bruce, V. (1996). Effects of lighting on the perception of facial surfaces. Journal of Experimental Psychology: Human Perception and Performance, 22(4), 986–1004.

    Article  Google Scholar 

  • Huber, P. (1981). Robust statistics. Chichester: Wiley.

    MATH  Google Scholar 

  • Lee, K., Ho, J., & Kriegman, D. (2005). Acquiring linear subspaces for face recognition under variable lighting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(5), 1–15.

    Google Scholar 

  • Lenglet, C., Rousson, M., Deriche, R., Faugeras, O., Lehericy, S., & Ugurbil, K. (2005). A Riemannian approach to diffusion tensor images segmentation. In Proceedings of the information processing in medical imaging (pp. 591–602).

  • Levine, M. D., & Bhattacharyya, J. (2005). Removing shadows. Pattern Recognition Letters, 26(3), 251–265.

    Article  Google Scholar 

  • Mardia, K. V., & Jupp, P. E. (2000). Directional statistics. New York: Wiley.

    MATH  Google Scholar 

  • Marr, D. (1982). Vision. San Francisco: Freeman.

    Google Scholar 

  • Nishino, K., Belhumeur, P. N., & Nayar, S. K. (2005). Using eye reflections for face recognition under varying illumination. In Proceedings of ICCV (Vol. 1, pp. 519–526).

  • Pennec, X. (1999). Probabilities and statistics on Riemannian manifolds: basic tools for geometric measurements. In Proceedings of the IEEE workshop on nonlinear signal and image processing.

  • Pennec, X. (2004). Probabilities and statistics on Riemannian manifolds: a geometric approach (Technical Report RR-5093). INRIA.

  • Prados, E., & Faugeras, O. (2004). A rigorous and realistic shape from shading method and some of its applications (Technical Report RR-5133). INRIA.

  • Prados, E., & Faugeras, O. D. (2004). Unifying approaches and removing unrealistic assumptions in shape from shading: mathematics can help. In Proceedings of ECCV (pp. 141–154).

  • Samaras, D., & Metaxas, D. (2003). Illumination constraints in deformable models for shape and light direction estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(2), 247–264.

    Article  Google Scholar 

  • Smith, W. A. P., & Hancock, E. R. (2006). Recovering facial shape using a statistical model of surface normal direction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(12), 1914–1930.

    Article  Google Scholar 

  • Turk, M., & Pentland, A. (1991). Face recognition using eigenfaces. In Proceedings of CVPR (pp. 586–591).

  • Worthington, P. L., & Hancock, E. R. (1999). New constraints on data-closeness and needle map consistency for shape-from-shading. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(12), 1250–1267.

    Article  Google Scholar 

  • Zhang, L., Wang, S., & Samaras, D. (2005). Face synthesis and recognition under arbitrary unknown lighting using a spherical harmonic basis morphable model. In Proceedings of CVPR (pp. 209–216).

  • Zhao, W. Y., & Chellappa, R. (2000). Illumination-insensitive face recognition using symmetric SFS. In Proceedings of CVPR (pp. 286–293).

  • Zhao, W. Y., & Chellappa, R. (2001). Symmetric shape-from-shading using self-ratio image. International Journal of Computer Vision, 45, 55–75.

    Article  MATH  Google Scholar 

  • Zhou, S., & Chellappa, R. (2003). Rank constrained recognition under unknown illuminations. In Proceedings of the IEEE international workshop on analysis and modeling of faces and gestures (pp. 11–18).

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of York, York, YO10 5DD, UK

    William A. P. Smith & Edwin R. Hancock

Authors
  1. William A. P. Smith
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Edwin R. Hancock
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to William A. P. Smith.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Smith, W.A.P., Hancock, E.R. Facial Shape-from-shading and Recognition Using Principal Geodesic Analysis and Robust Statistics. Int J Comput Vis 76, 71–91 (2008). https://doi.org/10.1007/s11263-007-0074-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-007-0074-8

Keywords

Search

Navigation