Skip to main content
SpringerLink
Log in

Estimating Facial Reflectance Properties Using Shape-from-Shading

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

In this paper we show how to estimate facial surface reflectance properties (a slice of the BRDF and the albedo) in conjunction with the facial shape from a single image. The key idea underpinning our approach is to iteratively interleave the two processes of estimating reflectance properties based on the current shape estimate and updating the shape estimate based on the current estimate of the reflectance function. For frontally illuminated faces, the reflectance properties can be described by a function of one variable which we estimate by fitting a curve to the scattered and noisy reflectance samples provided by the input image and estimated shape. For non-frontal illumination, we fit a smooth surface to the scattered 2D reflectance samples. We make use of a novel statistical face shape constraint which we term ‘model-based integrability’ which we use to regularise the shape estimation. We show that the method is capable of recovering accurate shape and reflectance information from single grayscale or colour images using both synthetic and real world imagery. We use the estimated reflectance measurements to render synthetic images of the face in varying poses. To synthesise images under novel illumination, we show how to fit a parametric model of reflectance to the estimated reflectance function.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Ahmed, A. H., & Farag, A. A. (2006). A new formulation for shape from shading for non–Lambertian surfaces. In Proc. CVPR (Vol. 2, pp. 1817–1824).

  • Atick, J. J., Griffin, P. A., & Redlich, A. N. (1996). Statistical approach to SFS: Reconstruction of 3D face surfaces from single 2D images. Neural Computation, 8(6), 1321–1340.

    Article  Google Scholar 

  • Atkinson, G., & Hancock, E. R. (2008). Two-dimensional BRDF estimation from polarisation. Computer Vision and Image Understanding, 111(2), 126–141.

    Article  Google Scholar 

  • Baranoski, G. V. G., & Krishnaswamy, A. (2004). An introduction to light interaction with human skin. Revista de Informá, tica Teórica e Aplicada, XI(1), 33–62.

    Google Scholar 

  • Basri, R., & Jacobs, D. W. (2003). Lambertian reflectance and linear subspaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(2), 218–233.

    Article  Google Scholar 

  • Blanz, V., & Vetter, T. (2003). Face recognition based on fitting a 3D morphable model. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(9), 1063–1074.

    Article  Google Scholar 

  • Claridge, E., Cotton, S., Hall, P., & Moncrieff, M. (2003). From colour to tissue histology: physics based interpretation of images of pigmented skin lesions. Medical Image Analysis Journal, 7, 489–502.

    Article  Google Scholar 

  • Cootes, T. F., Edwards, G. J., & Taylor, C. J. (1998). Active appearance models. In Proc. ECCV (pp. 484–498).

  • Dana, K. J., van Ginneken, B., Nayar, S. K., & Koenderink, J. J. (1999). Reflectance and texture of real–world surfaces. ACM Transactions on Graphics, 18(1), 1–34.

    Article  Google Scholar 

  • Debevec, P., Hawkins, T., Tchou, C., Duiker, H.-P., & Sarokin, W. (2000). Acquiring the reflectance field of a human face. In Proc. SIGGRAPH (2000).

  • D’Errico, J. (2005). Surface fitting using gridfit. MATLAB Central File Exchange.

  • Donner, C., & Wann Jensen, H. (2006). A spectral BSSRDF for shading human skin. In Rendering techniques 2006: 17th eurographics workshop on rendering (pp. 409–418).

  • Dovgard, R., & Basri, R. (2004). Statistical symmetric shape from shading for 3D structure recovery of faces. In Proc. ECCV (Vol. 2, pp. 99–113).

  • Durou, J.-D., Falcone, M., & Sagona, M. (2008). Numerical methods for shape-from-shading: A new survey with benchmarks. Computer Vision and Image Understanding, 109(1), 22–43.

    Article  Google Scholar 

  • Frankot, R. T., & Chellappa, R. (1988). A method for enforcing integrability in shape from shading algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence, 10(4), 439–451.

    Article  MATH  Google Scholar 

  • Fuchs, M., Blanz, V., Lensch, H., & Seidel, H.-P. (2005). Reflectance from images: A model–based approach for human faces. IEEE Transactions on Visualization and Computer Graphics, 11(3), 296–305.

    Article  Google Scholar 

  • Georghiades, A. (2003). Recovering 3–d shape and reflectance from a small number of photographs. In Eurographics symposium on rendering (pp. 230–240).

  • Georghiades, A. S., Belhumeur, P. N., & Kriegman, D. J. (2001). From few to many: Illumination cone models for face recognition under variable lighting and pose. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(6), 643–660.

    Article  Google Scholar 

  • Hertzmann, A., & Seitz, S. M. (2005). Example–based photometric stereo: Shape reconstruction with general, varying BRDFs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(8), 1254–1264.

    Article  Google Scholar 

  • Horn, B. K. P., & Sjoberg, R. W. (1979). Calculating the reflectance map. Applied Optics, 18(11), 1770–1779.

    Article  Google Scholar 

  • Kemelmacher, I., & Basri, R. (2006). Molding face shapes by example. In Proc. ECCV (pp. 277–288).

  • Koenderink, J., & Pont, S. (2003). The secret of velvety skin. Machine Vision and Applications, 14, 260–268.

    Article  Google Scholar 

  • Marschner, S. R., Westin, S. H., Lafortune, E. P. F., Torrance, K. E., & Greenberg, D. P. (1999). Reflectance measurements of human skin (Technical Report PCG–99–2). Cornell University.

  • Oliensis, J. (1991). Uniqueness in shape from shading. International Journal of Computer Vision, 6(2), 75–104.

    Article  Google Scholar 

  • Oren, M., & Nayar, S. K. (1995). Generalization of the Lambertian model and implications for machine vision. International Journal of Computer Vision, 14(3), 227–251.

    Article  Google Scholar 

  • Phong, B. T. (1975). Illumination for computer generated images. Communications of the ACM, 18(6), 311–317.

    Article  Google Scholar 

  • Prados, E., & Faugeras, O. (2005). A generic and provably convergent shape-from-shading method for orthographic and pinhole cameras. International Journal of Computer Vision, 65(1–2), 97–125.

    Article  Google Scholar 

  • Ragheb, H., & Hancock, E. R. (2003). A probabilistic framework for specular shape–from–shading. Pattern Recognition, 36(2), 407–427.

    Article  Google Scholar 

  • Robles-Kelly, A., & Hancock, E. R. (2005). Estimating the surface radiance function from single images. Graphical Models, 67(6), 518–548.

    Article  MATH  Google Scholar 

  • Sim, T., Baker, S., & Bsat, M. (2003). The CMU pose, illumination, and expression database. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(12), 1615–1618.

    Article  Google Scholar 

  • Smith, W. A. P., & Hancock, E. R. (2007). A new framework for grayscale and colour non-Lambertian shape-from-shading. In Proc. ACCV (pp. 869–880).

  • Smith, W. A. P., & Hancock, E. R. (2008). Facial shape–from–shading and recognition using principal geodesic analysis and robust statistics. International Journal of Computer Vision, 76(1), 71–91.

    Article  Google Scholar 

  • Torrance, K., & Sparrow, E. (1967). Theory for off–specular reflection from roughened surfaces. Journal of the Optical Society of America, 57(9), 1105–1114.

    Article  Google Scholar 

  • USF HumanID 3D Face Database (1998). Courtesy of Sudeep Sarkar, University of South Florida, Tampa, FL.

  • van Gemert, M. J. C., Jacques, S. L., Sterenborg, H. J. C. M., & Star, W. M. (1989). Skin optics. IEEE Transactions on Biomedical Engineering, 36(12), 1146–1154.

    Article  Google Scholar 

  • Weyrich, T., Matusik, W., Pfister, H., Bickel, B., Donner, C., Tu, C., McAndless, J., Lee, J., Ngan, A., Wann Jensen, H., & Gross, M. (2006). Analysis of human faces using a measurement-based skin reflectance model. ACM Transactions on Graphics, 25(3), 1013–1024.

    Article  Google Scholar 

  • Zhang, L., & Samaras, D. (2006). Face recognition from a single training image under arbitrary unknown lighting using spherical harmonics. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(3), 351–363.

    Article  Google Scholar 

  • Zhang, R., Tsai, P. S., Cryer, J. E., & Shah, M. (1999). Shape–from–shading: a survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(8), 690–706.

    Article  Google Scholar 

  • Zhao, W. Y., & Chellappa, R. (2001). Symmetric shape–from–shading using self–ratio image. International Journal of Computer Vision, 45, 55–75.

    Article  MATH  Google Scholar 

  • Zickler, T., Mallick, S. P., Kriegman, D. J., & Belhumeur, P. N. (2008). Color subspaces as photometric invariants. International Journal of Computer Vision, 79(1), 13–30

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of York, York, YO10 5DD, UK

    William A. P. Smith & Edwin R. Hancock

Authors
  1. William A. P. Smith
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Edwin R. Hancock
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to William A. P. Smith.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Smith, W.A.P., Hancock, E.R. Estimating Facial Reflectance Properties Using Shape-from-Shading. Int J Comput Vis 86, 152–170 (2010). https://doi.org/10.1007/s11263-008-0175-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-008-0175-z

Keywords

Search

Navigation