Skip to main content
SpringerLink
Log in

Uncalibrated flatfielding and illumination vector estimationfor photometric stereo face reconstruction

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Within the context of photometric stereo reconstruction, flatfielding may be used to compensate for the effect of the inverse-square law of light propagation on the pixel brightness. This would require capturing a set of reference images at an off-line imaging session, which employs a calibrating device that should be captured under the exact conditions as the main session. Similarly, the illumination vectors, on which photometric stereo relies, are typically precomputed based on another dedicated calibration session. In practice, implementing such off-line sessions is inconvenient and often infeasible. This work aims at enabling accurate photometric stereo reconstruction for the case of non-interactive on-line capturing of human faces. We propose unsupervised methodologies, which extract all information that is required for accurate face reconstruction from the images of interest themselves. Specifically, we propose an uncalibrated flatfielding and an uncalibrated illumination vector estimation methodology, and we assess their effect on photometric stereo face reconstruction. Results demonstrate that incorporating our methodologies into the photometric stereo framework halves the reconstruction error, while eliminating the need of off-line calibration.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

Similar content being viewed by others

References

  1. Agrawal, A., Raskar, R., Chellappa, R.: What is the range of surface reconstructions from a gradient field?. In: European Conference on Computer Vision (ECCV), vol. 3951, p. 578591 (2006)

  2. Alldrin, N., Kriegman, D.: A planar light probe. In:IEEE Conference on Computer Vision and Pattern Recognition (CVPR) vol. 2, pp. 2324–2330 (2006)

  3. Alldrin, N.G., Mallick, S.P., Kriegman, D.J.: Resolving the generalized bas-relief ambiguity by entropy minimization. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–7 (2007)

  4. Angelopoulou, M.E., Petrou, M.: Evaluating the effect of diffuse light on photometric stereo reconstruction. Mach. Vis. Appl. 25(1), 199–210 (2014)

    Article  Google Scholar 

  5. Argyriou, V., Petrou, M.: Recursive photometric stereo when multiple shadows and highlights are present. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–6 (2008)

  6. Argyriou, V., Petrou, M.: Photometric stereo: an overview. Adv. Imaging Electron. Phys. 156, 1–54 (2009)

    Article  Google Scholar 

  7. Atkinson, G.A., Smith, M.L.: Using photometric stereo for face recognition. Int. J. Biosci. Biotechnol. 3(3), 35–44 (2011)

    Google Scholar 

  8. Barsky, S., Petrou, M.: The 4-source photometric stereo technique for three-dimensional surfaces in the presence of highlights and shadows. IEEE Trans. Pattern Anal. Mach. Intell. 25(10), 1239–1252 (2003)

    Article  Google Scholar 

  9. Basri, R., Jacobs, D., Kemelmacher, I.: Photometric stereo with general, unknown lighting. Int. J. Comput. Vision 72(3), 239–257 (2007)

    Article  Google Scholar 

  10. Belhumeur, P., Kriegman, D.J., Yuille, A.L.: The bas-relief ambiguity. Int. J. Comput. Vision 35(1), 33–44 (1999)

    Article  Google Scholar 

  11. Bouganis, C.S., Brookes, M.: Multiple light source detection. IEEE Trans. Pattern Anal. Mach. Intell. 26(4), 509–514 (2004)

    Article  Google Scholar 

  12. Capak, P., Cowie, L., Hu, E., Barger, A., Dickinson, M., Fernández, E., Giavalisco, M., Komiyama, Y., Kretchmer, C., McNally, C., Miyazaki, S., Okamura, S., Stern, D.: A deep, wide field, optical, and near infrared catalog of a large area around the hubble deep field north. Astron. J. 127(1), 180–198 (2004)

    Article  Google Scholar 

  13. Chandraker, M., Jiamin, J.B., Ramamoorthi, R.: A theory of differential photometric stereo for general isotropic brdfs. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2505–2512 (2011)

  14. Clark, J.: Active photometric stereo. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 29–34 (1992)

  15. Davis, C., Eisloeffel, J.: Near-infrared imaging in \(H_{2}\) of molecular (\(CO\)) outflows from young stars. Astron. Astrophys. 300, 851–869 (1995)

    Google Scholar 

  16. Debevec, P.E.: Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography. In: Annual Conference on Computer Graphics (ACM SIGGRAPH), pp. 189–198 (1998)

  17. Favaro, P., Papadhimitri, T.: A closed-form solution to uncalibrated photometric stereo via diffuse maxima. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 821–828 (2012)

  18. Frankot, R., Chellappa, R.: A method for enforcing integrability in shape from shading algorithms. IEEE Trans. Pattern Anal. Mach. Intell. 10(4), 439–451 (1988)

    Article  MATH  Google Scholar 

  19. Hansen, M.F., Atkinson, G.A., Smith, L.N., Smith, M.L.: 3d face reconstructions from photometric stereo using near infrared and visible light. Comput. Vis. Image Underst. 114(8), 942–951 (2010)

    Article  Google Scholar 

  20. Hara, K., Nishino, K., lkeuchi, K.: Light source position and reflectance estimation from a single view without the distant illumination assumption. IEEE Trans. Pattern Anal. Mach. Intell. 27(4), 493–505 (2005)

    Article  Google Scholar 

  21. Hernández, C., Vogiatzis, G., Brostow, G., Stenger, B., Cipolla, R.: Non-rigid photometric stereo with colored lights. In: IEEE International Conference on Computer Vision (ICCV), pp. 1–8 (2007)

  22. Hernandez, C., Vogiatzis, G., Cipolla, R.: Multiview photometric stereo. IEEE Trans. Pattern Anal. Mach. Intell. 30(3), 548–554 (2008)

    Article  Google Scholar 

  23. Hernández, C., Vogiatzis, G., Cipolla, R.: Overcoming shadows in 3-source photometric stereo. IEEE Trans. Pattern Anal. Mach. Intell. 33(2), 419–426 (2011)

    Article  Google Scholar 

  24. Hertzmann, A., Seitz, S.: Example-based photometric stereo: shape reconstruction with general, varying brdfs. IEEE Trans. Pattern Anal. Mach. Intell. 27(8), 1254–1264 (2005)

    Article  Google Scholar 

  25. Higo, T., Matsushita, Y., Joshi, N., Ikeuchi, K.: A hand-held photometric stereo camera for 3-d modeling. In: IEEE International Conference on Computer Vision (ICCV), pp. 1234–1241 (2009)

  26. Horn, B.: Robot Vision. The MIT Press, Cambridge (1986)

    Google Scholar 

  27. Iwahori, Y., Sugie, H., Ishii, N.: Reconstructing shape from shading images under point light source illumination. In: IEEE International Conference on Pattern Recognition (ICPR), pp. 83–87 (1990)

  28. Liao, M., Huang, X., Yang, R.: Interreflections removal for photometric stereo by using spectrum-dependent albedo. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 689–696 (2011)

  29. Liao, M., Wang, L., Yang, R., Gong, M.: Light fall-off stereo. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–8 (2007)

  30. Lim, J., Ho, J., Yang, M.H., Kriegman, D.: Passive photometric stereo from motion. In: IEEE International Conference on Computer Vision (ICCV), vol. 2, pp. 1635–1642 (2005)

  31. Llado, X., Oliver, A., Petrou, M., Freixenet, J., Marti, J.: Simultaneous surface texture classification and illumination tilt angle prediction. In: British Machine Vision Conference (BMVC), pp. 789–798 (2003)

  32. Magda, S., Kriegman, D.J., Zickler, T., Belhumeur, P.N.: Beyond lambert: Reconstructing surfaces with arbitrary brdfs. In: IEEE International Conference on Computer Vision (ICCV), vol. 2, pp. 391–398 (2001)

  33. Mulligan, J., Brolly, X.: Surface determination by photometric ranging. In: Conference on Computer Vision and Pattern Recognition Workshop (CVPRW), pp. 40–40 (2004)

  34. Narasimhan, S., Nayar, S., Sun, B., Koppal, S.: Structured light in scattering media. In: IEEE International Conference on Computer Vision (ICCV), pp. 420–427 (2005)

  35. Petrou, M., Petrou, C.: Image Processing: The Fundamentals, 2nd edn. Wiley, New Jersey (2010)

    Book  Google Scholar 

  36. Shi, B., Matsushita, Y., Wei, Y., Xu, C., Tan, P.: Self-calibrating photometric stereo. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1118–1125 (2010)

  37. Takai, T., Niinuma, K., Maki, A., Matsuyama, T.: Difference sphere: an approach to near light source estimation. In: IEEE International Conference on Computer Vision (CVPR), vol. 1, pp. 98–105 (2004)

  38. Tankus, A., Kiryati, N.: Photometric stereo under perspective projection. In: IEEE International Conference on Computer Vision (ICCV), vol. 1, pp. 611–616 (2005)

  39. Vlasic, D., Peers, P., Baran, I., Debevec, P., Popovic, J., Rusinkiewicz, S., Matusik, W.: Dynamic shape capture using multi-view photometric stereo. ACM Trans. Graph. 28(5), 174:1–174:11 (2009)

    Article  Google Scholar 

  40. Weber, M., Blake, A., Cipolla, R.: Towards a complete dense geometric and photometric reconstruction under varying pose and illumination. Image Vis. Comput. 22(10), 787–793 (2004)

    Article  Google Scholar 

  41. Weber, M., Cipolla, R.: A practical method for estimation of point light-sources. In: British Machine Vision Conference (BMVC), vol. 2, pp. 471–480 (2001)

  42. Woodham, R.: Photometric method for determining surface orientation from multiple images. Opt. Eng. 19(1), 139–144 (1980)

    Article  Google Scholar 

  43. Yoshiyasu, Y.: Topology-adaptive multi-view photometric stereo. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1001–1008 (2011)

  44. Zafeiriou, S., Hansen, M., Atkinson, G., Petrou, M., Smith, M.: Baseline face recognition using photometric stereo data. In: International Conference and Exhibition on Biometrics Technology (ICEBT), vol. 2, pp. 20–25 (2010)

  45. Zhang, L., Curless, B., Hertzmann, A., Seitz, S.: Shape and motion under varying illumination: unifying structure from motion, photometric stereo, and multiview stereo. In: IEEE International Conference on Computer Vision (CVPR), vol. 1, pp. 618–625 (2003)

  46. Zhou, S., Aggarwal, G., Chellappa, R., Jacobs, D.: Appearance characterization of linear lambertian objects, generalized photometric stereo, and illumination-invariant face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 29(2), 230–245 (2007)

    Article  Google Scholar 

  47. Zhou, W., Kambhamettu, C.: Estimation of illuminant direction and intensity of multiple light sources. In: European Conference on Computer Vision (ECCV), pp. 206–220 (2002)

Download references

Acknowledgments

The authors would like to acknowledge the contribution of Dr. Vasileios Argyriou and Dr. Stefanos Zafeiriou in the system setup.

Author information

Authors and Affiliations

  1. Department of Electrical and Electronic Engineering, Imperial College London, South Kensington Campus, London, SW7 2AZ, UK

    Maria E. Angelopoulou & Maria Petrou

Authors
  1. Maria E. Angelopoulou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maria Petrou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Maria E. Angelopoulou.

Additional information

The research leading to these results has received funding from the European Commission FP7-ICT Cognitive Systems, Interaction and Robotics under the contract #270180 (NOPTILUS).

Rights and permissions

Reprints and permissions

About this article

Cite this article

Angelopoulou, M.E., Petrou, M. Uncalibrated flatfielding and illumination vector estimationfor photometric stereo face reconstruction. Machine Vision and Applications 25, 1317–1332 (2014). https://doi.org/10.1007/s00138-014-0609-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-014-0609-2

Keywords

Search

Navigation