Skip to main content
Log in

Local Shape from Mirror Reflections

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

We study the problem of recovering the 3D shape of an unknown smooth specular surface from a single image. The surface reflects a calibrated pattern onto the image plane of a calibrated camera. The pattern is such that points are available in the image where position, orientations, and local scale may be measured (e.g. checkerboard). We first explore the differential relationship between the local geometry of the surface around the point of reflection and the local geometry in the image.We then study the inverse problem and give necessary and sufficient conditions for recovering surface position and shape.We prove that surface position and shape up to third order can be derived as a function of local position, orientation and local scale measurements in the image when two orientations are available at the same point (e.g. a corner). Information equivalent to scale and orientation measurements can be also extracted from the reflection of a planar scene patch of arbitrary geometry, provided that the reflections of (at least) 3 distinctive points may be identified.We validate our theoretical results with both numerical simulations and experiments with real surfaces.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Binford T. 1981. Inferring surfaces from images. Artificial Intelligence, 17:205–244.

    Article  Google Scholar 

  • Blake, A. 1985. Specular stereo. In IJCAI, pp. 973–976.

  • Blake, A. and Brelstaff, G. 1988. Geometry from specularities. In Proc. of Int. Conf. of Computer Vision, pp. 394–403.

  • Bonfort T. and Sturm, P. 2003. Voxel carving for specular surfaces. In Proc. of Int. Conf. of Computer Vision, pp. 394–403.

  • Born, M. and Wolf, E. 1965. Principle of Optics—Electromagnetic Theory and Propagation. OPergamon Press.

  • Bouguet, J. http://www.vision.caltech.edu/bouguetj/ .

  • Carmo, M.D. 1976. Differential Geometry of Curves and Surfaces. Prentice-Hall.

  • Chen, M. and Arvo, J. 2000. Theory and application of specular path perturbation. ACM Transactions on Graphics, 19:246–278

    Article  Google Scholar 

  • Cipolla, R. and Giblin, P. 2000. Visual Motion of Curves and Surfaces. Cambridge University Press

  • Dörrie, M. 1989. 100 Great Problems of Elementary Mathematics: Their history and Solution. Dover Publications.

  • Fleming, R. Torralba, A., and Adelson, E.H. 2003. How image statistics drive shape-from-texture and shape-from-specularities. In Proc. of Third Annual Meeting of the VVS.

  • Halsead, M., Barsky, A., Klein, S., and Mandell, R. 1996. Reconstructing curved surfaces from reflection patterns using spline surface fitting normals. In SIGGRAPH.

  • Healey, G. and Binford, T. 1988. Local shape from specularity. Computer Vision, Graphics, and Image Processing, 42:62–86.

    Google Scholar 

  • Ikeuchi, K. 1981. Determining surface orientation of specular surfaces by using the photometric stereo method. IEEE Journal of Pattern Analysis and Machine Intelligence, 3:661–669.

    Google Scholar 

  • Koenderink, J. and van Doorn, A. 1980. Photometric invariants related to solid shape. Optica Apta, 27:981–996.

    Google Scholar 

  • Goesele, M., Tarini, M., Lensch, H., and Seidel, H.P. 2002. Shape from distortion: 3d range scanning of mirroring objects. In Proc. of SIGGRAPH, Sketches & Applications, pp. 248.

  • Neumann, P. 1998. Reflections on reflection in a spherical mirror. In The American Mathematical Monthly.

  • Oren, M. and Nayar, S.K. 1997. A theory of specular surface geometry. Trans. International Journal of Computer Vision, pp. 105–124.

  • Perard, D. 2001. “Automated Visual Inspection of Specular Surfaces with Structured-lighting Reflection Techiniques.” PhD thesis, VDI Verlag (Nr. 869).

  • Savarese, S., Chen, M., and Perona, P. 2002. Second order local analysis for 3d reconstruction of specular surfaces. In Proc. of 3D Data Processing Visualization and Trasmission.

  • Savarese, S., Chen, M., and Perona, P. 2004. Recovering local shape of a mirror surface from reflection of a regular grid. In Proc. of European Conference of Computer Vision.

  • Savarese, S., Fei-Fei, L., and Perona, P. 2004. What do reflections tell us about the shape of a mirror? In ACM SIGGRAPH—Applied Perception in Graphics and Visualization.

  • Savarese, S. and Perona, P. 2001. Local analysis for 3d reconstruction of specular surfaces. In Proc. of IEEE Conference of Computer Vision and Pattern Recognition.

  • Savarese, S. and Perona, P. 2002. Local analysis for 3d reconstruction of specular surfaces—part ii. In Proc. of European Conference of Computer Vision.

  • Smith, J. 1992. The remarkable Ibn al-Haytham. Math. Gazette, 76:189–198.

    Google Scholar 

  • Solem, J., Aans, H., and Heyden, A. 2003. A variational analysis of shape from specularities using sparse data. In International Symposium on 3D Data Processing Visualization and Transmission.

  • Swaminathan, R., Grossberg, M.D., and Nayar, S.K. 2001. Caustics of catadioptric cameras. In Proc. International Conference on Computer Vision.

  • Swaminathan, R., Nayar, S.K., and Grossberg, M.D. 2004. Designing mirrors for catadioptric systems that minimize image errors. In Proc. of the Fifth Workshop on Omnidirectional Vision.

  • Wang, J. and Dana, K.L. 2003. A novel approach for texture shape recovery. In Proc. International Conference on Computer Vision.

  • Zheng, J. and Murata, A. 2000. Acquiring a complete 3d model from specular motion under the illumination of circular-shaped light sources. IEEE Journal of Pattern Analysis and Machine Intelligence, Vol. 8.

  • Zisserman, A., Giblin, P., and Blake, A. 1989. The information available to a moving observer from specularities. Image and Video Computing, 7:38–42.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Silvio Savarese.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Savarese, S., Chen, M. & Perona, P. Local Shape from Mirror Reflections. Int J Comput Vision 64, 31–67 (2005). https://doi.org/10.1007/s11263-005-1086-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-005-1086-x

Keywords

Navigation