Skip to main content
SpringerLink
Log in

Computational Schlieren Photography with Light Field Probes

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

We introduce a new approach to capturing refraction in transparent media, which we call light field background oriented Schlieren photography. By optically coding the locations and directions of light rays emerging from a light field probe, we can capture changes of the refractive index field between the probe and a camera or an observer. Our prototype capture setup consists of inexpensive off-the-shelf hardware, including inkjet-printed transparencies, lenslet arrays, and a conventional camera. By carefully encoding the color and intensity variations of 4D light field probes, we show how to code both spatial and angular information of refractive phenomena. Such coding schemes are demonstrated to allow for a new, single image approach to reconstructing transparent surfaces, such as thin solids or surfaces of fluids. The captured visual information is used to reconstruct refractive surface normals and a sparse set of control points independently from a single photograph.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20

Similar content being viewed by others

Notes

  1. The connection between Snell’s law and the ray equation of geometric optics (Eq. 1) is well-known in physics, see e.g. http://en.wikibooks.org/wiki/Introduction_to_Mathematical_Physics/Electromagnetism/Optics,_particular_case_of_electromagnetism.

References

  • Agarwal, S., Mallick, S.P., Kriegman, D., & Belongie, S. (2004). On refractive optical flow. In Proceeding of ECCV (pp. 483–494).

  • Agrawal, A., Raskar, R., & Chellappa, R. (2006) What is the range of surface reconstructions from a gradient field? In Proceedings of ECCV (pp. 483–494).

  • Arpa, A., Wetzstein, G., Lanman, D., & Raskar, R. (2012). Single lens off-chip cellphone microscopy. In Proceedings of IEEE PROCAMS.

  • Atcheson, B., Heide, F., & Heidrich, W. (2010). CALTag: High precision fiducial markers for camera calibration. In Proceedings of VMV.

  • Atcheson, B., Ihrke, I., Heidrich, W., Tevs, A., Bradley, D., Magnor, M., et al. (2008). Time-resolved 3D capture of non-stationary gas flows. ACM Transactions on Graphics (Siggraph Asia), 27, 132.

    Google Scholar 

  • Ben-Ezra, M., & Nayar, S.K. (2003) What does motion reveal about transparency? In Proceeding of ICCV (pp. 1025–1032).

  • Bonfort, T., Sturm, P., & Gargallo, P. (2006). General specular surface triangulation. In Proceedings of ACCV (pp. 872–881).

  • Born, M., & Wolf, E. (1999). Principles of Optics (7th ed.). Cambridge: Cambridge University Press.

    Book  Google Scholar 

  • Dalziel, S., Hughes, G., & Sutherland, B. (2000). Whole-field density measurements by synthetic schlieren. Experiments in Fluids, 28, 322–335.

    Article  Google Scholar 

  • Gao, C., & Ahuja, N. (2006). A refractive camera for acquiring stereo and super-resolution images. In Proceeding of CVPR (pp. 2316–2323).

  • Hargather, M. J., & Settles, G. S. (2009). Natural-background-oriented Schlieren imaging. Experiments in Fluids, 48, 59–68.

    Article  Google Scholar 

  • Hernandez, C., Vogiatzis, G., Brostow, G., Stenger, B., & Cipolla, R. (2007). Non-rigid photometric stereo with colored lights. In Proceeding of ICCV.

  • Horovitz, I., & Kiryati, N. (2004). Depth from gradient fields and control points: Bias correction in photometric stereo. In Proceedings of ICIVC (pp. 681–694).

  • Howes, W. L. (1984). Rainbow schlieren and its applications. Applied Optics, 23, 2449–2460.

    Article  Google Scholar 

  • Hullin, M. B., Fuchs, M., Ihrke, I., Seidel, H. P., & Lensch, H. P. A. (2008). Fluorescent Immersion Range Scanning. ACM Transactions on Graphics (Siggraph), 87(1–87), 10.

    Google Scholar 

  • Huynh, C. P., Robles-Kelly, A., & Hancock, E. (2010). Shape and refractive index recovery from single-view polarisation images. In Proceeding of CVPR.

  • Ihrke, I., Goldluecke, B., & Magnor, M. (2005). Reconstructing the geometry of flowing water. In Proceedings of ICCV (pp. 1055– 1060).

  • Ihrke, I., Kutulakos, K. N., Lensch, H. P. A., Magnor, M., & Heidrich, W. (2010). Transparent and specular object reconstruction. Computer Graphics Forum, 29, 2400–2426.

    Article  Google Scholar 

  • Ihrke, I., Ziegler, G., Tevs, A., Theobalt, C., Magnor, M., & Seidel, H.P. (2007). Eikonal rendering: Efficient light transport in refractive objects. In Proceeding of SIGGRAPH. ACM Transactions on Graphics, 26:59.

  • Kutulakos, K.N., & Steger, E. (2005) A theory of refractive and specular 3D shape by light-path triangulation. In Proceeding of ICCV (1448–1455).

  • Levoy, M., Zhang, Z., & McDowall, I. (2009). Recording and controlling the 4D light field in a microscope. Journal of Microscopy, 235, 144–162.

    Article  MathSciNet  Google Scholar 

  • Merzkirch, W. (1987). Flow visualization. New York: Academic Press.

    MATH  Google Scholar 

  • Miyazaki, D., & Ikeuchi, K. (2005). Inverse polarization raytracing: Estimating surface shapes of transparent objects. In Proceeding of CVPR (pp. 910–917).

  • Morris, N. J. W., & Kutulakos, K. N. (2005). Dynamic refraction stereo. In Proceedings of ICCV.

  • Morris, N. J. W., & Kutulakos, K. N. (2007). Reconstructing the surface of inhomogeneous transparent scenes by scatter trace photography. In Proceedings of ICCV.

  • Murase, H. (1990) Surface shape reconstruction of an undulating transparent object. In Proceedings of ICCV (pp. 313–317).

  • Murphy, D. B. (2001). Fundamentals of light microscopy and electronic imaging. New York: Wiley.

    Google Scholar 

  • Nehab, D., Rusinkiewicz, S., Davis, J., & Ramamoorthi, R. (2005). Efficiently combining positions and normals for precise 3D geometry. ACM Transactions on Graphics (Siggraph) 24.

  • Ng, H. S., Wu, T. P., & Tang, C. K. (2010). Surface-from-gradients without discrete integrability enforcement: A Gaussian Kernel approach. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32, 2085–2099.

    Article  Google Scholar 

  • Okano, F., Arai, J., Hoshino, H., & Yuyama, I. (1999). Three-dimensional video system based on integral photography. Optical Engineering, 38, 1072–1077.

    Article  Google Scholar 

  • Savarese, S., & Perona, P. (2002). Local analysis for 3D reconstruction of specular surfaces—part II. In Proceedings of ECCV (pp. 759–774).

  • Schardin, H. (1942). Die Schlierenverfahren und ihre Anwendungen. Ergebnisse der Exakten Naturwissenschaften. (English translation available as NASA TT F-12731), April 1970 (N70–25586) 20, 303–439.

  • Settles, G. S. (2001). Schlieren and shadowgraph techniques. Cambridge: Cambridge University Press.

    Book  MATH  Google Scholar 

  • Settles, G. S. (2010). Important Developments in Schlieren and Shadowgraph Visualization during the Last Decade. In Proceedings of International Symposium on Flow Visualization (p. 267).

  • Shimizu, M., & Okutomi, M. (2006) Reflection stereo—Novel monocular stereo using a transparent plate. In Proceeding of CRV (pp. 1–14).

  • Tarini, M., Lensch, H. P., Goesele, M., & Seidel, H. P. (2005). 3D acquisition of mirroring objects using striped patterns. Graphical Models, 67, 233–259.

    Article  Google Scholar 

  • Trifonov, B., Bradley, D., & Heidrich, W. (2006). Tomographic reconstruction of transparent objects. In Proceedings of EGSR (pp. 51–60).

  • Wetzstein, G., Lanman, D., Hirsch, M., & Raskar, R. (2012). Tensor displays: Compressive light field synthesis using multilayer displays with directional backlighting. ACM Transactions on Graphics (SIGGRAPH), 31, 1–11.

    Article  Google Scholar 

  • Wetzstein, G., Raskar, R., & Heidrich, W. (2011) Hand-held schlieren photography with light field probes. In Proceeding of ICCP (pp. 1–8).

  • Wetzstein, G., Roodnick, D., Raskar, R., & Heidrich, W. (2011). Refractive Shape from light field distortion. In International Conference on Computer Vision (ICCV).

  • Woodham, R. (1980). Photometric method for determining surface orientation from multiple images. Optical Engineering, 19(1), 139–144.

    Google Scholar 

  • Zhang, X., & Cox, C. S. (1994). Measuring the two-dimensional structure of a wavy water surface optically: A surface gradient detector. Experiments in Fluids, 17, 225–237.

    Article  MATH  Google Scholar 

  • Zongker, D.E., Werner, D.M., Curless, B., & Salesin, D.H. (1999). Environment matting and compositing. In Proceeding of SIGGRAPH. Computer Graphics (pp. 205–214).

Download references

Acknowledgments

Gordon Wetzstein was supported by an NSERC Postdoctoral Fellowship. Ramesh Raskar was supported by an Alfred P. Sloan Research Fellowship and a DARPA Young Faculty Award.

Author information

Authors and Affiliations

  1. MIT Media Lab, Cambridge, MA, USA

    Gordon Wetzstein & Ramesh Raskar

  2. University of British Columbia, Vancouver, BC, Canada

    Wolfgang Heidrich

Authors
  1. Gordon Wetzstein
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wolfgang Heidrich
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ramesh Raskar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gordon Wetzstein.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wetzstein, G., Heidrich, W. & Raskar, R. Computational Schlieren Photography with Light Field Probes. Int J Comput Vis 110, 113–127 (2014). https://doi.org/10.1007/s11263-013-0652-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-013-0652-x

Keywords

Search

Navigation