Skip to main content
SpringerLink
Log in

Decomposing Global Light Transport Using Time of Flight Imaging

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Global light transport is composed of direct and indirect components. In this paper, we take the first steps toward analyzing light transport using the high temporal resolution information of time of flight (ToF) images. With pulsed scene illumination, the time profile at each pixel of these images separates different illumination components by their finite travel time and encodes complex interactions between the incident light and the scene geometry with spatially-varying material properties. We exploit the time profile to decompose light transport into its constituent direct, subsurface scattering, and interreflection components. We show that the time profile is well modelled using a Gaussian function for the direct and interreflection components, and a decaying exponential function for the subsurface scattering component. We use our direct, subsurface scattering, and interreflection separation algorithm for five computer vision applications: recovering projective depth maps, identifying subsurface scattering objects, measuring parameters of analytical subsurface scattering models, performing edge detection using ToF images and rendering novel images of the captured scene with adjusted amounts of subsurface scattering.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

Similar content being viewed by others

References

  • Abramson, N. (1978). Light-in-flight recording by holography. Optics Letters, 3(4), 121–123.

    Article  Google Scholar 

  • Arvo, J. (1993). Transfer equations in global illumination. SIGGRAPH Course Notes.

  • Bai, J., Chandraker, M., Ng, T., & Ramamoorthi, R. (2010). A dual theory of inverse and forward light transport. European Conference on Computer Vision, 2010, 294–307.

    Google Scholar 

  • Brooker, G. (2009). Introduction to sensors for ranging and imaging. Raleigh: Scitech.

    Google Scholar 

  • Busck, J., & Heiselberg, H. (2004). Gated viewing and high-accuracy three-dimensional laser radar. Applied Optics, 43(24), 4705–4710.

    Article  Google Scholar 

  • Chen, T., Lensch, H., Fuchs, C., & Seidel, H. (2007). Polarization and phase-shifting for 3d scanning of translucent objects. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 1–8).

  • Gleckler, A., & Gelbart, A. (2000). Multiple-slit streak tube imaging lidar (ms-stil) applications. Proceedings SPIE (Vol. 4035, pp. 266–278).

  • Gupta, M., Agrawal, A., Veeraraghavan, A. & Narasimhan, S. (2011). Structured light 3d scanning in the presence of global illumination. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 713–720).

  • Gupta, M., Tian, Y., Narasimhan, S., & Zhang, L. (2009). (De) focusing on global light transport for active scene recovery. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 2969–2976).

  • Gupta, O., Willwacher, T., Velten, A., Veeraraghavan, A., & Raskar, R. (2012). Reconstruction of hidden 3d shapes using diffuse reflections. Optics Express, 20(17), 19096–19108.

    Article  Google Scholar 

  • Hamamatsu. (2012). Guide to streak cameras. http://sales.hamamatsu.com/assets/pdf/catsandguides/e_streakh.pdf.

  • Heide, F., Hullin, M., Gregson, J., & Heidrich, W. (2013). Low-budget transient imaging using photonic mixer devices. Proceedings of ACM SIGGRAPH, ACM Transactions on Graphics, 32(4), 146:1–146:12.

    Google Scholar 

  • Holroyd, M., Lawrence, J., & Zickler, T. (2010). A coaxial optical scanner for synchronous acquisition of 3d geometry and surface reflectance. ACM Transactions on Graphics, 29(4), 99.

    Article  Google Scholar 

  • Huang, D., Swanson, E., Lin, C., Schuman, J., Stinson, W., Chang, W., et al. (1991). Optical coherence tomography. Science, 254(5035), 1178–1181.

    Article  Google Scholar 

  • Jarabo, A., Masia, B., Velten, A., Barsi, C., Raskar, R., & Gutierrez, D. (2013). Rendering relativistic effects in transient imaging. Proceedings of CEIG.

  • Kajiya, J. (1986). The rendering equation. ACM SIGGRAPH Computer Graphics, 20(4), 143–150.

    Article  Google Scholar 

  • Liu, S., Ng, T., & Matsushita, Y. (2010). Shape from second-bounce of light transport. European Conference on Computer Vision, 2010, 280–293.

    Google Scholar 

  • Mukaigawa, Y., Yagi, Y., & Raskar, R. (2010). Analysis of light transport in scattering media. Proceedings of Computer Vision and Pattern Recognition (CVPR) (pp. 153–160).

  • Naik, N., Zhao, S., Velten, A., Raskar, R., & Bala, K. (2011). Single view reflectance capture using multiplexed scattering and time-of-flight imaging. ACM Transaction on Graphics, 30(6), 171.

    Article  Google Scholar 

  • Nayar, S., Krishnan, G., Grossberg, M., & Raskar, R. (2006). Fast separation of direct and global components of a scene using high frequency illumination. ACM Transaction on Graphics, 25(3), 935–944.

    Article  Google Scholar 

  • Ng, T., Pahwa, R., Bai, J., Tan, K., & Ramamoorthi, R. (2012). From the rendering equation to stratified light transport inversion. International Journal of Computer Vision, 96(2), 235–251.

    Google Scholar 

  • O’Toole, M., Raskar, R., & Kutulakos, K. (2012). Primal-dual coding to probe light transport. ACM Transactions on Graphics, 31(4), 39.

    Google Scholar 

  • Pandharkar, R., Velten, A., Bardagjy, A., Lawson, E., Bawendi, M., & Raskar, R. (2011). Estimating motion and size of moving non-line-of-sight objects in cluttered environments. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 265–272).

  • Reddy, D., Ramamoorthi, R., & Curless, B. (2012). Frequency-space decomposition and acquisition of light transport under spatially varying illumination. European Conference on Computer Vision (pp. 596–610).

  • Repasi, E., Lutzmann, P., Steinvall, O., Elmqvist, M., Göhler, B., & Anstett, G. (2009). Advanced short-wavelength infrared range-gated imaging for ground applications in monostatic and bistatic configurations. Applied Optics, 48(31), 5956–5969.

    Article  Google Scholar 

  • Seitz, S., Matsushita, Y., & Kutulakos, K. (2005). A theory of inverse light transport. Tenth IEEE International Conference on Computer Vision, (Vol. 2, pp. 1440–1447).

  • Sen, P., Chen, B., Garg, G., Marschner, S., Horowitz, M., Levoy, M., et al. (2005). Dual photography. ACM Transactions on Graphics, 24(3), 745–755.

    Article  Google Scholar 

  • Velten, A., Willwacher, T., Gupta, O., Veeraraghavan, A., Bawendi, M. G., & Raskar, R. (2012). Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging. Nature Communication, 3, 745–758.

    Article  Google Scholar 

  • Velten, A., Wu, D., Jarabo, A., Masia, B., Barsi, C., Joshi, C., et al. (2012). Relativistic ultrafast rendering using time-of-flight imaging. In ACM SIGGRAPH Talks, 41(1).

  • Velten, A., Wu, D., Jarabo, A., Masia, B., Barsi, C., Joshi, C., Lawson, E., Bawendi, M., Gutierrez, D., & Raskar, R. (2013). Femto-photography: Capturing and visualizing the propagation of light. Proceedindgs of SIGGRAPH, ACM Transaction on Graphics 32(4).

  • Wang, L., & Wu, H. (2007). Biomedical optics: Principles and imaging. Hoboken, NJ: Wiley.

    Google Scholar 

  • Wetzstein, G., & Bimber, O. (2007). Radiometric compensation through inverse light transport. Proceedings of the 15th Pacific Conference on Computer Graphics and Applications (pp. 391–399).

  • Wu, D., Wetzstein, G., Barsi, C., Willwacher, T., O’Toole, M., Naik, N., et al. (2012). Frequency analysis of transient light transport with applications in bare sensor imaging. European Conference on Computer Vision, 2012, 542–555.

    Google Scholar 

  • Wyant, J. (2002). White light interferometry. In AeroSense 2002 (pp. 98–107). Bellingham, WA: International Society for Optics and Photonics.

  • Xia, H., & Zhang, C. (2009). Ultrafast ranging lidar based on real-time Fourier transformation. Optics Letters, 34, 2108–2110.

    Article  Google Scholar 

  • Zhang, L., & Nayar, S. (2006). Projection defocus analysis for scene capture and image display. ACM Transactions on Graphics, 25(3), 907–915.

    Article  Google Scholar 

Download references

Acknowledgments

The work of the MIT affiliated coauthors was funded by the Media Lab Consortium Members, DARPA through the DARPA YFA grant, and the Institute for Soldier Nanotechnologies and U.S. Army Research Office under contract W911NF-07-D-0004. The work of the Tsinghua affiliated coauthors was supported by the National Basic Research Project (No.2010CB731800) of China and the Key Project of NSFC (No. 61120106003 and 60932007). O’Toole received the support of the NSERC PGS-D and GRAND NCE programs. Masia was additionally funded by an FPU grant, project TIN2010-21543 and an NVIDIA Graduate Fellowship.

Author information

Authors and Affiliations

  1. MIT Media Lab, Room E14-474G, 75 Amherst St, Cambridge, MA, 02139, USA

    Di Wu, Matthew O’Toole, Belen Masia & Ramesh Raskar

  2. Tsinghua University, Beijing, China

    Di Wu

  3. Medical Engineering, Morgridge Institute for Research, 330 North Orchard Street, Madison, WI, 53715, USA

    Andreas Velten

  4. Laboratory for Optical and Computational Instrumentation, University of Wisconsin at Madison, 1675 Observatory Drive, Madison, WI, 53706, USA

    Andreas Velten

  5. University of Toronto, Toronto, ON, Canada

    Matthew O’Toole

  6. Universidad de Zaragoza, Saragossa, Spain

    Belen Masia

  7. Mitsubishi Electric Research Labs, Cambridge, MA, USA

    Amit Agrawal

  8. Department of Automation, Tsinghua University, Room 410, Central Main Building, Beijing, China

    Qionghai Dai

Authors
  1. Di Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andreas Velten
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Matthew O’Toole
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Belen Masia
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Amit Agrawal
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Qionghai Dai
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Ramesh Raskar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Andreas Velten.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (mp4 4953 KB)

Supplementary material 2 (mp4 3233 KB)

Supplementary material 3 (mp4 3881 KB)

Supplementary material 4 (mp4 12513 KB)

Supplementary material 5 (mp4 9305 KB)

Supplementary material 6 (mp4 3621 KB)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wu, D., Velten, A., O’Toole, M. et al. Decomposing Global Light Transport Using Time of Flight Imaging. Int J Comput Vis 107, 123–138 (2014). https://doi.org/10.1007/s11263-013-0668-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-013-0668-2

Keywords

Search

Navigation