Abstract
We present a novel imaging technique, which we call femto-photography, to capture and visualize the propagation of light through table-top scenes with an effective exposure time of 1.85 ps per frame. This is equivalent to a resolution of about one half trillion frames per second; between frames, light travels approximately just 0.5 mm. Since cameras with such extreme shutter speed obviously do not exist, we first re-purpose modern imaging hardware to record an ensemble average of repeatable events that are synchronized to a streak sensor, in which the time of arrival of light from the scene is coded in one of the sensor's spatial dimensions. We then introduce reconstruction methods that allow us to visualize the propagation of femtosecond light pulses through the scenes. Given this fast resolution and the finite speed of light, we observe that the camera does not necessarily capture the events in the same order as they occur in reality: we thus introduce the notion of time-unwarping between the camera's and the world's space--time coordinate systems, to take this into account. We apply our femto-photography technique to visualizations of very different scenes, which allow us to observe the rich dynamics of time-resolved light transport effects, including scattering, specular reflections, diffuse interreflections, diffraction, caustics, and subsurface scattering. Our work has potential applications in artistic, educational, and scientific visualizations; industrial imaging to analyze material properties; and medical imaging to reconstruct subsurface elements. In addition, our time-resolved technique has already motivated new forms of computational photography, as well as novel algorithms for the analysis and synthesis of light transport.
- Charbon, E. Will avalanche photodiode arrays ever reach 1 megapixel? In International Image Sensor Workshop (Ogunquit, ME, 2007), 246--249.Google Scholar
- Colaço, A., Kirmani, A., Howland, G.A., Howell, J.C., Goyal, V.K. Compressive depth map acquisition using a single photon-counting detector: Parametric signal processing meets sparsity. In 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (Providence, RI, June 2012), IEEE, 96--102. Google ScholarDigital Library
- Goda, K., Tsia, K.K., Jalali, B. Serial time-encoded amplified imaging for real-time observation of fast dynamic phenomena. Nature 458 (2009), 1145--1149.Google ScholarCross Ref
- Gupta, O., Willwacher, T., Velten, A., Veeraraghavan, A., Raskar, R. Reconstruction of hidden 3D shapes using diffuse reflections. Opt. Expr. 20 (2012). 19096--19108.Google Scholar
- Heide, F., Hullin, M.B., Gregson, J., Heidrich, W. Low-budget transient imaging using photonic mixer devices. ACM Trans. Graph. 32, 4 (2013), 45:1--45:10. Google ScholarDigital Library
- Heide, F., Xiao, L., Heidrich, W., Hullin, M.B. Diffuse mirrors: 3D reconstruction from diffuse indirect illumination using inexpensive time-of-flight sensors. In CVPR (June 2014). Google ScholarDigital Library
- Jarabo, A., Marco, J., Muñoz, A., Buisan, R., Jarosz, W., Gutierrez, D. A framework for transient rendering. ACM Trans. Graph. 33, 6 (2014), 177:1--177:10. Google ScholarDigital Library
- Jarabo, A., Masia, B., Velten, A., Barsi, C., Raskar, R., Gutierrez, D. Relativistic effects for time-resolved light transport. Comput. Graph. Forum (2015) to appear. DOI: 10.1111/cgf.12604.Google ScholarDigital Library
- Kadambi, A., Whyte, R., Bhandari, A., Streeter, L., Barsi, C., Dorrington, A., Raskar, R. Coded time of flight cameras: Sparse deconvolution to address multipath interference and recover time profiles. ACM Trans. Graph. 32, 6 (2013), 167:1--167:10. Google ScholarDigital Library
- Kirmani, A., Hutchison, T., Davis, J., Raskar, R. Looking around the corner using ultrafast transient imaging. Int. J. Comp. Vision 95, 1 (2011), 13--28. Google ScholarDigital Library
- Laurenzis, M., Velten, A. Nonline-of-sight laser gated viewing of scattered photons. Opt. Eng. 53, 2 (2014), 023102--023102.Google ScholarCross Ref
- Naik, N., Zhao, S., Velten, A., Raskar, R., Bala, K. Single view reflectance capture using multiplexed scattering and TOF imaging. ACM Trans. Graph. 30 (2011), 171:1--171:10. Google ScholarDigital Library
- O'Toole, M., Heide, F., Xiao, L., Hullin, M.B., Heidrich, W., Kutulakos, K.N. Temporal frequency probing for 5D transient analysis of global light transport. ACM Trans. Graph. 33, 4 (2014), 87:1--87:11. Google ScholarDigital Library
- Pandharkar, R., Velten, A., Bardagjy, A., Bawendi, M., Raskar, R. Estimating motion and size of moving non-line-of-sight objects in cluttered environments. In 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (Colorado Springs, CO, June 2011), IEEE, 265--272. Google ScholarDigital Library
- Qu, J., Liu, L., Chen, D., Lin, Z., Xu, G., Guo, B., Niu, H. Temporally and spectrally resolved sampling imaging with a specially designed streak camera. Opt. Lett. 31 (2006), 368--370.Google ScholarCross Ref
- Velten, A., Fritz, A., Bawendi, M.G., Raskar, R. Multibounce time-of-flight imaging for object reconstruction from indirect light. In Conference for Lasers and Electro-Optics (OSA, 2012).Google ScholarCross Ref
- Velten, A., Willwacher, T., Gupta, O., Veeraraghavan, A., Bawendi, M.G., Raskar, R. Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging. Nat. Commun. 3, 745 (2012), 745:1--745:8.Google ScholarCross Ref
- Velten, A., Wu, D., Jarabo, A., Masia, B., Barsi, C., Joshi, C., Lawson, E., Bawendi, M.G., Gutierrez, D., Raskar, R. Femto-photography: Capturing and visualizing the propagation of light. ACM Trans. Graph. 32, 4 (2013), 44:1--44:8. Google ScholarDigital Library
- Velten, A., Wu, D., Jarabo, A., Masia, B., Barsi, C., Lawson, E., Joshi, C., Gutierrez, D., Bawendi, M.G., Raskar, R. Relativistic ultrafast rendering using time-of-flight imaging. In ACM SIGGRAPH Talks (2012). Google ScholarDigital Library
- Wu, D., Velten, A., O'Toole, M., Masia, B., Agrawal, A., Dai, Q., Raskar, R. Decomposing global light transport using time of flight imaging. Int. J. Comput. Vision 107, 2 (April 2014), 123--138. Google ScholarDigital Library
- Wu, D., Wetzstein, G., Barsi, C., Willwacher, T., O'Toole, M., Naik, N., Dai, Q., Kutulakos, K., Raskar, R. Frequency Analysis of Transient Light Transport with Applications in Bare Sensor Imaging. Springer, Berlin, Heidelberg, 2012, 542--555. Google ScholarDigital Library
Index Terms
- Imaging the propagation of light through scenes at picosecond resolution
Recommendations
Light source estimation of outdoor scenes for mixed reality
Illumination consistency is important for photorealistic rendering of mixed reality. However, it is usually difficult to acquire illumination conditions of natural environments. In this paper, we propose a novel method for evaluating the light ...
Real-time multiple scattering using light propagation volumes
I3D '12: Proceedings of the ACM SIGGRAPH Symposium on Interactive 3D Graphics and GamesThis paper introduces a new GPU-based, real-time method for rendering volumetric lighting effects produced by scattering in a participating medium. The method includes support for indirect illumination by scattered light, high-quality single-scattered ...
Precomputed radiance transfer for dynamic scenes taking into account light interreflection
EGSR'07: Proceedings of the 18th Eurographics conference on Rendering TechniquesFast rendering of dynamic scenes taking into account global illumination is one of the most challenging tasks in computer graphics. This paper proposes a new precomputed radiance transfer (PRT) method for rendering dynamic scenes of rigid objects taking ...
Comments