Shikhar Shrestha
Skip Abstract Section
Skip Supplemental Material SectionAbstract
Depth cameras are a ubiquitous technology used in a wide range of applications, including robotic and machine vision, human-computer interaction, autonomous vehicles as well as augmented and virtual reality. In this paper, we explore the design and applications of phased multi-camera time-of-flight (ToF) systems. We develop a reproducible hardware system that allows for the exposure times and waveforms of up to three cameras to be synchronized. Using this system, we analyze waveform interference between multiple light sources in ToF applications and propose simple solutions to this problem. Building on the concept of orthogonal frequency design, we demonstrate state-of-the-art results for instantaneous radial velocity capture via Doppler time-of-flight imaging and we explore new directions for optically probing global illumination, for example by de-scattering dynamic scenes and by non-line-of-sight motion detection via frequency gating.
Supplemental Material
- Bamji, C., O'Connor, P., Elkhatib, T., Mehta, S., Thompson, B., Prather, L., Snow, D., Akkaya, O., Daniel, A., Payne, A., amd M. Fenton, T. P., and Chan, V. 2015. A 0.13 um CMOS System-on-Chip for a 512 x 424 Time-of-Flight Image Sensor With Multi-Frequency Photo-Demodulation up to 130 MHz and 2 GS/s ADC. IEEE Journal of Solid-State Circuits 50, 1, 303--319.Google Scholar
Cross Ref
- Bhandari, A., Kadambi, A., Whyte, R., Barsi, C., Feigin, M., Dorrington, A., and Raskar, R. 2014. Resolving multipath interference in time-of-flight imaging via modulation frequency diversity and sparse regularization. Optics Letters 39, 1705--1708.Google ScholarCross Ref
- Buades, A., Coll, B., and Morel, J.-M. 2005. A non-local algorithm for image denoising. In Proc. IEEE CVPR, vol. 2. Google ScholarDigital Library
- Buehler, C., Bosse, M., McMillan, L., Gortler, S., and Cohen, M. 2001. Unstructured lumigraph rendering. In Proc. SIGGRAPH, 425--432. Google ScholarDigital Library
- Butler, D., Izadi, S., Hilliges, O., Molyneaux, D., Hodges, S., and Kim, D. 2012. Shake'n'sense: reducing interference for overlapping structured light depth cameras. In Proc. ACM UIST, 1933--1936. Google ScholarDigital Library
- Büttgen, B., and Seitz, P. 2008. Robust optical time-of-flight range imaging based on smart pixel structures. IEEE Trans. Circuits and Systems 55, 6, 1512--1525.Google ScholarCross Ref
- Carranza, J., Theobalt, C., Magnor, M. A., and Seidel, H.-P. 2003. Free-viewpoint video of human actors. ACM Trans. Graph. (SIGGRAPH) 22, 3, 569--577. Google ScholarDigital Library
- Castaneda, V., Mateus, D., and Navab, N. 2014. Stereo time-of-flight with constructive interference. IEEE Trans. PAMI 36, 7, 1402--1413. Google ScholarDigital Library
- Debevec, P., Hawkins, T., Tchou, C., Duiker, H.-P., Sarokin, W., and Sagar, M. 2000. Acquiring the reflectance field of a human face. In Proc. SIGGRAPH, 145--156. Google ScholarDigital Library
- Dong, S., Horstmeyer, R., Shiradkar, R., Guo, K., Ou, X., Bian, Z., Xin, H., and Zheng, G. 2014. Aperture-scanning fourier ptychography for 3d refocusing and super-resolution macroscopic imaging. Optics Express 22, 11, 13586--99.Google ScholarCross Ref
- Dorrington, A., Godbaz, J., Cree, M., Payne, A., and Streeter, L. 2011. Separating true range measurements from multi-path and scattering interference in commercial range cameras. In Proc. Electronic Imaging.Google Scholar
- Freedman, D., Krupka, E., Smolin, Y., Leichter, I., and Schmidt, M. 2014. Sra: fast removal of general multipath for tof sensors. In Proc. ECCV.Google Scholar
- Fuchs, S. 2010. Multipath interference compensation in time-of-flight camera images. In Proc. ICPR. Google ScholarDigital Library
- Gall, J., Ho, H., Izadi, S., Kohli, P., Ren, X., and Yang, R. 2014. Towards solving real-world vision problems with rgb-d cameras. In CVPR Tutorial.Google Scholar
- Gariepy, G., Tonolini, F., and Jonathan Leach, R. H., and Faccio, D. 2016. Detection and tracking of moving objects hidden from view. Nature Photonics Letters 10, 23--26.Google ScholarCross Ref
- Gokturk, S., Yalcin, H., and Bamji, C. 2004. A time-of-flight depth sensor - system description, issues and solutions. In Proc. CVPR, 35--35. Google ScholarDigital Library
- Gortler, S. J., Grzeszczuk, R., Szeliski, R., and Cohen, M. F. 1996. The lumigraph. In Proc. SIGGRAPH. Google ScholarDigital Library
- Hansard, M., Lee, S., Choi, O., and Horaud, R. 2012. Time of Flight Cameras: Principles, Methods, and Applications. Springer. Google ScholarDigital Library
- Heide, F., Hullin, M. B., Gregson, J., and Heidrich, W. 2013. Low-budget transient imaging using photonic mixer devices. ACM Trans. Graph. (SIGGRAPH) 32, 4, 45:1--45:10. Google ScholarDigital Library
- Heide, F., Xiao, L., Heidrich, W., and Hullin, M. B. 2014. Diffuse mirrors: 3D reconstruction from diffuse indirect illumination using inexpensive time-of-flight sensors. In Proc. CVPR. Google ScholarDigital Library
- Heide, F., Xiao, L., Kolb, A., Hullin, M. B., and Heidrich, W. 2014. Imaging in scattering media using correlation image sensors and sparse convolutional coding. Optics Express 22, 21, 26338--26350.Google ScholarCross Ref
- Heide, F., Heidrich, W., Hullin, M., and Wetzstein, G. 2015. Doppler Time-of-Flight Imaging. ACM Trans. Graph. (SIGGRAPH), 4. Google ScholarDigital Library
- Holloway, J., Salman Asif, M., Sharma, M. K., Matsuda, N., Horstmeyer, R., Cossairt, O., and Veeraraghavan, A. 2015. Toward Long Distance, Sub-diffraction Imaging Using Coherent Camera Arrays. ArXiv 1510.08470.Google Scholar
- Jayasuriya, S., Pediredla, A., Sivaramakrishnan, S., Molnar, A., and Veeraraghavan, A. 2015. Depth fields: Extending light field techniques to time-of-flight imaging. In Proc. 3DV, 1--9. Google ScholarDigital Library
- Jimenez, D., Pizarro, D., Mazo, M., and Palazuelos, S. 2012. Modelling and correction of multipath interference in time of flight cameras. In Proc. CVPR. Google ScholarDigital Library
- Kadambi, A., Whyte, R., Bhandari, A., Streeter, L., Barsi, C., Dorrington, A., and Raskar, R. 2013. Coded time of flight cameras: sparse deconvolution to address multipath interference and recover time profiles. ACM Trans. Graph. (SIGGRAPH Asia) 32, 6. Google ScholarDigital Library
- Kadambi, A., Bhandari, A., Whyte, R., Dorrington, A., and Raskar, R. 2014. Demultiplexing Illumination via Low Cost Sensing and Nanosecond Coding. In Proc. ICCP.Google Scholar
- Kim, S. K., Kang, B., Heo, J., Jung, S.-W., and Choi, O. 2014. Photometric stereo-based single time-of-flight camera. Optics Letters 39, 1, 166--169.Google ScholarCross Ref
- Kirmani, A., Hutchison, T., Davis, J., and Raskar, R. 2009. Looking around the corner using transient imaging. In Proc. ICCV, 159--166.Google Scholar
- Lange, R., and Seitz, P. 2001. Solid-state time-of-flight range camera. IEEE J. Quantum Electronics 37, 3, 390--397.Google ScholarCross Ref
- Levoy, M., and Hanrahan, P. 1996. Light field rendering. In Proc. SIGGRAPH, 31--42. Google ScholarDigital Library
- Li, L., Xiang, S., Yang, Y., and Yu, L. 2015. Multi-camera interference cancellation of time-of-flight (tof) cameras. In Proc. IEEE ICIP, 556--560.Google Scholar
- Maimone, A., and Fuchs, H. 2012. Reducing interference between multiple structured light depth sensors using motion. In Proc. VR. Google ScholarDigital Library
- Matusik, W., Buehler, C., Raskar, R., Gortler, S. J., and McMillan, L. 2000. Image-based visual hulls. In Proc. SIGGRAPH, 369--374. Google ScholarDigital Library
- McCandless, S. W., and Jackson, C. R. 2004. Principles of synthetic aperture radar. In AR Marine Users Manual, J. Fagerberg, D. C. Mowery, and R. R. Nelson, Eds. NOAA, ch. 1, 11.Google Scholar
- Naik, N., Zhao, S., Velten, A., Raskar, R., and Bala, K. 2011. Single view reflectance capture using multiplexed scattering and time-of-flight imaging. ACM Trans. Graph. (SIGGRAPH Asia) 30, 6, 171:1--171:10. Google ScholarDigital Library
- Naik, N., Kadambi, A., Rhemann, C., Izadi, S., Raskar, R., and Kang, S. 2015. A light transport model for mitigating multipath interference in tof sensors. In Proc. CVPR.Google Scholar
- O'Toole, M., Heide, F., Xiao, L., Hullin, M. B., Heidrich, W., and Kutulakos, K. N. 2014. Temporal frequency probing for 5d transient analysis of global light transport. ACM Trans. Graph. (SIGGRAPH) 33, 4, 87:1--87:11. Google ScholarDigital Library
- Payne, A., Jongenelen, A., Dorrington, A., Cree, M., and Carnegie, D. 2009. Multiple Frequency Range Imaging to Remove Measurement Ambiguity. In Proc. Optical 3-D measurement techniques IX.Google Scholar
- Peters, C., Klein, J., Hullin, M. B., and Klein, R. 2015. Solving trigonometric moment problems for fast transient imaging. ACM Trans. Graph. (SIGGRAPH Asia) 34, 6. Google ScholarDigital Library
- Rander, P., Narayanan, P. J., and Kanade, T. 1997. Virtualized reality: Constructing time-varying virtual worlds from real events. In Proc. IEEE Visualization, 277--283. Google ScholarDigital Library
- Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., and Blake, A. 2011. Real-time human pose recognition in parts from single depth images. In Proc. CVPR. Google ScholarDigital Library
- Tadano, R., Pediredla, A. K., and Veeraraghavan, A. 2015. Depth selective camera: A direct, on-chip, programmable technique for depth selectivity in photography. In Proc. IEEE ICCV. Google ScholarDigital Library
- Ti, C., Yang, R., Davis, J., and Pan, Z. 2015. Simultaneous Time-of-Flight Sensing and Photometric Stereo With a Single ToF Sensor. In Proc. CVPR.Google Scholar
- Velten, A., Willwacher, T., Gupta, O., Veeraraghavan, A., Bawendi, M., and Raskar, R. 2012. Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging. Nat Commun 745, 3.Google Scholar
- Velten, A., Wu, D., Jarabo, A., Masia, B., Barsi, C., Joshi, C., Lawson, E., Bawendi, M., Gutierrez, D., and Raskar, R. 2013. Femto-photography: Capturing and visualizing the propagation of light. ACM Trans. Graph. (SIGGRAPH) 32, 4, 44:1--44:8. Google ScholarDigital Library
- Wilburn, B., Joshi, N., Vaish, V., Talvala, E.-V., Antunez, E., Barth, A., Adams, A., Horowitz, M., and Levoy, M. 2005. High performance imaging using large camera arrays. ACM Trans. Graph. (SIGGRAPH) 24, 3, 765--776. Google ScholarDigital Library
- Woodham, R. J. 1980. Photometric method for determining surface orientation from multiple images. Optical Engineering 19, 1.Google ScholarCross Ref
- Wu, D., Wetzstein, G., Barsi, C., Willwacher, T., O'Toole, M., Naik, N., Dai, Q., Kutulakos, K., and Raskar, R. 2012. Frequency analysis of transient light transport with applications in bare sensor imaging. In Proc. ECCV. Google ScholarDigital Library
Index Terms
- Computational imaging with multi-camera time-of-flight systems
Recommendations
Doppler time-of-flight imaging
Over the last few years, depth cameras have become increasingly popular for a range of applications, including human-computer interaction and gaming, augmented reality, machine vision, and medical imaging. Many of the commercially-available devices use ...
Panorama light-field imaging
SIGGRAPH '12: ACM SIGGRAPH 2012 PostersWe present a first approach towards panorama light-field imaging. By converting overlapping sub-light-fields into individual focal stacks, computing a panoramic focal stack from them, and converting the panoramic focal stack back into a panoramic light ...
Panorama light-field imaging
SIGGRAPH '12: ACM SIGGRAPH 2012 TalksWe present a first approach towards panorama light-field imaging. By converting overlapping sub-light-fields into individual focal stacks, computing a panoramic focal stack from them, and converting the panoramic focal stack back into a panoramic light ...
Comments