Abstract
We introduce the new non-line-of-sight imaging problem of imaging behind an occluder. The behind-an-occluder problem can be solved if the hidden space is flanked by opposing visible surfaces. We illuminate one surface and observe light that scatters off of the opposing surface after traveling through the hidden space. Hidden objects attenuate light that passes through the hidden space, leaving an observable signature that can be used to reconstruct their shape. Our method uses a simple capture setup—we use an eye-safe laser pointer as a light source and off-the-shelf RGB or RGB-D cameras to estimate the geometry of relay surfaces and observe two-bounce light. We analyze the photometric and geometric challenges of this new imaging problem, and develop a robust method that produces high-quality 3D reconstructions in uncontrolled settings where relay surfaces may be non-planar.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ahn, B., Dave, A., Veeraraghavan, A., Gkioulekas, I., Sankaranarayanan, A.C.: Convolutional approximations to the general non-line-of-sight imaging operator. In: The IEEE International Conference on Computer Vision (ICCV) (2019)
Baradad, M., et al.: Inferring light fields from shadows. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018)
Batarseh, M., Sukhov, S., Shen, Z., Gemar, H., Rezvani, R., Dogariu, A.: Passive sensing around the corner using spatial coherence. Nat. Commun. 9(1), 1–6 (2018)
Baumgart, B.: Geometric modeling for computer vision. Ph.D. thesis, Stanford University (1974)
Bouman, K.L., et al.: Turning corners into cameras: principles and methods. In: Proceedings of the IEEE International Conference on Computer Vision, vol. 572, pp. 2270–2278 (2017)
Chen, W., Daneau, S., Mannan, F., Heide, F.: Steady-state non-line-of-sight imaging. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2019)
Cheung, G.K., Kanade, T., Bouguet, J.Y., Holler, M.: A real time system for robust 3D voxel reconstruction of human motions. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR, vol. 2, pp. 714–720. IEEE (2000)
Franco, J.S., Boyer, E.: Fusion of multiview silhouette cues using a space occupancy grid. In: The IEEE International Conference on Computer Vision (ICCV), vol. 1. IEEE (2005)
Heide, F., O’Toole, M., Zang, K., Lindell, D.B., Diamond, S., Wetzstein, G.: Non-line-of-sight imaging with partial occluders and surface normals. ACM Trans. Graph. 38, 1–10 (2019)
Katz, O., Heidmann, P., Fink, M., Gigan, S.: Non-invasive real-time imaging through scattering layers and around corners via speckle correlations. Nat. Photonics 8, 784–790 (2014)
Klein, J., Peters, C., Martín, J., Laurenzis, M., Hullin, M.B.: Tracking objects outside the line of sight using 2D intensity images. Sci. Rep. 6(1), 1–9 (2016)
Landabaso, J.L., Pardas, M., Casas, J.R.: Shape from inconsistent silhouette. Comput. Vis. Image Underst. 112(2), 210–224 (2008)
Laurentini, A.: How far 3D shapes can be understood from 2D silhouettes. IEEE Trans. Pattern Anal. Mach. Intell. 3(2), 188–195 (1995)
Lazebnik, S., Furukawa, Y., Ponce, J.: Projective visual hulls. Int. J. Comput. Vision 74(2), 137–165 (2007). https://doi.org/10.1007/s11263-006-0008-x
Liu, X., et al.: Non-line-of-sight imaging using phasor-field virtual wave optics. Nature, 1–4 (2019)
Maeda, T., Satat, G., Swedish, T., Sinha, L., Raskar, R.: Recent advances in imaging around corners. arXiv preprint arXiv:1910.05613 (2019)
Martin, W., Aggarwal, J.K.: Volumetric descriptions of objects from multiple views. IEEE Trans. Pattern Anal. Mach. Intell. 5(2), 150–158 (1983)
Matusik, W., Buehler, C., McMillan, L., Gortler, S.: An efficient visual hull computation algorithm (2002)
Otsu, N.: A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man Cybern. 9(1), 62–66 (1979)
Raskar, R., Davis, J.: 5D time-light transport matrix: what can we reason about scene properties? (2008)
Saunders, C., Murray-Bruce, J., Goyal, V.: Computational periscopy with an ordinary digital camera. Nature 565, 472 (2019)
Savarese, S., Andreetto, M., Rushmeier, H., Bernardini, F., Perona, P.: 3D reconstruction by shadow carving: theory and practical evaluation. Int. J. Comput. Vision 71(3), 305–336 (2007). https://doi.org/10.1007/s11263-006-8323-9
Tabb, A.: Shape from silhouette probability maps: reconstruction of thin objects in the presence of silhouette extraction and calibration error. In: The IEEE International Conference on Computer Vision (ICCV), pp. 161–168 (2013)
Tancik, M., Satat, G., Raskar, R.: Flash photography for data-driven hidden scene recovery (2018)
Tarini, M., Callieri, M., Montani, C., Rocchini, C., Olsson, K., Persson, T.: Marching intersections: an efficient approach to shape-from-silhouette. In: 2002 Proceedings of VMV, pp. 255–262 (2002)
Thrampoulidis, C., et al.: Exploiting occlusion in non-line-of-sight active imaging. IEEE Trans. Comput. Imaging 4, 419–431 (2018)
Velten, A., Wilwacher, T., Gupta, O., Veeraraghavan, A., Bawendi, M., Raskar, R.: Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging. Nat. Commun. 3 (2012). Article Number: 745. https://www.nature.com/articles/ncomms1747?page=2
Xin, S., Nousias, S., Kutulakos, K.N., Sankaranarayanan, A.C., Narasimhan, S.G., Gkioulekas, I.: A theory of fermat paths for non-line-of-sight shape reconstruction. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6800–6809 (2019)
Yamazaki, S., Narasimhan, S.G., Baker, S., Kanade, T.: The theory and practice of coplanar shadowgram imaging for acquiring visual hulls of intricate objects. Int. J. Comput. Vision 81(3), 259–280 (2009). https://doi.org/10.1007/s11263-008-0170-4
Acknowledgements:
We thank our reviewers for their helpful comments. This work was supported by DARPA REVEAL (N00014-18-1-2894) and the Media Lab Consortium. TS was supported in part by NSF GRFP (No. 1122374).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Supplementary material 2 (mp4 9373 KB)
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Henley, C., Maeda, T., Swedish, T., Raskar, R. (2020). Imaging Behind Occluders Using Two-Bounce Light. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, JM. (eds) Computer Vision – ECCV 2020. ECCV 2020. Lecture Notes in Computer Science(), vol 12374. Springer, Cham. https://doi.org/10.1007/978-3-030-58526-6_34
Download citation
DOI: https://doi.org/10.1007/978-3-030-58526-6_34
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-58525-9
Online ISBN: 978-3-030-58526-6
eBook Packages: Computer ScienceComputer Science (R0)