Skip to main content
SpringerLink
Log in
Book cover

European Conference on Computer Vision

ECCV 2022: Computer Vision – ECCV 2022 pp 217–233Cite as

L-Tracing: Fast Light Visibility Estimation on Neural Surfaces by Sphere Tracing

  • Conference paper
  • First Online:

  • 2090 Accesses

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13675))

Abstract

We introduce a highly efficient light visibility estimation method, called L-Tracing, for reflectance factorization on neural implicit surfaces. Light visibility is indispensable for modeling shadows and specular of high quality on object’s surface. For neural implicit representations, former methods of computing light visibility suffer from efficiency and quality drawbacks. L-Tracing leverages the distance meaning of the Signed Distance Function(SDF), and computes the light visibility of the solid object surface according to binary geometry occlusions. We prove the linear convergence of L-Tracing algorithm and give out the theoretical lower bound of tracing iteration. Based on L-Tracing, we propose a new surface reconstruction and reflectance factorization framework. Experiments show our framework performs nearly speedup on factorization, and achieves competitive albedo and relighting results with existing approaches.

The work is done when Ziyu Chen is an intern at SenseTime.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Barron, J.T., Malik, J.: Shape, illumination, and reflectance from shading. IEEE Trans. Pattern Anal. Mach. Intell. 37(8), 1670–1687 (2014)

    Article  Google Scholar 

  2. Bi, S., et al.: Neural reflectance fields for appearance acquisition. arXiv preprint arXiv:2008.03824 (2020)

  3. Boss, M., Braun, R., Jampani, V., Barron, J.T., Liu, C., Lensch, H.: NeRD: neural reflectance decomposition from image collections. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 12684–12694 (2021)

    Google Scholar 

  4. Boss, M., Groh, F., Herholz, S., Lensch, H.P.: Deep dual loss BRDF parameter estimation. In: MAM@ EGSR, pp. 41–44 (2018)

    Google Scholar 

  5. Boss, M., Jampani, V., Braun, R., Liu, C., Barron, J., Lensch, H.: Neural-PIL: neural pre-integrated lighting for reflectance decomposition. In: Advances in Neural Information Processing Systems, vol. 34 (2021)

    Google Scholar 

  6. Chen, Z., Zhang, H.: Learning implicit fields for generative shape modeling. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5939–5948 (2019)

    Google Scholar 

  7. Debevec, P.: Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography. In: ACM SIGGRAPH 2008 Classes, pp. 1–10 (2008)

    Google Scholar 

  8. Deng, K., Liu, A., Zhu, J.Y., Ramanan, D.: Depth-supervised NeRF: fewer views and faster training for free. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12882–12891 (2022)

    Google Scholar 

  9. Fridovich-Keil, S., Yu, A., Tancik, M., Chen, Q., Recht, B., Kanazawa, A.: Plenoxels: radiance fields without neural networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5501–5510 (2022)

    Google Scholar 

  10. Garbin, S.J., Kowalski, M., Johnson, M., Shotton, J., Valentin, J.: FastNeRF: high-fidelity neural rendering at 200FPS. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 14346–14355 (2021)

    Google Scholar 

  11. Kajiya, J.T.: The rendering equation. In: Proceedings of the 13th Annual Conference on Computer Graphics and Interactive Techniques, pp. 143–150 (1986)

    Google Scholar 

  12. Knodt, J., Baek, S.H., Heide, F.: Neural ray-tracing: learning surfaces and reflectance for relighting and view synthesis. arXiv preprint arXiv:2104.13562 (2021)

  13. Kuang, Z., Olszewski, K., Chai, M., Huang, Z., Achlioptas, P., Tulyakov, S.: NeROIC: neural rendering of objects from online image collections. arXiv preprint arXiv:2201.02533 (2022)

  14. Lensch, H.P.A., Kautz, J., Goesele, M., Heidrich, W., Seidel, H.-P.: Image-based reconstruction of spatially varying materials. In: Gortler, S.J., Myszkowski, K. (eds.) EGSR 2001. E, pp. 103–114. Springer, Vienna (2001). https://doi.org/10.1007/978-3-7091-6242-2_10

    Chapter  Google Scholar 

  15. Lensch, H.P., Lang, J., Sá, A.M., Seidel, H.P.: Planned sampling of spatially varying BRDFs. In: Computer Graphics Forum, vol. 22, pp. 473–482. Wiley Online Library (2003)

    Google Scholar 

  16. Li, Z., Shafiei, M., Ramamoorthi, R., Sunkavalli, K., Chandraker, M.: Inverse rendering for complex indoor scenes: shape, spatially-varying lighting and svbrdf from a single image. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2475–2484 (2020)

    Google Scholar 

  17. Martin-Brualla, R., Radwan, N., Sajjadi, M.S., Barron, J.T., Dosovitskiy, A., Duckworth, D.: NeRF in the wild: neural radiance fields for unconstrained photo collections. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7210–7219 (2021)

    Google Scholar 

  18. Matusik, W.: A data-driven reflectance model. Ph.D. thesis, Massachusetts Institute of Technology (2003)

    Google Scholar 

  19. Michalkiewicz, M., Pontes, J.K., Jack, D., Baktashmotlagh, M., Eriksson, A.: Implicit surface representations as layers in neural networks. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4743–4752 (2019)

    Google Scholar 

  20. Mildenhall, B., Hedman, P., Martin-Brualla, R., Srinivasan, P.P., Barron, J.T.: NeRF in the dark: high dynamic range view synthesis from noisy raw images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16190–16199 (2022)

    Google Scholar 

  21. Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: NeRF: representing scenes as neural radiance fields for view synthesis. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12346, pp. 405–421. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58452-8_24

    Chapter  Google Scholar 

  22. Munkberg, J., et al.: Extracting triangular 3D models, materials, and lighting from images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8280–8290 (2022)

    Google Scholar 

  23. Niemeyer, M., Geiger, A.: GIRAFFE: representing scenes as compositional generative neural feature fields. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11453–11464 (2021)

    Google Scholar 

  24. Oechsle, M., Peng, S., Geiger, A.: UNISURF: unifying neural implicit surfaces and radiance fields for multi-view reconstruction. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5589–5599 (2021)

    Google Scholar 

  25. Park, J.J., Florence, P., Straub, J., Newcombe, R., Lovegrove, S.: DeepSDF: learning continuous signed distance functions for shape representation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 165–174 (2019)

    Google Scholar 

  26. Srinivasan, P.P., Deng, B., Zhang, X., Tancik, M., Mildenhall, B., Barron, J.T.: NeRV: neural reflectance and visibility fields for relighting and view synthesis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7495–7504 (2021)

    Google Scholar 

  27. Sun, C., Sun, M., Chen, H.T.: Direct voxel grid optimization: super-fast convergence for radiance fields reconstruction. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5459–5469 (2022)

    Google Scholar 

  28. Verbin, D., Hedman, P., Mildenhall, B., Zickler, T., Barron, J.T., Srinivasan, P.P.: Ref-NeRF: structured view-dependent appearance for neural radiance fields. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5491–5500 (2022)

    Google Scholar 

  29. Wang, P., Liu, L., Liu, Y., Theobalt, C., Komura, T., Wang, W.: NeuS: learning neural implicit surfaces by volume rendering for multi-view reconstruction. In: Advances in Neural Information Processing Systems, vol. 34, pp. 27171–27183 (2021)

    Google Scholar 

  30. Wang, Z., Wu, S., Xie, W., Chen, M., Prisacariu, V.A.: NeRF-: neural radiance fields without known camera parameters. arXiv preprint arXiv:2102.07064 (2021)

  31. Wimbauer, F., Wu, S., Rupprecht, C.: De-rendering 3D objects in the wild. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18490–18499 (2022)

    Google Scholar 

  32. Wizadwongsa, S., Phongthawee, P., Yenphraphai, J., Suwajanakorn, S.: NeX: real-time view synthesis with neural basis expansion. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8534–8543 (2021)

    Google Scholar 

  33. Yariv, L., Gu, J., Kasten, Y., Lipman, Y.: Volume rendering of neural implicit surfaces. In: Advances in Neural Information Processing Systems, vol. 34 (2021)

    Google Scholar 

  34. Yifan, W., Wu, S., Oztireli, C., Sorkine-Hornung, O.: ISO-points: optimizing neural implicit surfaces with hybrid representations. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 374–383 (2021)

    Google Scholar 

  35. Yu, A., Ye, V., Tancik, M., Kanazawa, A.: pixelNeRF: neural radiance fields from one or few images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4578–4587 (2021)

    Google Scholar 

  36. Zhang, J., Yang, G., Tulsiani, S., Ramanan, D.: NeRS: neural reflectance surfaces for sparse-view 3D reconstruction in the wild. In: Advances in Neural Information Processing Systems, vol. 34 (2021)

    Google Scholar 

  37. Zhang, K., Luan, F., Wang, Q., Bala, K., Snavely, N.: PhySG: inverse rendering with spherical gaussians for physics-based material editing and relighting. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5453–5462 (2021)

    Google Scholar 

  38. Zhang, K., Riegler, G., Snavely, N., Koltun, V.: NeRF++: analyzing and improving neural radiance fields. arXiv preprint arXiv:2010.07492 (2020)

  39. Zhang, X., Srinivasan, P.P., Deng, B., Debevec, P., Freeman, W.T., Barron, J.T.: NeRFactor: neural factorization of shape and reflectance under an unknown illumination. ACM Trans. Graph. (TOG) 40(6), 1–18 (2021)

    Article  Google Scholar 

Download references

Acknowledgement

The authors Ziyu Chen and Li Song were supported by the Shanghai Key Laboratory of Digital Media Processing and Transmissions.

Author information

Authors and Affiliations

  1. Department of Electronic Engineering, Shanghai Jiao Tong University, Shanghai, China

    Ziyu Chen & Li Song

  2. SenseTime Research, Shanghai, China

    Chenjing Ding, Dongliang Wang, Yikang Li & Wei Wu

  3. Shanghai AI Laboratory, Shanghai, China

    Chenjing Ding, Jianfei Guo, Dongliang Wang, Yikang Li & Wei Wu

  4. Department of Mechanical Engineering, Tsinghua University, Beijing, China

    Xuan Xiao

Authors
  1. Ziyu Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chenjing Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jianfei Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dongliang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yikang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Xuan Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Wei Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Li Song
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Li Song .

Editor information

Editors and Affiliations

  1. Tel Aviv University, Tel Aviv, Israel

    Shai Avidan

  2. University College London, London, UK

    Gabriel Brostow

  3. Google AI, Accra, Ghana

    Moustapha Cissé

  4. University of Catania, Catania, Italy

    Giovanni Maria Farinella

  5. Facebook (United States), Menlo Park, CA, USA

    Tal Hassner

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 4091 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Chen, Z. et al. (2022). L-Tracing: Fast Light Visibility Estimation on Neural Surfaces by Sphere Tracing. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds) Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol 13675. Springer, Cham. https://doi.org/10.1007/978-3-031-19784-0_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-19784-0_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-19783-3

  • Online ISBN: 978-3-031-19784-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation