skip to main content
research-article

Fast volume rendering with spatiotemporal reservoir resampling

Published:10 December 2021Publication History
Skip Abstract Section

Abstract

Volume rendering under complex, dynamic lighting is challenging, especially if targeting real-time. To address this challenge, we extend a recent direct illumination sampling technique, spatiotemporal reservoir resampling, to multi-dimensional path space for volumetric media.

By fully evaluating just a single path sample per pixel, our volumetric path tracer shows unprecedented convergence. To achieve this, we properly estimate the chosen sample's probability via approximate perfect importance sampling with spatiotemporal resampling. A key observation is recognizing that applying cheaper, biased techniques to approximate scattering along candidate paths (during resampling) does not add bias when shading. This allows us to combine transmittance evaluation techniques: cheap approximations where evaluations must occur many times for reuse, and unbiased methods for final, per-pixel evaluation.

With this reformulation, we achieve low-noise, interactive volumetric path tracing with arbitrary dynamic lighting, including volumetric emission, and maintain interactive performance even on high-resolution volumes. When paired with denoising, our low-noise sampling helps preserve smaller-scale volumetric details.

Skip Supplemental Material Section

Supplemental Material

a279-lin.mp4

mp4

472.3 MB

References

  1. Philippe Bekaert, Mateu Sbert, and John H Halton. 2002. Accelerating Path Tracing by Re-Using Paths. In Rendering Techniques. 125--134. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Nir Benty, Kai-Hwa Yao, Petrik Clarberg, Lucy Chen, Simon Kallweit, Tim Foley, Matthew Oakes, Conor Lavelle, and Chris Wyman. 2020. The Falcor Rendering Framework. https://github.com/NVIDIAGameWorks/FalcorGoogle ScholarGoogle Scholar
  3. Benedikt Bitterli. 2021. Correlations and reuse for fast and accurate physically based light transport. Ph.D. Dissertation. Dartmouth College.Google ScholarGoogle Scholar
  4. Benedikt Bitterli and Wojciech Jarosz. 2017. Beyond points and beams: Higher-dimensional photon samples for volumetric light transport. ACM Trans. Graph. 36, 4 (2017), 1--12. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Benedikt Bitterli, Chris Wyman, Matt Pharr, Peter Shirley, Aaron Lefohn, and Wojciech Jarosz. 2020. Spatiotemporal reservoir resampling for real-time ray tracing with dynamic direct lighting. ACM Trans. Graph. 39, 4 (2020), 148:1--148:17.Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Jakub Boksansky, Paula Jukarainen, and Chris Wyman. 2021. Rendering Many Lights with Grid-Based Reservoirs. In Ray Tracing Gems II. Springer, 351--365.Google ScholarGoogle Scholar
  7. Min-Te Chao. 1982. A general purpose unequal probability sampling plan. Biometrika 69, 3 (1982), 653--656.Google ScholarGoogle ScholarCross RefCross Ref
  8. Cyril Delalandre, Pascal Gautron, Jean-Eudes Marvie, and Guillaume François. 2011. Transmittance function mapping. In Symposium on Interactive 3D Graphics and Games. 31--38. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Xi Deng, Shaojie Jiao, Benedikt Bitterli, and Wojciech Jarosz. 2019. Photon surfaces for robust, unbiased volumetric density estimation. ACM Trans. Graph. 38, 4 (2019), 46. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Eugene d'Eon and Jan Novák. 2021. Zero-variance Transmittance Estimation. In Eurographics Symposium on Rendering - DL-only Track.Google ScholarGoogle Scholar
  11. Mathieu Galtier, Stephane Blanco, Cyril Caliot, Christophe Coustet, Jérémi Dauchet, Mouna El Hafi, Vincent Eymet, Richard Fournier, Jacques Gautrais, Anais Khuong, et al. 2013. Integral formulation of null-collision Monte Carlo algorithms. Journal of Quantitative Spectroscopy and Radiative Transfer 125 (2013), 57--68.Google ScholarGoogle ScholarCross RefCross Ref
  12. Pascal Gautron, Cyril Delalandre, Jean-Eudes Marvie, and Pascal Lecocq. 2013. Boundary-aware extinction mapping. Computer Graphics Forum 32, 7 (2013), 305--314.Google ScholarGoogle ScholarCross RefCross Ref
  13. Iliyan Georgiev, Jaroslav Krivanek, Toshiya Hachisuka, Derek Nowrouzezahrai, and Wojciech Jarosz. 2013. Joint importance sampling of low-order volumetric scattering. ACM Trans. Graph. 32, 6 (2013), 164--1. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Iliyan Georgiev, Zackary Misso, Toshiya Hachisuka, Derek Nowrouzezahrai, Jaroslav Křivánek, and Wojciech Jarosz. 2019. Integral formulations of volumetric transmittance. ACM Trans. Graph. 38, 6 (2019), 1--17. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Adrien Gruson, Binh-Son Hua, Nicolas Vibert, Derek Nowrouzezahrai, and Toshiya Hachisuka. 2018. Gradient-domain volumetric photon density estimation. ACM Trans. Graph. 37, 4 (2018), 1--13. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Louis G Henyey and Jesse L Greenstein. 1941. Diffuse radiation in the galaxy. Astro-physical Journal 93 (1941), 70--83. Google ScholarGoogle ScholarCross RefCross Ref
  17. Sebastian Herholz, Yangyang Zhao, Oskar Elek, Derek Nowrouzezahrai, Hendrik PA Lensch, and Jaroslav Křivánek. 2019. Volume path guiding based on zero-variance random walk theory. ACM Trans. Graph. 38, 3 (2019), 1--19. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Sébastien Hillaire. 2015. Physically-based and Unified Volumetric Rendering in Frostbite. In SIGGRAPH Courses: Advances in Real-Time Rendering in Games.Google ScholarGoogle Scholar
  19. Rama Karl Hoetzlein. 2016. GVDB: Raytracing Sparse Voxel Database Structures on the GPU. In High Performance Graphics. 109--117. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Nikolai Hofmann, Jon Hasselgren, Petrik Clarberg, and Jacob Munkberg. 2021. Interactive Path Tracing and Reconstruction of Sparse Volumes. Proc. ACM Comput. Graph. Interact. Tech. 4, 1, Article 5 (April 2021), 19 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Jon Jansen and Louis Bavoil. 2010. Fourier opacity mapping. In Proceedings of the 2010 ACM SIGGRAPH symposium on Interactive 3D Graphics and Games. 165--172. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Wojciech Jarosz, Derek Nowrouzezahrai, Iman Sadeghi, and Henrik Wann Jensen. 2011. A comprehensive theory of volumetric radiance estimation using photon points and beams. ACM Trans. Graph. 30, 1 (2011), 1--19. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Wojciech Jarosz, Matthias Zwicker, and Henrik Wann Jensen. 2008. The beam radiance estimate for volumetric photon mapping. In ACM SIGGRAPH 2008 classes. 1--112. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Henrik Wann Jensen and Per H Christensen. 1998. Efficient simulation of light transport in scenes with participating media using photon maps. In Proceedings of the 25th annual conference on Computer graphics and interactive techniques. 311--320. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Anton Kaplanyan and Carsten Dachsbacher. 2010. Cascaded Light Propagation Volumes for Real-Time Indirect Illumination. In Proceedings of the 2010 ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games. 99--107. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Markus Kettunen, Eugene D'Eon, Jacopo Pantaleoni, and Jan Novák. 2021. An Unbiased Ray-Marching Transmittance Estimator. ACM Trans. Graph. 40, 4, Article 137 (July 2021), 20 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Markus Kettunen, Marco Manzi, Miika Aittala, Jaakko Lehtinen, Frédo Durand, and Matthias Zwicker. 2015. Gradient-Domain Path Tracing. ACM Trans. Graph. 34, 4, Article 123 (July 2015), 13 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Emmett Kilgariff, Henry Moreton, Nick Stam, and Brandon Bell. 2018. NVIDIA Turing Architecture In-Depth. https://developer.nvidia.com/blog/nvidia-turing-architecture-in-depth/. [Online; accessed 19-January-2021].Google ScholarGoogle Scholar
  29. Tae-Yong Kim and Ulrich Neumann. 2001. Opacity shadow maps. In Rendering Techniques 2001. Springer, 177--182. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Jaroslav Křivánek and Eugene d'Eon. 2014. A zero-variance-based sampling scheme for Monte Carlo subsurface scattering. In ACM SIGGRAPH 2014 Talks. 1--1. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Jaroslav Křivánek, Iliyan Georgiev, Toshiya Hachisuka, Petr Vévoda, Martin Šik, Derek Nowrouzezahrai, and Wojciech Jarosz. 2014. Unifying points, beams, and paths in volumetric light transport simulation. ACM Trans. Graph. 33, 4 (2014), 1--13. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Christopher Kulla and Marcos Fajardo. 2012. Importance sampling techniques for path tracing in participating media. Computer graphics forum 31, 4 (2012), 1519--1528. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Peter Kutz, Ralf Habel, Yining Karl Li, and Jan Novák. 2017. Spectral and decomposition tracking for rendering heterogeneous volumes. ACM Trans. Graph. 36, 4 (2017), 1--16. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Jaakko Lehtinen, Tero Karras, Samuli Laine, Miika Aittala, Frédo Durand, and Timo Aila. 2013. Gradient-Domain Metropolis Light Transport. 32, 4, Article 95 (July 2013), 12 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Zander Majercik, Jean-Philippe Guertin, Derek Nowrouzezahrai, and Morgan McGuire. 2019. Dynamic Diffuse Global Illumination with Ray-Traced Irradiance Fields. Journal of Computer Graphics Techniques (JCGT) 8, 2 (5 June 2019), 1--30.Google ScholarGoogle Scholar
  36. Johannes Meng, Johannes Hanika, and Carsten Dachsbacher. 2016. Improving the Dwivedi sampling scheme. Computer Graphics Forum 35, 4 (2016), 37--44. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Bailey Miller, Iliyan Georgiev, and Wojciech Jarosz. 2019. A null-scattering path integral formulation of light transport. ACM Trans. Graph. 38, 4 (2019), 1--13. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Pierre Moreau, Matt Pharr, Petrik Clarberg, M Steinberger, and T Foley. 2019. Dynamic Many-Light Sampling for Real-Time Ray Tracing. In High Performance Graphics (Short Papers). 21--26.Google ScholarGoogle Scholar
  39. Ken Museth. 2013. VDB: High-Resolution Sparse Volumes with Dynamic Topology. ACM Trans. Graph. 32, 3, Article 27 (2013), 22 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Jan Novák, Derek Nowrouzezahrai, Carsten Dachsbacher, and Wojciech Jarosz. 2012a. Progressive virtual beam lights. Computer Graphics Forum 31, 4 (2012), 1407--1413. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Jan Novák, Derek Nowrouzezahrai, Carsten Dachsbacher, and Wojciech Jarosz. 2012b. Virtual ray lights for rendering scenes with participating media. ACM Trans. Graph. 31, 4 (2012), 1--11. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Jan Novák, Andrew Selle, and Wojciech Jarosz. 2014. Residual ratio tracking for estimating attenuation in participating media. ACM Trans. Graph. 33, 6 (2014), 179--1. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Jan Novák, Iliyan Georgiev, Johannes Hanika, and Wojciech Jarosz. 2018. Monte Carlo methods for volumetric light transport simulation. Computer Graphics Forum (Proceedings of Eurographics - State of the Art Reports) 37, 2 (May 2018).Google ScholarGoogle ScholarCross RefCross Ref
  44. NVIDIA. 2017. NVIDIA® OptiX™ AI-Accelerated Denoiser. https://developer.nvidia.com/optix-denoiserGoogle ScholarGoogle Scholar
  45. Yaobin Ouyang, Shiqiu Liu, Markus Kettunen, Matt Pharr, and Jacopo Pantaleoni. 2021. ReSTIR GI: Path Resampling for Real-Time Path Tracing. Computer Graphics Forum (2021).Google ScholarGoogle Scholar
  46. Matthias Raab, Daniel Seibert, and Alexander Keller. 2008. Unbiased global illumination with participating media. In Monte Carlo and Quasi-Monte Carlo Methods 2006. Springer, 591--605.Google ScholarGoogle Scholar
  47. Marco Salvi, Kiril Vidimče, Andrew Lauritzen, and Aaron Lefohn. 2010. Adaptive volumetric shadow maps. Computer Graphics Forum 29, 4 (2010), 1289--1296. Florian Simon, Johannes Hanika, Tobias Zirr, and Carsten Dachsbacher. 2017. Line Integration for Rendering Heterogeneous Emissive Volumes. Computer Graphics Forum 36 (July 2017), 101--110. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. TM Sutton, FB Brown, FG Bischoff, DB MacMillan, CL Ellis, JT Ward, CT Ballinger, DJ Kelly, and L Schindler. 1999. The physical models and statistical procedures used in the RACER Monte Carlo code. Technical Report. Knolls Atomic Power Lab.Google ScholarGoogle Scholar
  49. László Szirmay-Kalos, Iliyan Georgiev, Milán Magdics, Balazs Molnar, and David Legrady. 2017. Unbiased Light Transport Estimators for Inhomogeneous Participating Media. Computer Graphics Forum 36 (May 2017), 9--19. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. László Szirmay-Kalos, Balázs Tóth, and Milán Magdics. 2011. Free Path Sampling in High Resolution Inhomogeneous Participating Media. Computer Graphics Forum 30 (March 2011), 85--97.Google ScholarGoogle Scholar
  51. Justin Talbot, David Cline, and Parris Egbert. 2005. Importance Resampling for Global Illumination. In Eurographics Symposium on Rendering. 139--146. Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Justin F Talbot. 2005. Importance resampling for global illumination. Master's thesis. Brigham Young University.Google ScholarGoogle Scholar
  53. Jiří Vorba, Johannes Hanika, Sebastian Herholz, Thomas Müller, Jaroslav Křivánek, and Alexander Keller. 2019. Path Guiding in Production. In ACM SIGGRAPH 2019 Courses (Los Angeles, California). ACM, New York, NY, USA, Article 18, 77 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. E Woodcock, T Murphy, P Hemmings, and S Longworth. 1965. Techniques used in the GEM code for Monte Carlo neutronics calculations in reactors and other systems of complex geometry. In Proc. Conf. Applications of Computing Methods to Reactor Problems, Vol. 557.Google ScholarGoogle Scholar
  55. Chris Wyman. 2016. Exploring and Expanding the Continuum of OIT Algorithms. In High Performance Graphics. 1--11. Chris Wyman and Alexey Panteleev. 2021. Rearchitecting Spatiotemporal Resampling for Production. In High-Performance Graphics - Symposium Papers. Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. Yonghao Yue, Kei Iwasaki, B. Chen, Y. Dobashi, and T. Nishita. 2011. Toward Optimal Space Partitioning for Unbiased, Adaptive Free Path Sampling of Inhomogeneous Participating Media. Computer Graphics Forum 30 (2011).Google ScholarGoogle Scholar
  57. Cem Yuksel and John Keyser. 2008. Deep Opacity Maps. Computer Graphics Forum (Proceedings of EUROGRAPHICS 2008) 27, 2 (2008), 675--680.Google ScholarGoogle Scholar

Index Terms

  1. Fast volume rendering with spatiotemporal reservoir resampling

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Graphics
      ACM Transactions on Graphics  Volume 40, Issue 6
      December 2021
      1351 pages
      ISSN:0730-0301
      EISSN:1557-7368
      DOI:10.1145/3478513
      Issue’s Table of Contents

      Copyright © 2021 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 10 December 2021
      Published in tog Volume 40, Issue 6

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader