skip to main content
10.1145/3587423.3595511acmconferencesArticle/Chapter ViewAbstractPublication PagessiggraphConference Proceedingsconference-collections
course

A Gentle Introduction to ReSTIR Path Reuse in Real-Time

Published: 24 July 2023 Publication History

Abstract

In recent years, reservoir-based spatiotemporal importance resampling (ReSTIR) algorithms appeared out of nowhere to take parts of the realtime rendering community by storm, with sample reuse speeding direct lighting from millions of dynamic lights [1], diffuse multi-bounce lighting [2], participating media [3], and even complex global illumination paths [4]. Highly optimized variants (e.g. [5]) can give 100x efficiency improvement over traditional ray- and path-tracing methods; this is key to achieve 30 or 60 Hz framerates. In production engines, tracing even one ray or path per pixel may only be feasible on the highest-end systems, so maximizing image quality per sample is vital.
ReSTIR builds on the math in Talbot et al.'s [6] resampled importance sampling (RIS), which previously was not widely used or taught, leaving many practitioners missing key intuitions and theoretical grounding. A firm grounding is vital, as seemingly obvious "optimizations" arising during ReSTIR engine integration can silently introduce conditional probabilities and dependencies that, left ignored, add uncontrollable bias to the results.
In this course, we plan to:
1. Provide concrete motivation and intuition for why ReSTIR works, where it applies, what assumptions it makes, and the limitations of today's theory and implementations;
2. Gently develop the theory, targeting attendees with basic Monte Carlo sampling experience but without prior knowledge of resampling algorithms (e.g., Talbot et al. [6]);
3. Give explicit algorithmic samples and pseudocode, pointing out easily-encountered pitfalls when implementing ReSTIR;
4. Discuss actual game integrations, highlighting the gotchas, challenges, and corner cases we encountered along the way, and highlighting ReSTIR's practical benefits.

Supplemental Material

MP4 File
presentation

References

[1]
Benedikt Bitterli et al. 'Spatiotemporal Reservoir Resampling for Real-Time Ray Tracing with Dynamic Direct Lighting'. In: ACM Transactions on Graphics (Proceedings of SIGGRAPH) 39.4 (July 8, 2020). (Visited on 08/23/2020) (cited on pages i, ii, 1, 11--13, 21, 22).
[2]
Yaobin Ouyang et al. 'ReSTIR GI: Path Resampling for Real-Time Path Tracing'. In: Computer Graphics Forum 40.8 (2021), pp. 17--29. (cited on pages i, ii, 15).
[3]
Daqi Lin, Chris Wyman, and Cem Yuksel. 'Fast Volume Rendering with Spatiotemporal Reservoir Resampling'. In: ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia) 40.6 (Dec. 10, 2021), 279:1--279:18. (cited on pages i, ii).
[4]
Daqi Lin et al. 'Generalized Resampled Importance Sampling: Foundations of ReSTIR'. In: ACM Transactions on Graphics (Proceedings of SIGGRAPH) 41.4 (July 22, 2022), 75:1--75:23. (Visited on 07/23/2022) (cited on pages i--iii, 4--6, 9--11, 16, 17).
[5]
Chris Wyman and Alexey Panteleev. 'Rearchitecting Spatiotemporal Resampling for Production'. In: High-Performance Graphics - Symposium Papers. Eurographics Association, 2021. (cited on pages i, ii, 12, 23).
[6]
Justin F. Talbot, David Cline, and Parris Egbert. 'Importance Resampling for Global Illumination'. In: Rendering Techniques (Proceedings of the Eurographics Symposium on Rendering). Eurographics Association, June 2005, pp. 139--146. (cited on pages i, iii, iv, 1, 4, 8, 12).
[7]
Benedikt Bitterli. 'Correlations and Reuse for Fast and Accurate Physically Based Light Transport'. PhD thesis. Hanover, NH: Dartmouth College, Jan. 1, 2022 (cited on pages ii, iv, 12).
[8]
Jakub Boksansky, Paula Jukarainen, and Chris Wyman. 'Rendering Many Lights with Grid-Based Reservoirs'. In: Ray Tracing Gems II: Next Generation Real-Time Rendering with DXR, Vulkan, and OptiX. Ed. by Adam Marrs, Peter Shirley, and Ingo Wald. Berkeley, CA: Apress, 2021, pp. 351--365. (Visited on 02/10/2023) (cited on pages ii, 21).
[9]
NVIDIA. NVIDIA® RTX Direct Illumination. 2021. url: https://developer.nvidia.com/rtxdi (visited on 05/28/2021) (cited on page ii).
[10]
Alex Battaglia. Sword and Fairy 7 is the cutting-edge PC exclusive nobody's talking about. 2021. url: https://www.eurogamer.net/digitalfoundry-2021-sword-and-fairy-7-tech-review (visited on 02/07/2023) (cited on page ii).
[11]
Andrew Burnes. Portal with RTX Out Now: A Breathtaking Reimagining Of Valve's Classic With Full Ray Tracing & DLSS 3. 2022. url: https://www.nvidia.com/en-us/geforce/news/portal-with-rtx-ray-tracing/ (visited on 02/07/2023) (cited on page ii).
[12]
Jiayin Cao. Understanding The Math Behind ReSTIR DI. 2022. url: https://agraphicsguynotes.com/posts/understanding_the_math_behind_restir_di/ (visited on 02/07/2023) (cited on page ii).
[13]
Julien Guertault. Reading list on ReSTIR. 2022. url: https://lousodrome.net/blog/light/2022/05/14/reading-list-on-restir/ (visited on 02/07/2023) (cited on page ii).
[14]
Shubham Sachdeva. Spatiotemporal Reservoir Resampling (ReSTIR) - Theory and Basic Implementation. 2021. url: https://gamehacker1999.github.io/posts/restir/ (visited on 02/07/2023) (cited on page ii).
[15]
Jacco Bikker. Lecture 14 - "TAA & ReSTIR". 2023. url: http://www.cs.uu.nl/docs/vakken/magr/2022-2023/slides/lecture%2014%20-%20ReSTIR.pdf (visited on 02/07/2023) (cited on page ii).
[16]
Tomasz Stachowiak. Global Illumination in 'kajiya' Renderer. 2022. url: https://github.com/EmbarkStudios/kajiya/blob/main/docs/gi-overview.md (visited on 02/07/2023) (cited on page ii).
[17]
Mr. Zyanide. Shared Twitter results for Jedi Outcast integration. 2023. url: https://twitter.com/MZyanide/status/1610172199146586112 (visited on 02/07/2023) (cited on page ii).
[18]
Adam Badke. 'Next event estimation via reservoir-based spatio-temporal importance resampling'. MA thesis. Simon Fraser University, June 2021 (cited on page ii).
[19]
Ege Ciklabakkal et al. 'Single-Pass Stratified Importance Resampling'. In: Computer Graphics Forum (Proceedings of the Eurographics Symposium on Rendering) 41.4 (2022). (Visited on 07/21/2022) (cited on page ii).
[20]
Guillaume Boissé. 'World-Space Spatiotemporal Reservoir Reuse for Ray-Traced Global Illumination'. In: SIGGRAPH Asia 2021 Technical Communications. New York, NY, USA: ACM Press, Dec. 14, 2021, pp. 1--4. (cited on pages ii, 21).
[21]
Xander Hermans. 'The Effectiveness of the ReSTIR Technique When Ray Tracing a Voxel World'. MA thesis. Utrecht University, July 2022. (Visited on 02/10/2023) (cited on page ii).
[22]
Fuyan Liu and Junwen Gan. 'Light Subpath Reservoir for Interactive Ray-Traced Global Illumination'. In: Computers & Graphics 111 (Apr. 1, 2023), pp. 37--46. (cited on page ii).
[23]
Shinji Ogaki. 'Vectorized Reservoir Sampling'. In: SIGGRAPH Asia 2021 Technical Communications. New York, NY, USA: ACM Press, Dec. 14, 2021, pp. 1--4. (Visited on 02/09/2023) (cited on page ii).
[24]
Stefan Krake. hdRstr: A ReSTIR/RTXDI-based Hydra Render Delegate. 2021. url: https://stkrake.net/ (visited on 02/07/2023) (cited on page ii).
[25]
Stefan Krake. blRstr: A ReSTIR/RTXDI-based Blender Render Engine. 2022. url: https://stkrake.net/blRstr/ (visited on 02/07/2023) (cited on page ii).
[26]
Eric Veach and Leonidas J. Guibas. 'Optimally Combining Sampling Techniques for Monte Carlo Rendering'. In: Annual Conference Series (Proceedings of SIGGRAPH). Vol. 29. ACM Press, Aug. 1995, pp. 419--428. (cited on pages iv, 9, 24).
[27]
Markus Kettunen et al. 'Gradient-Domain Path Tracing'. In: ACM Transactions on Graphics (Proceedings of SIGGRAPH) 34.4 (July 27, 2015), p. 123. (cited on pages iv, 16, 17, 19).
[28]
Min-Te Chao. 'A General Purpose Unequal Probability Sampling Plan'. In: Biometrika 69.3 (Dec. 1, 1982), pp. 653--656. (cited on pages iv, 1, 11, 21).
[29]
Christoph Schied et al. 'Spatiotemporal Variance-Guided Filtering: Real-Time Reconstruction for Path-Traced Global Illumination'. In: Proceedings of High Performance Graphics. New York, NY, USA: ACM, 2017, 2:1--2:12. (Visited on 12/10/2019) (cited on page 1).
[30]
Christoph Schied, Christoph Peters, and Carsten Dachsbacher. 'Gradient Estimation for Real-Time Adaptive Temporal Filtering'. In: Proceedings of the ACM on Computer Graphics and Interactive Techniques 1.2 (Aug. 2018), 24:1--24:16. (Visited on 12/10/2019) (cited on page 1).
[31]
Takafumi Saito and Tokiichiro Takahashi. 'Comprehensible Rendering of 3-D Shapes'. In: Computer Graphics (Proceedings of SIGGRAPH) 24.4 (Sept. 1990), pp. 197--206. (cited on page 1).
[32]
Eric Veach. 'Robust Monte Carlo Methods for Light Transport Simulation'. PhD thesis. Stanford University, Dec. 1997 (cited on page 2).
[33]
Jaakko Lehtinen et al. 'Gradient-Domain Metropolis Light Transport'. In: ACM Transactions on Graphics (Proceedings of SIGGRAPH) 32.4 (July 2013), 95:1--95:12. (cited on pages 14, 16, 17).
[34]
Yusuke Tokuyoshi. 'Efficient Spatial Resampling Using the PDF Similarity'. In: Proceedings of the ACM on Computer Graphics and Interactive Techniques 6.1 (2023), pp. 1--19 (cited on page 16).
[35]
Marco Manzi et al. 'Improved Sampling for Gradient-Domain Metropolis Light Transport'. In: ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia) 33.6 (2014). (cited on pages 16, 17).
[36]
Binh-Son Hua et al. 'A Survey on Gradient-Domain Rendering'. In: Computer Graphics Forum. Vol. 38. 2. Wiley Online Library. 2019, pp. 455--472 (cited on pages 16, 17).
[37]
Anton S. Kaplanyan, Johannes Hanika, and Carsten Dachsbacher. 'The Natural-Constraint Representation of the Path Space for Efficient Light Transport Simulation'. In: ACM Transactions on Graphics (Proceedings of SIGGRAPH) 33.4 (July 2014), 102:1--102:13. (cited on page 17).
[38]
Wenzel Jakob and Steve Marschner. 'Manifold Exploration: A Markov Chain Monte Carlo Technique for Rendering Scenes with Difficult Specular Transport'. In: ACM Transactions on Graphics (Proceedings of SIGGRAPH) 31.4 (July 2012), 58:1--58:13. (cited on page 17).

Cited By

View all
  • (2024)Cache Points for Production-Scale Occlusion-Aware Many-Lights Sampling and Volumetric ScatteringProceedings of the 2024 Digital Production Symposium10.1145/3665320.3670993(1-19)Online publication date: 24-Jul-2024
  • (2024)Area ReSTIR: Resampling for Real-Time Defocus and AntialiasingACM Transactions on Graphics10.1145/365821043:4(1-13)Online publication date: 19-Jul-2024
  • (2024)Enhancing Spatiotemporal Resampling with a Novel MIS WeightComputer Graphics Forum10.1111/cgf.1504943:2Online publication date: 27-Apr-2024

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SIGGRAPH '23: ACM SIGGRAPH 2023 Courses
July 2023
2170 pages
ISBN:9798400701450
DOI:10.1145/3587423
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 July 2023

Check for updates

Qualifiers

  • Course

Conference

SIGGRAPH '23
Sponsor:

Acceptance Rates

Overall Acceptance Rate 1,822 of 8,601 submissions, 21%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)412
  • Downloads (Last 6 weeks)35
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Cache Points for Production-Scale Occlusion-Aware Many-Lights Sampling and Volumetric ScatteringProceedings of the 2024 Digital Production Symposium10.1145/3665320.3670993(1-19)Online publication date: 24-Jul-2024
  • (2024)Area ReSTIR: Resampling for Real-Time Defocus and AntialiasingACM Transactions on Graphics10.1145/365821043:4(1-13)Online publication date: 19-Jul-2024
  • (2024)Enhancing Spatiotemporal Resampling with a Novel MIS WeightComputer Graphics Forum10.1111/cgf.1504943:2Online publication date: 27-Apr-2024

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media