skip to main content
10.1145/3588432.3591533acmconferencesArticle/Chapter ViewAbstractPublication PagessiggraphConference Proceedingsconference-collections
research-article

Neural Parametric Mixtures for Path Guiding

Published: 23 July 2023 Publication History

Abstract

Previous path guiding techniques typically rely on spatial subdivision structures to approximate directional target distributions, which may cause failure to capture spatio-directional correlations and introduce parallax issue. In this paper, we present Neural Parametric Mixtures (NPM), a neural formulation to encode target distributions for path guiding algorithms. We propose to use a continuous and compact neural implicit representation for encoding parametric models while decoding them via lightweight neural networks. We then derive a gradient-based optimization strategy to directly train the parameters of NPM with noisy Monte Carlo radiance estimates. Our approach efficiently models the target distribution (incident radiance or the product integrand) for path guiding, and outperforms previous guiding methods by capturing the spatio-directional correlations more accurately. Moreover, our approach is more training efficient and is practical for parallelization on modern GPUs.

Supplemental Material

MP4 File
presentation
MP4 File
These files contain the supplemental materials of our paper, including: - a video that visualizes the training and rendering process of our method; - an interactive HTML viewer that visualizes our image results; - and the code implementations our our method.
ZIP File
These files contain the supplemental materials of our paper, including: - a video that visualizes the training and rendering process of our method; - an interactive HTML viewer that visualizes our image results; - and the code implementations our our method.
ZIP File
These files contain the supplemental materials of our paper, including: - a video that visualizes the training and rendering process of our method; - an interactive HTML viewer that visualizes our image results; - and the code implementations our our method.

References

[1]
Benedikt Bitterli. 2016. Rendering resources. https://benedikt-bitterli.me/resources/.
[2]
Norbert Bus and Tamy Boubekeur. 2017. Double Hierarchies for Directional Importance Sampling in Monte Carlo Rendering. Journal of Computer Graphics Techniques (JCGT) 6, 3 (28 August 2017), 25–37. http://jcgt.org/published/0006/03/02
[3]
R. R. Currius, D. Dolonius, U. Assarsson, and E. Sintorn. 2020. Spherical Gaussian Light-field Textures for Fast Precomputed Global Illumination. Computer Graphics Forum 39, 2 (2020), 133–146.
[4]
Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. 2017. Density estimation using Real NVP. In International Conference on Learning Representations.
[5]
Stavros Diolatzis, Julien Philip, and George Drettakis. 2022. Active Exploration for Neural Global Illumination of Variable Scenes. ACM Transactions on Graphics (2022).
[6]
Ana Dodik, Marios Papas, Cengiz Öztireli, and Thomas Müller. 2022. Path Guiding Using Spatio-Directional Mixture Models. In Computer Graphics Forum, Vol. 41. Wiley Online Library, 172–189.
[7]
Saeed Hadadan, Shuhong Chen, and Matthias Zwicker. 2021. Neural radiosity. ACM Transactions on Graphics (TOG) 40, 6 (2021), 1–11.
[8]
Sebastian Herholz, Oskar Elek, Jiří Vorba, Hendrik Lensch, and Jaroslav Křivánek. 2016. Product importance sampling for light transport path guiding. In Computer Graphics Forum, Vol. 35. Wiley Online Library, 67–77.
[9]
Yuchi Huo, Rui Wang, Ruzahng Zheng, Hualin Xu, Hujun Bao, and Sung-Eui Yoon. 2020. Adaptive incident radiance field sampling and reconstruction using deep reinforcement learning. ACM Transactions on Graphics (TOG) 39, 1 (2020), 1–17.
[10]
Wenzel Jakob. 2012. Numerically stable sampling of the von Mises-Fisher distribution on Sˆ2 (and other tricks). Interactive Geometry Lab, ETH Zürich, Tech. Rep (2012), 6.
[11]
Henrik Wann Jensen. 1995. Importance driven path tracing using the photon map. In Eurographics Workshop on Rendering Techniques. Springer, 326–335.
[12]
James T. Kajiya. 1986. The Rendering Equation. SIGGRAPH Comput. Graph. (1986).
[13]
Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. ICLR (2015).
[14]
Eric P Lafortune and Yves D Willems. 1995. A 5D tree to reduce the variance of Monte Carlo ray tracing. In Eurographics Workshop on Rendering Techniques. Springer, 11–20.
[15]
Samuli Laine, Tero Karras, and Timo Aila. 2013. Megakernels considered harmful: Wavefront path tracing on GPUs. In Proceedings of the 5th High-Performance Graphics Conference. 137–143.
[16]
Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. 2020. NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. In ECCV.
[17]
Thomas Müller. 2019. "Practical Path Guiding" in Production. In ACM SIGGRAPH 2019 Courses(SIGGRAPH ’19). ACM, New York, NY, USA, Article 18, 77 pages.
[18]
Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. 2022. Instant Neural Graphics Primitives with a Multiresolution Hash Encoding. ACM Trans. Graph. 41, 4, Article 102 (July 2022), 15 pages.
[19]
Thomas Müller, Markus Gross, and Jan Novák. 2017. Practical path guiding for efficient light-transport simulation. In Computer Graphics Forum, Vol. 36. Wiley Online Library, 91–100.
[20]
Thomas Müller, Brian McWilliams, Fabrice Rousselle, Markus Gross, and Jan Novák. 2019. Neural importance sampling. ACM Transactions on Graphics (TOG) 38, 5 (2019), 1–19.
[21]
Thomas Müller, Fabrice Rousselle, Alexander Keller, and Jan Novák. 2020. Neural control variates. ACM Transactions on Graphics (TOG) 39, 6 (2020), 1–19.
[22]
Thomas Müller, Fabrice Rousselle, Jan Novák, and Alexander Keller. 2021. Real-Time Neural Radiance Caching for Path Tracing. ACM Trans. Graph. 40, 4, Article 36 (jul 2021), 16 pages.
[23]
Jacob Munkberg, Jon Hasselgren, Tianchang Shen, Jun Gao, Wenzheng Chen, Alex Evans, Thomas Mueller, and Sanja Fidler. 2022. Extracting Triangular 3D Models, Materials, and Lighting From Images. CVPR (2022).
[24]
Thomas Müller. 2021. tiny-cuda-nn. https://github.com/NVlabs/tiny-cuda-nn
[25]
Steven G Parker, James Bigler, Andreas Dietrich, Heiko Friedrich, Jared Hoberock, David Luebke, David McAllister, Morgan McGuire, Keith Morley, Austin Robison, 2010. Optix: a general purpose ray tracing engine. ACM Transactions on Graphics (TOG) 29, 4 (2010), 1–13.
[26]
S. Popov, R. Ramamoorthi, F. Durand, and G. Drettakis. 2015. Probabilistic Connections for Bidirectional Path Tracing. Computer Graphics Forum 34, 4 (07 2015), 75–86.
[27]
Alexander Rath, Pascal Grittmann, Sebastian Herholz, Petr Vévoda, Philipp Slusallek, and Jaroslav Křivánek. 2020. Variance-aware path guiding. ACM Transactions on Graphics (TOG) 39, 4 (2020), 151–1.
[28]
Lukas Ruppert, Sebastian Herholz, and Hendrik PA Lensch. 2020. Robust fitting of parallax-aware mixtures for path guiding. ACM Transactions on Graphics (TOG) 39, 4 (2020), 147–1.
[29]
Fujia Su, Sheng Li, and Guoping Wang. 2022. SPCBPT: Subspace-Based Probabilistic Connections for Bidirectional Path Tracing. ACM Trans. Graph. 41, 4, Article 77 (jul 2022), 14 pages. https://doi.org/10.1145/3528223.3530183
[30]
Dor Verbin, Peter Hedman, Ben Mildenhall, Todd Zickler, Jonathan T. Barron, and Pratul P. Srinivasan. 2022. Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields. CVPR (2022).
[31]
Jiří Vorba, Johannes Hanika, Sebastian Herholz, Thomas Müller, Jaroslav Křivánek, and Alexander Keller. 2019. Path Guiding in Production. In ACM SIGGRAPH 2019 Courses (Los Angeles, California) (SIGGRAPH ’19). ACM, New York, NY, USA, Article 18, 77 pages.
[32]
Jiří Vorba, Ondřej Karlík, Martin Šik, Tobias Ritschel, and Jaroslav Křivánek. 2014. On-line learning of parametric mixture models for light transport simulation. ACM Transactions on Graphics (TOG) 33, 4 (2014), 1–11.
[33]
Alex Yu, Ruilong Li, Matthew Tancik, Hao Li, Ren Ng, and Angjoo Kanazawa. 2021. PlenOctrees for Real-time Rendering of Neural Radiance Fields. In ICCV.
[34]
Shilin Zhu, Zexiang Xu, Tiancheng Sun, Alexandr Kuznetsov, Mark Meyer, Henrik Wann Jensen, Hao Su, and Ravi Ramamoorthi. 2021. Hierarchical neural reconstruction for path guiding using hybrid path and photon samples. ACM Transactions on Graphics (TOG) 40, 4 (2021), 1–16.

Cited By

View all
  • (2024)Neural Product Importance Sampling via Warp CompositionSIGGRAPH Asia 2024 Conference Papers10.1145/3680528.3687566(1-11)Online publication date: 3-Dec-2024
  • (2024)Conditional Mixture Path Guiding for Differentiable RenderingACM Transactions on Graphics10.1145/365813343:4(1-11)Online publication date: 19-Jul-2024
  • (2024)Online Neural Path Guiding with Normalized Anisotropic Spherical GaussiansACM Transactions on Graphics10.1145/364931043:3(1-18)Online publication date: 9-Apr-2024

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SIGGRAPH '23: ACM SIGGRAPH 2023 Conference Proceedings
July 2023
911 pages
ISBN:9798400701597
DOI:10.1145/3588432
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 July 2023

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Global Illumination
  2. Mixture Models
  3. Neural Networks
  4. Ray Tracing
  5. Sampling and Reconstruction

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Funding Sources

  • National Science Foundation of China

Conference

SIGGRAPH '23
Sponsor:

Acceptance Rates

Overall Acceptance Rate 1,822 of 8,601 submissions, 21%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)299
  • Downloads (Last 6 weeks)22
Reflects downloads up to 08 Mar 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Neural Product Importance Sampling via Warp CompositionSIGGRAPH Asia 2024 Conference Papers10.1145/3680528.3687566(1-11)Online publication date: 3-Dec-2024
  • (2024)Conditional Mixture Path Guiding for Differentiable RenderingACM Transactions on Graphics10.1145/365813343:4(1-11)Online publication date: 19-Jul-2024
  • (2024)Online Neural Path Guiding with Normalized Anisotropic Spherical GaussiansACM Transactions on Graphics10.1145/364931043:3(1-18)Online publication date: 9-Apr-2024

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media