Abstract
Stochastic progressive photon mapping (SPPM) is one of the important global illumination methods in computer graphics. It can simulate caustics and specular-diffuse-specular lighting effects efficiently. However, as a biased method, it always suffers from both bias and variance with limited iterations, and the bias and the variance bring multi-scale noises into SPPM renderings. Recent learning-based methods have shown great advantages on denoising unbiased Monte Carlo (MC) methods, but have not been leveraged for biased ones. In this paper, we present the first learning-based method specially designed for denoising-biased SPPM renderings. Firstly, to avoid conflicting denoising constraints, the radiance of final images is decomposed into two components: caustic and global. These two components are then denoised separately via a two-network framework. In each network, we employ a novel multi-residual block with two sizes of filters, which significantly improves the model’s capabilities, and makes it more suitable for multi-scale noises on both low-frequency and high-frequency areas. We also present a series of photon-related auxiliary features, to better handle noises while preserving illumination details, especially caustics. Compared with other state-of-the-art learning-based denoising methods that we apply to this problem, our method shows a higher denoising quality, which could efficiently denoise multi-scale noises while keeping sharp illuminations.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Hachisuka T, Jensen H W. Stochastic progressive photon mapping. ACM Transactions on Graphics, 2009, 28(5): Article No. 141.
Jensen H W. Realistic Image Synthesis Using Photon Mapping (1st edition). Routledge, 2001.
Hachisuka T, Ogaki S, Jensen H W. Progressive photon mapping. ACM Transactions on Graphics, 2008, 27(5): Article No. 130.
Kang C, Wang L, Xu Y et al. A survey of photon mapping state-of-the-art research and future challenges. Frontiers of Information Technology & Electronic Engineering, 2016, 17(3): 185-199.
Ritschel T, Dachsbacher C, Grosch T et al. The state of the art in interactive global illumination. Computer Graphics Forum, 2012, 31(1): 160-188.
Bako S, Vogels T, McWilliams B et al. Kernel-predicting convolutional networks for denoising Monte Carlo renderings. ACM Transactions on Graphics, 2017, 36(4): Article No. 97.
Vogels T, Rousselle F, McWilliams B et al. Denoising with kernel prediction and asymmetric loss functions. ACM Transactions on Graphics, 2018, 37(4): Article No. 124.
Wong K M, Wong T T. Robust deep residual denoising for Monte Carlo rendering. In Proc. the 2018 SIGGRAPH Asia Technical Briefs, December 2018, Article No. 14.
Wong K M, Wong T T. Deep residual learning for denoising Monte Carlo renderings. Computational Visual Media, 2019, 5(3): 239-255.
Kalantari N K, Bako S, Sen P. A machine learning approach for filtering Monte Carlo noise. ACM Transactions on Graphics, 2015, 34(4): Article No. 122.
Xu B, Zhang J, Wang R et al. Adversarial Monte Carlo denoising with conditioned auxiliary feature modulation. ACM Transactions on Graphics, 2019, 38(6): Article No. 224.
Yang X, Wang D, Hu W et al. DEMC: A deep dual-encoder network for denoising Monte Carlo rendering. Journal of Computer Science and Technology, 2019, 34(5): 1123-1135.
Gharbi M, Li T M, Aittala M et al. Sample-based Monte Carlo denoising using a kernel-splatting network. ACM Transactions on Graphics, 2019, 38(4): Article No. 125.
Günther T, Grosch T. Distributed out-of-core stochastic progressive photon mapping. Computer Graphics Forum, 2014, 33(6): 154-166.
Havran V, Bittner J, Herzog R et al. Ray maps for global illumination. Rendering Techniques, 2005, 2005: 43-54.
Frisvad J R, Schjøth L, Erleben K et al. Photon differential splatting for rendering caustics. Computer Graphics Forum, 2014, 33(6): 252-263.
Spencer B, Jones M W. Into the blue: Better caustics through photon relaxation. Computer Graphics Forum, 2009, 28(2): 319-328.
Fu Z, Jensen H W. Noise reduction for progressive photon mapping. In Proc. the International Conference on Computer Graphics and Interactive Techniques, August 2012, Article No. 29.
Kaplanyan A S, Dachsbacher C. Adaptive progressive photon mapping. ACM Transactions on Graphics, 2013, 32(2): Article No. 16.
Bengio Y, Simard P, Frasconi P. Learning long-term dependencies with gradient descent is difficult. IEEE Transactions on Neural Networks, 1994, 5(2): 157-166.
He K, Zhang X, Ren S et al. Identity mappings in deep residual networks. In Proc. the 14th European Conference on Computer Vision, October 2016, pp.630-645.
He K, Zhang X, Ren S et al. Deep residual learning for image recognition. In Proc. the 2016 IEEE Conference on Computer Vision and Pattern Recognition, June 2016, pp.770-778.
Zagoruyko S, Komodakis N. Wide residual networks. arXiv:1605.07146, 2016. https://arxiv.org/abs/1605.07146, Feb. 2019.
Abdi M, Nahavandi S. Multi-residual networks: Improving the speed and accuracy of residual networks. arXiv:1609.05672, 2016. https://arxiv.org/abs/1609.05672, Feb. 2019.
Silverman B W. Density Estimation for Statistics and Data Analysis. Chapman and Hall, 1986.
Pharr M, Jakob W, Humphreys G. Physically Based Rendering: From Theory to Implementation (3rd edition). Morgan Kaufmann, 2016.
He K, Zhang X, Ren S et al. Delving deep into rectifiers: Surpassing human-level performance on imageNet classification. In Proc. the IEEE International Conference on Computer Vision, December 2015, pp.1026-1034.
Schjøth L, Sporring J, Olsen F O. Diffusion based photon mapping. Computer Graphics Forum, 2008, 27(8): 2114- 2127.
Abadi M, Barham P, Chen J et al. TensorFlow: A system for large-scale machine learning. In Proc. the 12th USENIX Symposium on Operating Systems Design and Implementation, November 2016, pp.265-283.
Kingma D P, Ba J. Adam: A method for stochastic optimization. arXiv:1412.6980, 2014. https://arxiv.org/abs/1412.6980, Dec. 2019.
Glorot X, Bengio Y. Understanding the difficulty of training deep feed forward neural networks. In Proc. the 13th International Conference on Artificial Intelligence and Statistics, May 2010, pp.249-256.
Author information
Authors and Affiliations
Corresponding authors
Electronic supplementary material
ESM 1
(PDF 220 kb)
Rights and permissions
About this article
Cite this article
Zeng, Z., Wang, L., Wang, BB. et al. Denoising Stochastic Progressive Photon Mapping Renderings Using a Multi-Residual Network. J. Comput. Sci. Technol. 35, 506–521 (2020). https://doi.org/10.1007/s11390-020-0264-1
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11390-020-0264-1