Fabrice Rousselle
Matthias Zwicker
Abstract
We propose a novel approach for image space adaptive sampling and filtering in Monte Carlo rendering. We use an iterative scheme composed of three steps. First, we adaptively distribute samples in the image plane. Second, we denoise the image using a non-linear filter. Third, we estimate the residual per-pixel error of the filtered rendering, and the error estimate guides the sample distribution in the next iteration. The effectiveness of our approach hinges on the use of a state of the art image denoising technique, which we extend to an adaptive rendering framework. A key idea is to split the Monte Carlo samples into two buffers. This improves denoising performance and facilitates variance and error estimation. Our method relies only on the Monte Carlo samples, allowing us to handle arbitrary light transport and lens effects. In addition, it is robust to high noise levels and complex image content. We compare our approach to a state of the art adaptive rendering technique based on adaptive bandwidth selection and demonstrate substantial improvements in terms of both numerical error and visual quality. Our framework is easy to implement on top of standard Monte Carlo renderers and it incurs little computational overhead.
- Bala, K., Walter, B., and Greenberg, D. P. 2003. Combining edges and points for interactive high-quality rendering. ACM Trans. Graph. 22 (July), 631--640. Google Scholar
Digital Library
- Bauszat, P., Eisemann, M., and Magnor, M. 2011. Guided image filtering for interactive high-quality global illumination. Computer Graphics Forum (Proc. of Eurographics Symposium on Rendering (EGSR)) 30, 4 (June), 1361--1368. Google ScholarDigital Library
- Bolin, M. R., and Meyer, G. W. 1998. A perceptually based adaptive sampling algorithm. In SIGGRAPH '98, 299--309. Google ScholarDigital Library
- Buades, A., Coll, B., Morel, J., et al. 2005. A review of image denoising algorithms, with a new one. SIAM Journal on Multiscale Modeling and Simulation 4, 2, 490--530.Google ScholarCross Ref
- Buades, A., Coll, B., and Morel, J. 2008. Nonlocal image and movie denoising. International Journal of Computer Vision 76, 2, 123--139. Google ScholarDigital Library
- Chen, J., Wang, B., Wang, Y., Overbeck, R. S., Yong, J.-H., and Wang, W. 2011. Efficient depth-of-field rendering with adaptive sampling and multiscale reconstruction. Computer Graphics Forum.Google Scholar
- Dabov, K., Foi, A., Katkovnik, V., and Egiazarian, K. 2007. Image denoising by sparse 3-d transform-domain collaborative filtering. Image Processing, IEEE Transactions on 16, 8 (aug.), 2080--2095. Google ScholarDigital Library
- Dammertz, H., Sewtz, D., Hanika, J., and Lensch, H. P. A. 2010. Edge-avoiding à-trous wavelet transform for fast global illumination filtering. In Proceedings of the Conference on High Performance Graphics, Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, HPG '10, 67--75. Google ScholarDigital Library
- Deussen, O., Hanrahan, P., Lintermann, B., Měch, R., Pharr, M., and Prusinkiewicz, P. 1998. Realistic modeling and rendering of plant ecosystems. In Proceedings of the 25th annual conference on Computer graphics and interactive techniques, ACM, New York, NY, USA, SIGGRAPH '98, 275--286. Google ScholarDigital Library
- Donoho, D. L., and Johnstone, J. M. 1994. Ideal spatial adaptation by wavelet shrinkage. Biometrika 81, 3, 425--455.Google ScholarCross Ref
- Egan, K., Tseng, Y.-T., Holzschuch, N., Durand, F., and Ramamoorthi, R. 2009. Frequency analysis and sheared reconstruction for rendering motion blur. ACM Trans. Graph. 28 (July), 93:1--93:13. Google ScholarDigital Library
- Egan, K., Hecht, F., Durand, F., and Ramamoorthi, R. 2011. Frequency analysis and sheared filtering for shadow light fields of complex occluders. ACM Transactions on Graphics 30, 2 (Apr.), 9:1--9:13. Google ScholarDigital Library
- Gastal, E. S. L., and Oliveira, M. M. 2012. Adaptive manifolds for real-time high-dimensional filtering. ACM Trans. Graph. 31, 4 (July), 33:1--33:13. Google ScholarDigital Library
- Hachisuka, T., Jarosz, W., Weistroffer, R. P., Dale, K., Humphreys, G., Zwicker, M., and Jensen, H. W. 2008. Multidimensional adaptive sampling and reconstruction for ray tracing. ACM Trans. Graph. 27 (August), 33:1--33:10. Google ScholarDigital Library
- Ji, Z., Chen, Q., Sun, Q., and Xia, D. 2009. A moment-based nonlocal-means algorithm for image denoising. Information Processing Letters 109, 23, 1238--1244. Google ScholarDigital Library
- Kervrann, C., and Boulanger, J. 2006. Optimal spatial adaptation for patch-based image denoising. Image Processing, IEEE Transactions on 15, 10, 2866--2878. Google ScholarDigital Library
- Kollig, T., and Keller, A. 2002. Efficient multidimensional sampling. Computer Graphics Forum 21, 3, 557--563.Google ScholarCross Ref
- Lehtinen, J., Aila, T., Chen, J., Laine, S., and Durand, F. 2011. Temporal light field reconstruction for rendering distribution effects. ACM Trans. Graph. 30 (August), 55:1--55:12. Google ScholarDigital Library
- Liu, Y., Wang, J., Chen, X., Guo, Y., and Peng, Q. 2008. A robust and fast non-local means algorithm for image denoising. Journal of Computer Science and Technology 23, 2, 270--279. Google ScholarDigital Library
- Mairal, J., Elad, M., and Sapiro, G. 2008. Sparse representation for color image restoration. Image Processing, IEEE Transactions on 17, 1 (jan.), 53--69. Google ScholarDigital Library
- Mitchell, D. P. 1987. Generating antialiased images at low sampling densities. SIGGRAPH Comput. Graph. 21 (August), 65--72. Google ScholarDigital Library
- Overbeck, R. S., Donner, C., and Ramamoorthi, R. 2009. Adaptive wavelet rendering. ACM Trans. Graph. 28 (December), 140:1--140:12. Google ScholarDigital Library
- Parker, S. G., Bigler, J., Dietrich, A., Friedrich, H., Hoberock, J., Luebke, D., McAllister, D., McGuire, M., Morley, K., Robison, A., and Stich, M. 2010. Optix: a general purpose ray tracing engine. In ACM SIGGRAPH 2010 papers, ACM, New York, NY, USA, SIGGRAPH '10, 66:1--66:13. Google ScholarDigital Library
- Pharr, M., and Humphreys, G. 2010. Physically Based Rendering, Second Edition: From Theory To Implementation, 2nd ed. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA. Google ScholarDigital Library
- Portilla, J., Strela, V., Wainwright, M., and Simoncelli, E. 2003. Image denoising using scale mixtures of gaussians in the wavelet domain. Image Processing, IEEE Transactions on 12, 11, 1338--1351. Google ScholarDigital Library
- Ritschel, T., Engelhardt, T., Grosch, T., Seidel, H.-P., Kautz, J., and Dachsbacher, C. 2009. Micro-rendering for scalable, parallel final gathering. ACM Trans. Graph. 28 (December), 132:1--132:8. Google ScholarDigital Library
- Rousselle, F., Knaus, C., and Zwicker, M. 2011. Adaptive sampling and reconstruction using greedy error minimization. In Proceedings of the 2011 SIGGRAPH Asia Conference, ACM, New York, NY, USA, SA '11, 159:1--159:12. Google ScholarDigital Library
- Sen, P., and Darabi, S. 2012. On filtering the noise from the random parameters in monte carlo rendering. ACM Trans. Graph. 31, 3 (June), 18:1--18:15. Google ScholarDigital Library
- Shirley, P., Aila, T., Cohen, J., Enderton, E., Laine, S., Luebke, D., and McGuire, M. 2011. A local image reconstruction algorithm for stochastic rendering. In Symposium on Interactive 3D Graphics and Games, ACM, New York, NY, USA, I3D '11, 9--14 PAGE@5. Google ScholarDigital Library
- Sijbers, J., Den Dekker, A., Van Audekerke, J., Verhoye, M., and Van Dyck, D. 1998. Estimation of the noise in magnitude mr images. Magnetic Resonance Imaging 16, 1, 87--90.Google ScholarCross Ref
- Soler, C., Subr, K., Durand, F., Holzschuch, N., and Sillion, F. 2009. Fourier depth of field. ACM Trans. Graph. 28 (May), 18:1--18:12. Google ScholarDigital Library
- Tomasi, C., and Manduchi, R. 1998. Bilateral filtering for gray and color images. In Computer Vision, 1998. Sixth International Conference on, IEEE, 839--846. Google ScholarDigital Library
- Veach, E., and Guibas, L. J. 1997. Metropolis light transport. In Proceedings of the 24th annual conference on Computer graphics and interactive techniques, ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, SIGGRAPH '97, 65--76. Google ScholarDigital Library
- Wang, Z., Bovik, A., Sheikh, H., and Simoncelli, E. 2004. Image quality assessment: from error visibility to structural similarity. Image Processing, IEEE Transactions on 13, 4 (april), 600--612. Google ScholarDigital Library
- Xu, R., and Pattanaik, S. 2005. Non-iterative, robust monte carlo noise reduction. IEEE Computer Graphics and Applications 25, 2, 31--35. Google ScholarDigital Library
Index Terms
- Adaptive rendering with non-local means filtering
Recommendations
Adaptive sampling and reconstruction using greedy error minimization
We introduce a novel approach for image space adaptive sampling and reconstruction in Monte Carlo rendering. We greedily minimize relative mean squared error (MSE) by iterating over two steps. First, given a current sample distribution, we optimize over ...
A framework for developing and benchmarking sampling and denoising algorithms for Monte Carlo rendering
Although many adaptive sampling and reconstruction techniques for Monte Carlo (MC) rendering have been proposed in the last few years, the case for which one should be used for a specific scene is still to be made. Moreover, developing a new technique ...
Factored axis-aligned filtering for rendering multiple distribution effects
Monte Carlo (MC) ray-tracing for photo-realistic rendering often requires hours to render a single image due to the large sampling rates needed for convergence. Previous methods have attempted to filter sparsely sampled MC renders but these methods have ...
Comments