Skip to main content
SpringerLink
Log in

A framework for developing and benchmarking sampling and denoising algorithms for Monte Carlo rendering

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Although many adaptive sampling and reconstruction techniques for Monte Carlo (MC) rendering have been proposed in the last few years, the case for which one should be used for a specific scene is still to be made. Moreover, developing a new technique has required selecting a particular rendering system, which makes the technique tightly coupled to the chosen renderer and limits the amount of scenes it can be tested on to those available for that renderer. In this paper, we propose a renderer-agnostic framework for testing and benchmarking sampling and denoising techniques for MC rendering, which allows an algorithm to be easily deployed to multiple rendering systems and tested on a wide variety of scenes. Our system achieves this by decoupling the techniques from the rendering systems, hiding the renderer details behind an API. This improves productivity and allows for direct comparisons among techniques originally developed for different rendering systems. We demonstrate the effectiveness of our API by using it to instrument four rendering systems and then using them to benchmark several state-of-the-art MC denoising techniques and sampling strategies.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. https://doi.org/10.7919/F46H4FGW.

References

  1. Anderson, L., Li, T.-M., Lehtinen, J., Durand, F.: Aether: An embedded domain specific sampling language for Monte Carlo rendering. ACM Trans. Graph. 36(4), 99:1–99:16 (2017). https://doi.org/10.1145/3072959.3073704

    Article  Google Scholar 

  2. Baker, S., Scharstein, D., Lewis, J., Roth, S., Black, M., Szeliski, R.: Middlebury Flow Accuracy and Interpolation Evaluation. http://vision.middlebury.edu/flow/eval/ (2011). Accessed 4 Mar 2018

  3. Baker, S., Scharstein, D., Lewis, J.P., Roth, S., Black, M.J., Szeliski, R.: A database and evaluation methodology for optical flow. IJCV 92(1), 1–31 (2011)

    Article  Google Scholar 

  4. Bako, S., Vogels, T., Mcwilliams, B., Meyer, M., Novák, J., Harvill, A., Sen, P., Derose, T., Rousselle, F.: Kernel-predicting convolutional networks for denoising Monte Carlo renderings. ACM Trans. Graph. 36(4), 97:1–97:14 (2017)

    Article  Google Scholar 

  5. Barron, J.L., Fleet, D.J., Beauchemin, S.S.: Performance of optical flow techniques. IJCV 12(1), 43–77 (1994)

    Article  Google Scholar 

  6. Bauszat, P., Eisemann, M., Eisemann, E., Magnor, M.: General and robust error estimation and reconstruction for Monte Carlo rendering. Comput. Graph. Forum 34(2), 597–608 (2015)

    Article  Google Scholar 

  7. Bitterli, B., Rousselle, F., Moon, B., Iglesias-Guitián, J.A., Adler, D., Mitchell, K., Jarosz, W., Novák, J.: Nonlinearly weighted first-order regression for denoising Monte Carlo renderings. Comput. Graph. Forum 35(4), 107–117 (2016)

    Article  Google Scholar 

  8. Buck, I., Foley, T., Horn, D., Sugerman, J., Fatahalian, K., Houston, M., Hanrahan, P.: Brook for GPUs: stream computing on graphics hardware. ACM Trans. Graph. 23(3), 777–786 (2004)

    Article  Google Scholar 

  9. Chaitanya, C.R.A., Kaplanyan, A.S., Schied, C., Salvi, M., Lefohn, A., Nowrouzezahrai, D., Aila, T.: Interactive reconstruction of Monte Carlo image sequences using a recurrent denoising autoencoder. ACM Trans. Graph. 36(4), 98:1–98:12 (2017)

    Article  Google Scholar 

  10. Cook, R.L., Porter, T., Carpenter, L.: Distributed ray tracing. In: Proceedings of SIGGRAPH ’84, pp. 137–145

  11. Delbracio, M., Musé, P., Buades, A., Chauvier, J., Phelps, N., Morel, J.M.: Boosting Monte Carlo rendering by ray histogram fusion. ACM Trans. Graph. 33(1), 8:1–8:15 (2014)

    Article  MATH  Google Scholar 

  12. Erofeev, M., Gitman, Y., Vatolin, D., Fedorov, A., Wang, J.: Videomatting. http://videomatting.com/ (2014). Accessed 4 Mar 2018

  13. Erofeev, M., Gitman, Y., Vatolin, D., Fedorov, A., Wang, J.: Perceptually motivated benchmark for video matting. In: Xie, X., Jones, M.W., Tam, G.K.L. (eds.) BMVA Press Proceedings, pp. 99.1–99.12 (2015)

  14. Hachisuka, T., Jarosz, W., Weistroffer, R.P., Dale, K., Humphreys, G., Zwicker, M., Jensen, H.W.: Multidimensional adaptive sampling and reconstruction for ray tracing. ACM TOG 27(212), 1 (2008)

    Google Scholar 

  15. Heck, D., Schlömer, T., Deussen, O.: Blue noise sampling with controlled aliasing. ACM Trans. Graph. 32(3), 25:1–25:12 (2013)

    Article  MATH  Google Scholar 

  16. Jakob, W.: Mitsuba renderer. http://www.mitsuba-renderer.org (2010). Accessed 4 Mar 2018

  17. Kajiya, J.T.: The rendering equation. SIGGRAPH’86 20(4), 143–150 (1986)

    Article  Google Scholar 

  18. Kalantari, N.K., Bako, S., Sen, P.: A machine learning approach for filtering Monte Carlo noise. ACM Trans. Graph. 34(4), 122:1–122:12 (2015)

    Article  Google Scholar 

  19. Kalantari, N.K., Sen, P.: Removing the noise in Monte Carlo rendering with general image denoising algorithms. Comput. Graph. Forum 32(2pt1), 93–102 (2013)

    Article  Google Scholar 

  20. Lee, M.E., Redner, R.A.: Filtering: a note on the use of nonlinear filtering in computer graphics. IEEE Comput. Graph. Appl. 10(3), 23–29 (1990)

    Article  Google Scholar 

  21. Li, T.M., Wu, Yt, Chuang, Yy: SURE-based optimization for adaptive sampling and reconstruction. ACM Trans. Graph. 31, 1 (2012)

    Google Scholar 

  22. Mark, W.R., Glanville, R.S., Akeley, K., Kilgard, M.J.: Cg: a system for programming graphics hardware in a c-like language. ACM Trans. Graph. 22(3), 896–907 (2003)

    Article  Google Scholar 

  23. Moon, B., Carr, N., Yoon, S.E.: Adaptive rendering based on weighted local regression. ACM Trans. Graph. 33(5), 170:1–170:14 (2014)

    Article  Google Scholar 

  24. Mullapudi, R.T., Adams, A., Sharlet, D., Ragan-Kelley, J., Fatahalian, K.: Automatically scheduling halide image processing pipelines. ACM Trans. Graph. 35(4), 83:1–83:11 (2016)

    Article  Google Scholar 

  25. Pharr, M., Humphreys, G.: Physically Based Rendering, from Theory to Implementation, 2nd edn. Morgan Kaufmann, Los Altos (2010)

    Google Scholar 

  26. Pharr, M., Jakob, W., Humphreys, G.: Physically Based Rendering, from Theory to Implementation, 3rd edn. Morgan Kaufmann, Los Altos (2016)

    Google Scholar 

  27. Ragan-Kelley, J., Adams, A., Paris, S., Levoy, M., Amarasinghe, S., Durand, F.: Decoupling algorithms from schedules for easy optimization of image processing pipelines. ACM Trans. Graph. 31(4), 32:1–32:12 (2012)

    Article  Google Scholar 

  28. Ren, P., Wang, J., Gong, M., Lin, S., Tong, X., Guo, B.: Global illumination with radiance regression functions. ACM Trans. Graph. 32, 1 (2013)

    MATH  Google Scholar 

  29. Rhemann, C., Rother, C., Wang, J., Gelautz, M., Kohli, P., Rott, P.: Alpha matting evaluation website. http://www.alphamatting.com/eval_25.php (2009). Accessed 4 Mar 2018

  30. Rhemann, C., Rother, C., Wang, J., Gelautz, M., Kohli, P., Rott, P.: A perceptually motivated online benchmark for image matting. In: CVPR, pp. 1826–1833 (2009)

  31. Rousselle, F., Knaus, C., Zwicker, M.: Adaptive sampling and reconstruction using greedy error minimization. ACM Trans. Graph. 30(6), 159:1–159:12 (2011)

    Article  Google Scholar 

  32. Rousselle, F., Knaus, C., Zwicker, M.: Adaptive rendering with non-local means filtering. ACM Trans. Graph. 31(6), 195:1–195:11 (2012)

    Article  Google Scholar 

  33. Rousselle, F., Manzi, M., Zwicker, M.: Robust denoising using feature and color information. Comput. Graph. Forum 32(7), 121–130 (2013)

    Article  Google Scholar 

  34. Rushmeier, H.E., Ward, G.J.: Energy preserving non-linear filters. In: Proceedings of SIGGRAPH ’94, pp. 131–138 (1994)

  35. Samet, H.: Sorting in space: multidimensional, spatial, and metric data structures for computer graphics applications. In: ACM SIGGRAPH ASIA 2010 Courses, SA ’10, pp. 3:1–3:52 (2010)

  36. Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. IJCV 47(1–3), 7–42 (2002)

    Article  MATH  Google Scholar 

  37. Scharstein, D., Szeliski, R., Hirschmüller, H.: Middlebury Stereo Vision Page. http://vision.middlebury.edu/stereo/ (2002). Accessed 4 Mar 2018

  38. Sen, P., Darabi, S.: Implementation of Random Parameter Filtering. Tech. Rep. EECE-TR-11-0004, University of New Mexico (2011)

  39. Sen, P., Darabi, S.: On filtering the noise from the random parameters in Monte Carlo rendering. ACM Trans. Graph. 31(3), 1–15 (2012)

    Article  Google Scholar 

  40. Stein, C.M.: Estimation of the mean of a multivariate normal distribution. Ann. Stat. 9(6), 1135–1151 (1981)

    Article  MathSciNet  MATH  Google Scholar 

  41. Veach, E., Guibas, L.J.: Metropolis light transport. In: Proceedings of SIGGRAPH ’97, pp. 65–76 (1997)

  42. Zwicker, M., Jarosz, W., Lehtinen, J., Moon, B., Ramamoorthi, R., Rousselle, F., Sen, P., Soler, C., Yoon, S.E.: Recent advances in adaptive sampling and reconstruction for Monte Carlo rendering. Comput. Graph. Forum 34(2), 667–681 (2015)

    Article  Google Scholar 

Download references

Acknowledgements

The authors thank Matt Pharr, Greg Humphreys, and Wenzel Jakob for making the source code of PBRT-v2, PBRT-v3, and Mitsuba publicly available. We also thank the authors of the following techniques for kindly providing their source code: LBF, RHF, LWR, RDFC, RPF, SBF, NLM, GEM. The following individuals and institutions provided the scenes used in the paper: Martin Lubich (Crown), Cem Yuksel (Curly Hair), Jesper Lloyd (Toy Gyro), Wojciech Jarosz (Chess), Andrew Kensler (Toasters), Bernhard Vogl and Stanford CG Lab (Furry Bunny), Mareck (Bathroom), aXel (Glass of Water), Jay-Artist (Country Kitchen), Beeple (Measure One), Anat Grynberg and GregWard (Conference), Duc Nguyen, Ron Fedkiw, and Nolan Goodnight (Smoke).

Funding

This work was funded by CAPES and CNPq-Brazil (fellowships and Grants 306196/2014-0 and 423673/2016-5), and US National Science Foundation Grants IIS-1321168 and IIS-1619376.

Author information

Authors and Affiliations

  1. Instituto de Informática, UFRGS, Porto Alegre, RS, Brazil

    Jonas Deyson Brito dos Santos & Manuel M. Oliveira

  2. University of California, Santa Barbara, USA

    Pradeep Sen

Authors
  1. Jonas Deyson Brito dos Santos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pradeep Sen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Manuel M. Oliveira
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jonas Deyson Brito dos Santos.

Ethics declarations

Conflict of interest

The authors Jonas Deyson B. Santos, Pradeep Sen, and Manuel M. Oliveira declare they have no conflict of interest.

Additional information

This work was sponsored by CAPES and CNPq-Brazil (fellowships and Grants 306196/2014-0 and 423673/2016-5), as well as US National Science Foundation Grants IIS-1321168 and IIS-1619376.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

dos Santos, J.D.B., Sen, P. & Oliveira, M.M. A framework for developing and benchmarking sampling and denoising algorithms for Monte Carlo rendering. Vis Comput 34, 765–778 (2018). https://doi.org/10.1007/s00371-018-1521-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-018-1521-y

Keywords

Search

Navigation