skip to main content
research-article

MetaLayer: A Meta-Learned BSDF Model for Layered Materials

Published: 05 December 2023 Publication History

Abstract

Reproducing the appearance of arbitrary layered materials has long been a critical challenge in computer graphics, with regard to the demanding requirements of both physical accuracy and low computation cost. Recent studies have demonstrated promising results by learning-based representations that implicitly encode the appearance of complex (layered) materials by neural networks. However, existing generally-learned models often struggle between strong representation ability and high runtime performance, and also lack physical parameters for material editing. To address these concerns, we introduce MetaLayer, a new methodology leveraging meta-learning for modeling and rendering layered materials. MetaLayer contains two networks: a BSDFNet that compactly encodes layered materials into implicit neural representations, and a MetaNet that establishes the mapping between the physical parameters of each material and the weights of its corresponding implicit neural representation. A new positional encoding method and a well-designed training strategy are employed to improve the performance and quality of the neural model. As a new learning-based representation, the proposed MetaLayer model provides both fast responses to material editing and high-quality results for a wide range of layered materials, outperforming existing layered BSDF models.

Supplemental Material

MP4 File
supplemental
ZIP File
supplemental

References

[1]
Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dandelion Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. https://www.tensorflow.org/ Software available from tensorflow.org.
[2]
Miika Aittala, Timo Aila, and Jaakko Lehtinen. 2016. Reflectance Modeling by Neural Texture Synthesis. ACM Trans. Graph. 35, 4, Article 65 (jul 2016), 13 pages.
[3]
Marcin Andrychowicz, Misha Denil, Sergio Gómez Colmenarejo, Matthew W. Hoffman, David Pfau, Tom Schaul, Brendan Shillingford, and Nando de Freitas. 2016. Learning to Learn by Gradient Descent by Gradient Descent. In Proceedings of the 30th International Conference on Neural Information Processing Systems (Barcelona, Spain) (NIPS'16). 3988--3996.
[4]
Gladimir V. G. Baranoski and Jon G. Rokne. 2001. Efficiently simulating scattering of light by leaves. The Visual Computer 17 (2001), 491--505.
[5]
Ronen Basri, Meirav Galun, Amnon Geifman, David Jacobs, Yoni Kasten, and Shira Kritchman. 2020. Frequency Bias in Neural Networks for Input of Non-Uniform Density. In Proceedings of the 37th International Conference on Machine Learning (ICML'20). JMLR.org, Article 64, 10 pages.
[6]
Mégane Bati, Pascal Barla, and Romain Pacanowski. 2021. An Inverse Method for the Exploration of Layered Material Appearance. ACM Trans. Graph. 40, 4, Article 176 (jul 2021), 15 pages.
[7]
Mégane Bati, Romain Pacanowski, and Pascal Barla. 2019. Comparative Study of Layered Material Models. In Workshop on Material Appearance Modeling, Reinhard Klein and Holly Rushmeier (Eds.). The Eurographics Association, 17--21.
[8]
Laurent Belcour. 2018. Efficient Rendering of Layered Materials Using an Atomic Decomposition with Statistical Operators. ACM Trans. Graph. 37, 4, Article 73 (jul 2018), 15 pages.
[9]
Sai Bi, Stephen Lombardi, Shunsuke Saito, Tomas Simon, Shih-En Wei, Kevyn Mcphail, Ravi Ramamoorthi, Yaser Sheikh, and Jason Saragih. 2021. Deep Relightable Appearance Models for Animatable Faces. ACM Trans. Graph. 40, 4, Article 89 (jul 2021), 15 pages.
[10]
Qiang Dai, Jiaping Wang, Yiming Liu, John Snyder, Enhua Wu, and Baining Guo. 2009. The Dual-microfacet Model for Capturing Thin Transparent Slabs. Computer Graphics Forum 28, 7 (2009), 1917--1925.
[11]
Heloise de Dinechin and Laurent Belcour. 2022. Rendering Layered Materials with Diffuse Interfaces. Proc. ACM Comput. Graph. Interact. Tech. 5, 1, Article 13 (may 2022), 12 pages.
[12]
Valentin Deschaintre, Miika Aittala, Fredo Durand, George Drettakis, and Adrien Bousseau. 2018. Single-Image SVBRDF Capture with a Rendering-Aware Deep Network. ACM Trans. Graph. 37, 4, Article 128 (July 2018), 15 pages.
[13]
Valentin Deschaintre, Miika Aittala, Fredo Durand, George Drettakis, and Adrien Bousseau. 2019. Flexible SVBRDF Capture with a Multi-Image Deep Network. Computer Graphics Forum 38, 4 (2019), 1--13.
[14]
Yue Dong. 2019. Deep appearance modeling: A survey. Visual Informatics 3, 2 (2019), 59--68.
[15]
Julie Dorsey and Pat Hanrahan. 1996. Modeling and Rendering of Metallic Patinas. In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '96). Association for Computing Machinery, New York, NY, USA, 387--396.
[16]
Jonathan Dupuy and Wenzel Jakob. 2018. An Adaptive Parameterization for Efficient Material Acquisition and Rendering. ACM Trans. Graph. 37, 6, Article 274 (dec 2018), 14 pages.
[17]
Serkan Ergun, Sermet Önel, and Aydin Ozturk. 2016. A General Micro-flake Model for Predicting the Appearance of Car Paint. In Eurographics Symposium on Rendering - Experimental Ideas & Implementations, Elmar Eisemann and Eugene Fiume (Eds.). The Eurographics Association.
[18]
Jiahui Fan, Beibei Wang, Miloš Hašan, Jian Yang, and Ling-Qi Yan. 2022. Neural Layered BRDFs. (2022).
[19]
J. Filip and R. Vávra. 2014. Template-Based Sampling of Anisotropic BRDFs. Computer Graphics Forum 33, 7 (2014), 91--99.
[20]
Chelsea Finn, Pieter Abbeel, and Sergey Levine. 2017. Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. In Proceedings of the 34th International Conference on Machine Learning - Volume 70 (Sydney, NSW, Australia) (ICML'17). JMLR.org, 1126--1135.
[21]
Michael Fischer and Tobias Ritschel. 2022. Metappearance: Meta-Learning for Visual Appearance Reproduction. ACM Trans Graph (Proc. SIGGRAPH Asia) 41, 4 (2022).
[22]
Luis E. Gamboa, Adrien Gruson, and Derek Nowrouzezahrai. 2020. An Efficient Transport Estimator for Complex Layered Materials. Computer Graphics Forum 39, 2 (2020), 363--371.
[23]
Duan Gao, Xiao Li, Yue Dong, Pieter Peers, Kun Xu, and Xin Tong. 2019. Deep Inverse Rendering for High-Resolution SVBRDF Estimation from an Arbitrary Number of Images. ACM Trans. Graph. 38, 4, Article 134 (July 2019), 15 pages.
[24]
Jinwei Gu, Ravi Ramamoorthi, Peter Belhumeur, and Shree Nayar. 2007. Dirty Glass: Rendering Contamination on Transparent Surfaces. In Rendering Techniques, Jan Kautz and Sumanta Pattanaik (Eds.). The Eurographics Association.
[25]
Jie Guo, Shuichang Lai, Chengzhi Tao, Yuelong Cai, Lei Wang, Yanwen Guo, and Ling-Qi Yan. 2021. Highlight-Aware Two-Stream Network for Single-Image SVBRDF Acquisition. ACM Trans. Graph. 40, 4, Article 123 (jul 2021), 14 pages.
[26]
Jie Guo, Jinghui Qian, Yanwen Guo, and Jingui Pan. 2017. Rendering Thin Transparent Layers with Extended Normal Distribution Functions. IEEE Transactions on Visualization and Computer Graphics 23, 9 (2017), 2108--2119.
[27]
Yu Guo, Miloš Hašan, and Shuang Zhao. 2018. Position-Free Monte Carlo Simulation for Arbitrary Layered BSDFs. ACM Trans. Graph. 37, 6, Article 279 (dec 2018), 14 pages.
[28]
Yu Guo, Cameron Smith, Miloš Hašan, Kalyan Sunkavalli, and Shuang Zhao. 2020. MaterialGAN: Reflectance Capture Using a Generative SVBRDF Model. ACM Trans. Graph. 39, 6, Article 254 (Nov. 2020), 13 pages.
[29]
David Ha, Andrew M. Dai, and Quoc V. Le. 2017. HyperNetworks. In International Conference on Learning Representations. https://openreview.net/forum?id=rkpACe1lx
[30]
Pat Hanrahan and Wolfgang Krueger. 1993. Reflection from Layered Surfaces Due to Subsurface Scattering. Association for Computing Machinery, New York, NY, USA, 165--174.
[31]
E. Heitz and E. d'Eon. 2014. Importance Sampling Microfacet-Based BSDFs using the Distribution of Visible Normals. Computer Graphics Forum 33, 4 (2014), 103--112.
[32]
Timothy Hospedales, Antreas Antoniou, Paul Micaelli, and Amos Storkey. 2022. Meta-Learning in Neural Networks: A Survey. IEEE Transactions on Pattern Analysis and Machine Intelligence 44, 9 (2022), 5149--5169.
[33]
Bingyang Hu, Jie Guo, Yanjun Chen, Mengtian Li, and Yanwen Guo. 2020. DeepBRDF: A Deep Representation for Manipulating Measured BRDF. Computer Graphics Forum 39, 2 (2020), 157--166.
[34]
Xuecai Hu, Haoyuan Mu, Xiangyu Zhang, Zilei Wang, Tieniu Tan, and Jian Sun. 2019. Meta-SR: A Magnification-Arbitrary Network for Super-Resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
[35]
Intel. 2023. Intel Advanced Vector Extensions 512. https://www.intel.com/content/www/us/en/architecture-and-technology/avx-512-overview.html.
[36]
Wenzel Jakob. 2010. Mitsuba renderer. http://www.mitsuba-renderer.org.
[37]
Wenzel Jakob, Eugene d'Eon, Otto Jakob, and Steve Marschner. 2014. A Comprehensive Framework for Rendering Layered Materials. ACM Trans. Graph. 33, 4, Article 118 (jul 2014), 14 pages.
[38]
Nima Khademi Kalantari and Ravi Ramamoorthi. 2017. Deep High Dynamic Range Imaging of Dynamic Scenes. ACM Transactions on Graphics (Proceedings of SIGGRAPH 2017) 36, 4 (2017).
[39]
Alexandr Kuznetsov, Krishna Mullia, Zexiang Xu, Miloš Hašan, and Ravi Ramamoorthi. 2021. NeuMIP: Multi-Resolution Neural Materials. Transactions on Graphics (Proceedings of SIGGRAPH) 40, 4, Article 175 (July 2021), 13 pages.
[40]
Alexandr Kuznetsov, Xuezheng Wang, Krishna Mullia, Fujun Luan, Zexiang Xu, Miloš Hašan, and Ravi Ramamoorthi. 2022. Rendering Neural Materials on Curved Surfaces. SIGGRAPH '22 Conference Proceedings (2022), 9 pages.
[41]
Iro Laina, Christian Rupprecht, Vasileios Belagiannis, Federico Tombari, and Nassir Navab. 2016. Deeper Depth Prediction with Fully Convolutional Residual Networks. In 2016 Fourth International Conference on 3D Vision (3DV). 239--248.
[42]
Xiao Li, Yue Dong, Pieter Peers, and Xin Tong. 2017. Modeling Surface Appearance from a Single Photograph Using Self-Augmented Convolutional Neural Networks. ACM Trans. Graph. 36, 4, Article 45 (July 2017), 11 pages.
[43]
Zhengqin Li, Kalyan Sunkavalli, and Manmohan Chandraker. 2018. Materials for Masses: SVBRDF Acquisition with a Single Mobile Phone Image. In Computer Vision - ECCV 2018. Springer International Publishing, Cham, 74--90.
[44]
Wojciech Matusik, Hanspeter Pfister, Matt Brand, and Leonard McMillan. 2003. A Data-Driven Reflectance Model. ACM Trans. Graph. 22, 3 (jul 2003), 759--769.
[45]
M. Maximov, T. Ritschel, L. Leal-Taixe, and M. Fritz. 2019. Deep Appearance Maps. In 2019 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE Computer Society, Los Alamitos, CA, USA, 8728--8737.
[46]
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. 2020. Nerf: Representing scenes as neural radiance fields for view synthesis. In European conference on computer vision. Springer, 405--421.
[47]
Thomas Müller, Fabrice Rousselle, Jan Novák, and Alexander Keller. 2021. Real-Time Neural Radiance Caching for Path Tracing. ACM Trans. Graph. 40, 4, Article 36 (jul 2021), 16 pages.
[48]
Jan Novák, Iliyan Georgiev, Johannes Hanika, Jaroslav Křivánek, and Wojciech Jarosz. 2018. Monte Carlo Methods for Physically Based Volume Rendering. In ACM SIGGRAPH 2018 Courses (Vancouver, British Columbia, Canada) (SIGGRAPH '18). Association for Computing Machinery, New York, NY, USA, Article 14, 1 pages.
[49]
Marios Papas, Krystle de Mesa, and Henrik Wann Jensen. 2014. A Physically-Based BSDF for Modeling the Appearance of Paper. Computer Graphics Forum 33, 4 (2014), 133--142.
[50]
Matt Pharr and Pat Hanrahan. 2000. Monte Carlo Evaluation of Non-Linear Scattering Equations for Subsurface Reflection. In Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '00). ACM Press/Addison-Wesley Publishing Co., USA, 75--84.
[51]
Nasim Rahaman, Aristide Baratin, Devansh Arpit, Felix Draxler, Min Lin, Fred Hamprecht, Yoshua Bengio, and Aaron Courville. 2019. On the Spectral Bias of Neural Networks. In Proceedings of the 36th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 97), Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.). PMLR, 5301--5310.
[52]
Gilles Rainer, Abhijeet Ghosh, Wenzel Jakob, and Tim Weyrich. 2020. Unified Neural Encoding of BTFs. Computer Graphics Forum 39, 2 (2020), 167--178.
[53]
Gilles Rainer, Wenzel Jakob, Abhijeet Ghosh, and Tim Weyrich. 2019. Neural BTF Compression and Interpolation. Computer Graphics Forum (Proceedings of Eurographics)
[54]
38, 2 (March 2019).
[55]
Joël Randrianandrasana, Patrick Callet, and Laurent Lucas. 2021. Transfer Matrix Based Layered Materials Rendering. ACM Trans. Graph. 40, 4, Article 177 (jul 2021), 16 pages.
[56]
Szymon M. Rusinkiewicz. 1998. A New Change of Variables for Efficient BRDF Representation. In Rendering Techniques '98, George Drettakis and Nelson Max (Eds.). Springer Vienna, Vienna, 11--22.
[57]
Jos Stam. 2001. An Illumination Model for a Skin Layer Bounded by Rough Surfaces. In Proceedings of the 12th Eurographics Conference on Rendering (London, UK) (EGWR'01). Eurographics Association, Goslar, DEU, 39--52.
[58]
Alejandro Sztrajman, Gilles Rainer, Tobias Ritschel, and Tim Weyrich. 2021. Neural BRDF Representation and Importance Sampling. In Computer Graphics Forum, Vol. 40. Wiley Online Library, 332--346.
[59]
Matthew Tancik, Pratul P. Srinivasan, Ben Mildenhall, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan T. Barron, and Ren Ng. 2020. Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains. CoRR abs/2006.10739 (2020). arXiv:2006.10739 https://arxiv.org/abs/2006.10739
[60]
Sebastian Thrun and Lorien Pratt. 1998. Learning to Learn: Introduction and Overview. Kluwer Academic Publishers, USA, 3--17.
[61]
Tanaboon Tongbuasirilai, Jonas Unger, Christine Guillemot, and Ehsan Miandji. 2022. A Sparse Non-Parametric BRDF Model. ACM Trans. Graph. (apr 2022). Just Accepted.
[62]
H.C. van de Hulst. 1980. Multiple Light Scattering. Academic Press.
[63]
Fu-En Wang, Yu-Hsuan Yeh, Min Sun, Wei-Chen Chiu, and Yi-Hsuan Tsai. 2020. Bi-Fuse: Monocular 360 Depth Estimation via Bi-Projection Fusion. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 459--468.
[64]
Andrea Weidlich and Alexander Wilkie. 2007. Arbitrarily Layered Micro-Facet Surfaces. In Proceedings of the 5th International Conference on Computer Graphics and Interactive Techniques in Australia and Southeast Asia (GRAPHITE '07). 171--178.
[65]
Andrea Weidlich and Alexander Wilkie. 2009. Exploring the Potential of Layered BRDF Models. In ACM SIGGRAPH ASIA 2009 Courses (SIGGRAPH ASIA '09). Association for Computing Machinery, New York, NY, USA, Article 7, 58 pages.
[66]
Andrea Weidlich and Alexander Wilkie. 2011. Thinking in Layers: Modeling with Layered Materials. In SIGGRAPH Asia 2011 Courses (Hong Kong, China) (SA '11). Association for Computing Machinery, New York, NY, USA, Article 20, 43 pages.
[67]
Philippe Weier and Laurent Belcour. 2020. Rendering Layered Materials with Anisotropic Interfaces. Journal of Computer Graphics Techniques (JCGT) 9, 2 (20 June 2020), 37--57.
[68]
Mengqi (Mandy) Xia, Bruce Walter, Christophe Hery, and Steve Marschner. 2020. Gaussian Product Sampling for Rendering Layered Materials. Computer Graphics Forum 39, 1 (2020), 420--435.
[69]
Zilin Xu, Zheng Zeng, Lifan Wu, Lu Wang, and Ling-Qi Yan. 2022. Lightweight Neural Basis Functions for All-Frequency Shading. In SIGGRAPH Asia 2022 Conference Papers (SA '22). Association for Computing Machinery, Article 14, 9 pages.
[70]
Tomoya Yamaguchi, Tatsuya Yatagawa, Yusuke Tokuyoshi, and Shigeo Morishima. 2019. Real-Time Rendering of Layered Materials with Anisotropic Normal Distributions. In SIGGRAPH Asia 2019 Technical Briefs (Brisbane, QLD, Australia) (SA '19). Association for Computing Machinery, New York, NY, USA, 87--90.
[71]
Tizian Zeltner and Wenzel Jakob. 2018. The Layer Laboratory: A Calculus for Additive and Subtractive Composition of Anisotropic Surface Reflectance. ACM Trans. Graph. 37, 4, Article 74 (jul 2018), 14 pages.
[72]
Chuankun Zheng, Ruzhang Zheng, Rui Wang, Shuang Zhao, and Hujun Bao. 2022. A Compact Representation of Measured BRDFs Using Neural Processes. ACM Trans. Graph. 41, 2, Article 14 (nov 2022), 15 pages.

Cited By

View all
  • (2024)BSDF importance sampling using a diffusion modelSIGGRAPH Asia 2024 Conference Papers10.1145/3680528.3687684(1-11)Online publication date: 3-Dec-2024
  • (2024)Reconstructing translucent thin objects from photosSIGGRAPH Asia 2024 Conference Papers10.1145/3680528.3687572(1-11)Online publication date: 3-Dec-2024
  • (2024)Cross-reconstructed Augmentation for Dual-target Cross-domain RecommendationProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657902(2352-2356)Online publication date: 10-Jul-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics
ACM Transactions on Graphics  Volume 42, Issue 6
December 2023
1565 pages
ISSN:0730-0301
EISSN:1557-7368
DOI:10.1145/3632123
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 December 2023
Published in TOG Volume 42, Issue 6

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. importance sampling
  2. layered material
  3. meta learning
  4. neural networks

Qualifiers

  • Research-article

Funding Sources

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)173
  • Downloads (Last 6 weeks)24
Reflects downloads up to 03 Mar 2025

Other Metrics

Citations

Cited By

View all
  • (2024)BSDF importance sampling using a diffusion modelSIGGRAPH Asia 2024 Conference Papers10.1145/3680528.3687684(1-11)Online publication date: 3-Dec-2024
  • (2024)Reconstructing translucent thin objects from photosSIGGRAPH Asia 2024 Conference Papers10.1145/3680528.3687572(1-11)Online publication date: 3-Dec-2024
  • (2024)Cross-reconstructed Augmentation for Dual-target Cross-domain RecommendationProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657902(2352-2356)Online publication date: 10-Jul-2024
  • (2024)Practical Appearance Model for Foundation CosmeticsComputer Graphics Forum10.1111/cgf.1514843:4Online publication date: 24-Jul-2024

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media