skip to main content
research-article

Efficient reflectance capture using an autoencoder

Published: 30 July 2018 Publication History

Abstract

We propose a novel framework that automatically learns the lighting patterns for efficient reflectance acquisition, as well as how to faithfully reconstruct spatially varying anisotropic BRDFs and local frames from measurements under such patterns. The core of our framework is an asymmetric deep autoencoder, consisting of a nonnegative, linear encoder which directly corresponds to the lighting patterns used in physical acquisition, and a stacked, nonlinear decoder which computationally recovers the BRDF information from captured photographs. The autoencoder is trained with a large amount of synthetic reflectance data, and can adapt to various factors, including the geometry of the setup and the properties of appearance. We demonstrate the effectiveness of our framework on a wide range of physical materials, using as few as 16 ~ 32 lighting patterns, which correspond to 12 ~ 25 seconds of acquisition time. We also validate our results with the ground truth data and captured photographs. Our framework is useful for increasing the efficiency in both novel and existing acquisition setups.

Supplementary Material

ZIP File (127-118.zip)
Supplemental files.
MP4 File (127-118.mp4)
MP4 File (a127-kang.mp4)

References

[1]
Miika Aittala, Timo Aila, and Jaakko Lehtinen. 2016. Reflectance Modeling by Neural Texture Synthesis. ACM Trans. Graph. 35, 4, Article 65 (July 2016), 13 pages.
[2]
Miika Aittala, Tim Weyrich, and Jaakko Lehtinen. 2013. Practical SVBRDF Capture in the Frequency Domain. ACM Trans. Graph. 32, 4, Article 110 (July 2013), 12 pages.
[3]
Miika Aittala, Tim Weyrich, and Jaakko Lehtinen. 2015. Two-shot SVBRDF Capture for Stationary Materials. ACM Trans. Graph. 34, 4, Article 110 (July 2015), 13 pages.
[4]
Guojun Chen, Yue Dong, Pieter Peers, Jiawan Zhang, and Xin Tong. 2014. Reflectance Scanning: Estimating Shading Frame and BRDF with Generalized Linear Light Sources. ACM Trans. Graph. 33, 4, Article 117 (July 2014), 11 pages.
[5]
Kristin J. Dana, Bram van Ginneken, Shree K. Nayar, and Jan J. Koenderink. 1999. Reflectance and Texture of Real-world Surfaces. ACM Trans. Graph. 18, 1 (Jan. 1999), 1--34.
[6]
Yue Dong, Jiaping Wang, Xin Tong, John Snyder, Yanxiang Lan, Moshe Ben-Ezra, and Baining Guo. 2010. Manifold Bootstrapping for SVBRDF Capture. ACM Trans. Graph. 29, 4, Article 98 (July 2010), 10 pages.
[7]
Andrew Gardner, Chris Tchou, Tim Hawkins, and Paul Debevec. 2003. Linear light source reflectomerry. ACM Trans. Graph. 22, 3 (2003), 749--758.
[8]
Abhijeet Ghosh, Tongbo Chen, Pieter Peers, Cyrus A. Wilson, and Paul Debevec. 2009. Estimating Specular Roughness and Anisotropy from Second Order Spherical Gradient Illumination. Computer Graphics Forum 28, 4 (2009), 1161--1170.
[9]
Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. MIT Press. http://www.deeplearningbook.org.
[10]
G. E. Hinton and R. R. Salakhutdinov. 2006. Reducing the Dimensionality of Data with Neural Networks. Science 313, 5786 (2006), 504--507.
[11]
Eric P. F. Lafortune, Sing-Choong Foo, Kenneth E. Torrance, and Donald P. Greenberg. 1997. Non-linear Approximation of Reflectance Functions. In Proc. SIGGRAPH. 117--126.
[12]
Jason Lawrence, Aner Ben-Artzi, Christopher DeCoro, Wojciech Matusik, Hanspeter Pfister, Ravi Ramamoorthi, and Szymon Rusinkiewicz. 2006. Inverse Shade Trees for Non-parametric Material Representation and Editing. ACM Trans. Graph. 25, 3 (July 2006), 735--745.
[13]
Hendrik P. A. Lensch, Jan Kautz, Michael Goesele, Wolfgang Heidrich, and Hans-Peter Seidel. 2003. Image-based Reconstruction of Spatial Appearance and Geometric Detail. ACM Trans. Graph. 22, 2 (April 2003), 234--257.
[14]
Xiao Li, Yue Dong, Pieter Peers, and Xin Tong. 2017. Modeling Surface Appearance from a Single Photograph Using Self-augmented Convolutional Neural Networks. ACM Trans. Graph. 36, 4, Article 45 (July 2017), 11 pages.
[15]
Stephen R. Marschner, Stephen H. Westin, Eric P. F. Lafortune, Kenneth E. Torrance, and Donald P. Greenberg. 1999. Image-based BRDF Measurement Including Human Skin. In Proc. EGWR. 131--144.
[16]
Wojciech Matusik, Hanspeter Pfister, Matt Brand, and Leonard McMillan. 2003a. A Data-driven Reflectance Model. ACM Trans. Graph. 22, 3 (July 2003), 759--769.
[17]
Wojciech Matusik, Hanspeter Pfister, Matthew Brand, and Leonard McMillan. 2003b. Efficient Isotropic BRDF Measurement. In Proc. EGWR. 241--247.
[18]
Stephen McAuley, Stephen Hill, Naty Hoffman, Yoshiharu Gotanda, Brian Smits, Brent Burley, and Adam Martinez. 2012. Practical Physically-based Shading in Film and Game Production. In ACM SIGGRAPH 2012 Courses. Article 10, 7 pages.
[19]
Giljoo Nam, Joo Ho Lee, Hongzhi Wu, Diego Gutierrez, and Min H. Kim. 2016. Simultaneous Acquisition of Microscale Reflectance and Normals. ACM Trans. Graph. 35, 6, Article 185 (Nov. 2016), 11 pages.
[20]
Jannik Boll Nielsen, Henrik Wann Jensen, and Ravi Ramamoorthi. 2015. On Optimal, Minimal BRDF Sampling for Reflectance Acquisition. ACM Trans. Graph. 34, 6, Article 186 (Oct. 2015), 11 pages.
[21]
Pieter Peers, Dhruv K. Mahajan, Bruce Lamond, Abhijeet Ghosh, Wojciech Matusik, Ravi Ramamoorthi, and Paul Debevec. 2009. Compressive Light Transport Sensing. ACM Trans. Graph. 28, 1, Article 3 (Feb. 2009), 18 pages.
[22]
Peiran Ren, Jiaping Wang, John Snyder, Xin Tong, and Baining Guo. 2011. Pocket reflectometry. ACM Trans. Graph. 30, 4 (2011), 1--10.
[23]
Tijmen Tieleman and Geoffrey Hinton. 2012. Lecture 6.5-RMSProp: Divide the Gradient by a Running Average of Its Recent Magnitude. Neural Networks for Machine Learning 4 (2012), 26--31.
[24]
Borom Tunwattanapong, Graham Fyffe, Paul Graham, Jay Busch, Xueming Yu, Abhijeet Ghosh, and Paul Debevec. 2013. Acquiring Reflectance and Shape from Continuous Spherical Harmonic Illumination. ACM Trans. Graph. 32, 4, Article 109 (July 2013), 12 pages.
[25]
Bruce Walter, Stephen R. Marschner, Hongsong Li, and Kenneth E. Torrance. 2007. Microfacet Models for Refraction through Rough Surfaces. In Rendering Techniques (Proc. EGWR).
[26]
Jiaping Wang, Shuang Zhao, Xin Tong, John Snyder, and Baining Guo. 2008. Modeling Anisotropic Surface Reflectance with Example-based Microfacet Synthesis. ACM Trans. Graph. 27, 3, Article 41 (Aug. 2008), 9 pages.
[27]
Michael Weinmann and Reinhard Klein. 2015. Advances in Geometry and Reflectance Acquisition. In SIGGRAPH Asia Courses. Article 1, 71 pages.
[28]
Tim Weyrich, Jason Lawrence, Hendrik P. A. Lensch, Szymon Rusinkiewicz, and Todd Zickler. 2009. Principles of Appearance Acquisition and Representation. Found. Trends. Comput. Graph. Vis. 4, 2 (2009), 75--191.
[29]
Zexiang Xu, Jannik Boll Nielsen, Jiyang Yu, Henrik Wann Jensen, and Ravi Ramamoorthi. 2016. Minimal BRDF Sampling for Two-shot Near-field Reflectance Acquisition. ACM Trans. Graph. 35, 6, Article 188 (Nov. 2016), 12 pages.
[30]
Todd Zickler, Sebastian Enrique, Ravi Ramamoorthi, and Peter Belhumeur. 2005. Reflectance Sharing: Image-based Rendering from a Sparse Set of Images. In Proc. EGSR. 253--264.

Cited By

View all
  • (2024)NFPLight: Deep SVBRDF Estimation via the Combination of Near and Far Field Point LightingACM Transactions on Graphics10.1145/368797843:6(1-11)Online publication date: 19-Dec-2024
  • (2024)Deep SVBRDF Acquisition and Modelling: A SurveyComputer Graphics Forum10.1111/cgf.1519943:6Online publication date: 16-Sep-2024
  • (2024)FROST-BRDF: A Fast and Robust Optimal Sampling Technique for BRDF AcquisitionIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.335520030:7(4390-4402)Online publication date: Jul-2024
  • Show More Cited By

Index Terms

  1. Efficient reflectance capture using an autoencoder

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Transactions on Graphics
    ACM Transactions on Graphics  Volume 37, Issue 4
    August 2018
    1670 pages
    ISSN:0730-0301
    EISSN:1557-7368
    DOI:10.1145/3197517
    Issue’s Table of Contents
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 30 July 2018
    Published in TOG Volume 37, Issue 4

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. SV-BRDF
    2. lighting patterns
    3. optimal sampling
    4. reflectance acquisition

    Qualifiers

    • Research-article

    Funding Sources

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)30
    • Downloads (Last 6 weeks)1
    Reflects downloads up to 15 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)NFPLight: Deep SVBRDF Estimation via the Combination of Near and Far Field Point LightingACM Transactions on Graphics10.1145/368797843:6(1-11)Online publication date: 19-Dec-2024
    • (2024)Deep SVBRDF Acquisition and Modelling: A SurveyComputer Graphics Forum10.1111/cgf.1519943:6Online publication date: 16-Sep-2024
    • (2024)FROST-BRDF: A Fast and Robust Optimal Sampling Technique for BRDF AcquisitionIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.335520030:7(4390-4402)Online publication date: Jul-2024
    • (2024)Efficient Reflectance Capture With a Deep Gated Mixture-of-ExpertsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.326187230:7(4246-4256)Online publication date: Jul-2024
    • (2024)High-Fidelity Specular SVBRDF Acquisition From Flash PhotographsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.323527730:4(1885-1896)Online publication date: Apr-2024
    • (2024)Differentiable Display Photometric Stereo2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.01124(11831-11840)Online publication date: 16-Jun-2024
    • (2024)MatTrans: Material Reflectance Property Estimation of Complex Objects with TransformerComputational Visual Media10.1007/978-981-97-2095-8_11(197-217)Online publication date: 10-Apr-2024
    • (2024)Hypernetworks for Generalizable BRDF RepresentationComputer Vision – ECCV 202410.1007/978-3-031-73116-7_5(73-89)Online publication date: 29-Sep-2024
    • (2023)OpenSVBRDF: A Database of Measured Spatially-Varying ReflectanceACM Transactions on Graphics10.1145/361835842:6(1-14)Online publication date: 5-Dec-2023
    • (2023)Ultra-High Resolution SVBRDF Recovery from a Single ImageACM Transactions on Graphics10.1145/359379842:3(1-14)Online publication date: 5-Jun-2023
    • Show More Cited By

    View Options

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media