research-article

Kernel Nyström method for light transport

Authors:

Baining GuoAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 28, Issue 3

Article No.: 29, Pages 1 - 10

https://doi.org/10.1145/1531326.1531335

Published: 27 July 2009 Publication History

Abstract

We propose a kernel Nyström method for reconstructing the light transport matrix from a relatively small number of acquired images. Our work is based on the generalized Nyström method for low rank matrices. We introduce the light transport kernel and incorporate it into the Nyström method to exploit the nonlinear coherence of the light transport matrix. We also develop an adaptive scheme for efficiently capturing the sparsely sampled images from the scene. Our experiments indicate that the kernel Nyström method can achieve good reconstruction of the light transport matrix with a few hundred images and produce high quality relighting results. The kernel Nyström method is effective for modeling scenes with complex lighting effects and occlusions which have been challenging for existing techniques.

Supplementary Material

JPG File (tps042_09.jpg)

Download
15.05 KB

MP4 File (tps042_09.mp4)

Download
48.97 MB

References

[1]

An, X., and Pellacini, F. 2008. Appprop: All-pairs appearance-space edit propagation. ACM Transactions on Graphics 27, 3 (Aug.), 40:1--40:9.

Digital Library

[2]

Chuang, Y.-Y., Zongker, D. E., Hindorff, J., Curless, B., Salesin, D. H., and Szeliski, R. 2000. Environment matting extensions: Towards higher accuracy and real-time capture. In Proceedings of ACM SIGGRAPH 2000, Computer Graphics Proceedings, Annual Conference Series, 121--130.

Digital Library

[3]

Cristianini, N., and Shawe-Taylor, J. 2000. An introduction to support vector machines and other kernel-based learning methods. Cambridge University Press.

Digital Library

[4]

Debevec, P. E., and Malik, J. 1997. Recovering high dynamic range radiance maps from photographs. In Proceedings of SIGGRAPH 97, Computer Graphics Proceedings, Annual Conference Series, 369--378.

Digital Library

[5]

Debevec, P., Hawkins, T., Tchou, C., Duiker, H.-P., Sarokin, W., and Sagar, M. 2000. Acquiring the reflectance field of a human face. In Proceedings of ACM SIGGRAPH 2000, Computer Graphics Proceedings, Annual Conference Series, 145--156.

Digital Library

[6]

Fazel, M. 2002. Matrix rank minimization with applications. PhD thesis, Stanford University.

[7]

Fuchs, M., Blanz, V., Lensch, H. P. A., and Seidel, H.-P. 2007. Adaptive sampling of reflectance fields. ACM Transactions on Graphics 26, 2 (June), 10:1--10:18.

Digital Library

[8]

Garg, G., Talvala, E.-V., Levoy, M., and Lensch, H. P. A. 2006. Symmetric photography: Exploiting data-sparseness in reflectance fields. In Rendering Techniques 2006: 17th Eurographics Workshop on Rendering, 251--262.

Digital Library

[9]

Goesele, M., Lensch, H. P. A., Lang, J., Fuchs, C., and Seidel, H.-P. 2004. Disco: acquisition of translucent objects. ACM Transactions on Graphics 23, 3 (Aug.), 835--844.

Digital Library

[10]

Goreinov, S., Tyrtyshnikov, E. E., and Zamarashkin, N. L. 1997. A theory of pseudo-skeleton approximations. Linear Algeabra and Applications 261, 1--21.

[11]

Ha&#353;an, M., Pellacini, F., and Bala, K. 2007. Matrix rowcolumn sampling for the many-light problem. ACM Transactions on Graphics 26, 3 (July), 26:1--26:10.

Digital Library

[12]

Ha&#353;an, M., Velazquez-Armendariz, E., Pellacini, F., and Bala, K. 2008. Tensor clustering for rendering manylight animations. Computer Graphics Forum (Proc. Eurographics Rendering 2008) 27, 4, 1105--1114.

Digital Library

[13]

Hawkins, T., Einarsson, P., and Debevec, P. 2005. A dual light stage. In Rendering Techniques 2005: 16th Eurographics Workshop on Rendering, 91--98.

Digital Library

[14]

Levoy, M., and Hanrahan, P. M. 1996. Light field rendering. In Proceedings of SIGGRAPH 96, Computer Graphics Proceedings, Annual Conference Series, 31--42.

Digital Library

[15]

Lin, Z., Wong, T.-T., and Shum, H.-Y. Relighting with the reflected irradiance field: Representation, sampling and reconstruction. International Journal of Computer Vision 49, 2.

Digital Library

[16]

Mahajan, D., Shlizerman, I. K., Ramamoorthi, R., and Belhumeur, P. 2007. A theory of locally low dimensional light transport. ACM Transactions on Graphics 26, 3 (July), 62:1--62:10.

Digital Library

[17]

Masselus, V., Peers, P., Dutr&#233;, P., and Willems, Y. D. 2003. Relighting with 4d incident light fields. ACM Transactions on Graphics 22, 3 (July), 613--620.

Digital Library

[18]

Masselus, V., Peers, P., Dutr0108, P., and Willems, Y. D. 2004. Smooth reconstruction and compact representation of reflectance functions for image-based relighting. In Rendering Techniques 2004: 15th Eurographics Workshop on Rendering, 287--298.

Digital Library

[19]

Matusik, W., Pfister, H., Ngan, A., Beardsley, P., Ziegler, R., and McMillan, L. 2002. Image-based 3D photography using opacity hulls. ACM Transactions on Graphics 21, 3 (July), 427--437.

Digital Library

[20]

Matusik, W., Loper, M., and Pfister, H. 2004. Progressively-refined reflectance functions from natural illumination. In Rendering Techniques 2004: 15th Eurographics Workshop on Rendering, 299--308.

Digital Library

[21]

Ng, R., Ramamoorthi, R., and Hanrahan, P. 2003. Allfrequency shadows using non-linear wavelet lighting approximation. ACM Transactions on Graphics 22, 3 (July), 376--381.

Digital Library

[22]

Peers, P., and Dutr&#233;, P. 2005. Inferring reflectance functions from wavelet noise. In Rendering Techniques 2005: 16th Eurographics Workshop on Rendering, 173--182.

Digital Library

[23]

Peers, P., Vom Berge, K., Matusik, W., Ramamoorthi, R., Lawrence, J., Rusinkiewicz, S., and Dutr&#233;, P. 2006. A compact factored representation of heterogeneous subsurface scattering. ACM Transactions on Graphics 25, 3 (July), 746--753.

Digital Library

[24]

Peers, P., Mahajan, D. K., Lamond, B., Ghosh, A., Matusik, W., Ramamoorthi, R., and Debevec, P. 2009. Compressive light transport sensing. ACM Transactions on Graphics 28, 1 (Jan.), 3:1--3:18.

Digital Library

[25]

Platt, J. C. 2005. Fastmap, metricmap, and landmark mds are all nystr&#246;m algorithms. In 10th International Workshop on Artificial Intelligence and Statistics, 261--268.

[26]

Press, W. H., et al. 1992. Numerical recipes in C (second edition). Cambridge University Press.

Digital Library

[27]

Sen, P., and Darabi, S. 2009. Compressive Dual Photography. Computer Graphics Forum 28, 2, 609--618.

[28]

Sen, P., Chen, B., Garg, G., Marschner, S. R., Horowitz, M., Levoy, M., and Lensch, H. P. A. 2005. Dual photography. ACM Transactions on Graphics 24, 3, 745--755.

Digital Library

[29]

Wenger, A., Gardner, A., Tchou, C., Unger, J., Hawkins, T., and Debevec, P. 2005. Performance relighting and reflectance transformation with time-multiplexed illumination. ACM Transactions on Graphics 24, 3 (Aug.), 756--764.

Digital Library

[30]

Williams, C., and Seeger, M. 2000. Using the nystr&#246;m method to speed up kernel machines. Advances in Neural Information Processing Systems 13, 682--688.

[31]

Zongker, D. E., Werner, D. M., Curless, B., and Salesin, D. H. 1999. Environment matting and compositing. In Proceedings of SIGGRAPH 99, Computer Graphics Proceedings, Annual Conference Series, 205--214.

Digital Library

Cited By

Yasui MIwataki RIshikawa MWatanabe Y(2024)Projection Mapping with a Brightly Lit Surrounding Using a Mixed Light Field ApproachIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.337213230:5(2217-2227)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1109/TVCG.2024.3372132
Tetzlaff M(2024)High-Fidelity Specular SVBRDF Acquisition From Flash PhotographsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.323527730:4(1885-1896)Online publication date: Apr-2024
https://doi.org/10.1109/TVCG.2023.3235277
Zhu ZLi ZZhang RGuo CCheng M(2022)Designing an Illumination-Aware Network for Deep Image RelightingIEEE Transactions on Image Processing10.1109/TIP.2022.319536631(5396-5411)Online publication date: 2022
https://doi.org/10.1109/TIP.2022.3195366
Show More Cited By

Index Terms

Kernel Nyström method for light transport
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Reconstruction
      2. Image and video acquisition
  2. Modeling and simulation
    1. Model development and analysis
      1. Modeling methodologies
2. Mathematics of computing
  1. Mathematical analysis
    1. Numerical analysis
      1. Computations on matrices

Recommendations

Kernel Nyström method for light transport
SIGGRAPH '09: ACM SIGGRAPH 2009 papers

We propose a kernel Nyström method for reconstructing the light transport matrix from a relatively small number of acquired images. Our work is based on the generalized Nyström method for low rank matrices. We introduce the light transport kernel and ...
Modeling spatially-varying reflectance based on Kernel Nyström
VRST '10: Proceedings of the 17th ACM Symposium on Virtual Reality Software and Technology

We present a new method for modeling real-world surface reflectance, described with non-parametric spatially-varying bidirectional reflectance distribution functions(SVBRDF). Our method seeks to achieve high reconstruction accuracy, compactness and "...
Pencil light transport

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 28, Issue 3

August 2009

750 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/1531326

Issue’s Table of Contents

Copyright © 2009 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 July 2009

Published in TOG Volume 28, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

88
Total Citations
View Citations
1,208
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)2

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Yasui MIwataki RIshikawa MWatanabe Y(2024)Projection Mapping with a Brightly Lit Surrounding Using a Mixed Light Field ApproachIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.337213230:5(2217-2227)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1109/TVCG.2024.3372132
Tetzlaff M(2024)High-Fidelity Specular SVBRDF Acquisition From Flash PhotographsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.323527730:4(1885-1896)Online publication date: Apr-2024
https://doi.org/10.1109/TVCG.2023.3235277
Zhu ZLi ZZhang RGuo CCheng M(2022)Designing an Illumination-Aware Network for Deep Image RelightingIEEE Transactions on Image Processing10.1109/TIP.2022.319536631(5396-5411)Online publication date: 2022
https://doi.org/10.1109/TIP.2022.3195366
Chandran SKubo HUeda TFunatomi TMukaigawa YJayasuriya S(2022)Slope Disparity Gating: System and ApplicationsIEEE Transactions on Computational Imaging10.1109/TCI.2022.31622598(317-332)Online publication date: 2022
https://doi.org/10.1109/TCI.2022.3162259
Chen AXu ZGeiger AYu JSu H(2022)TensoRF: Tensorial Radiance FieldsComputer Vision – ECCV 202210.1007/978-3-031-19824-3_20(333-350)Online publication date: 11-Nov-2022
https://doi.org/10.1007/978-3-031-19824-3_20
Halmaoui HHaqiq A(2022)Computer Graphics Rendering Survey: From Rasterization and Ray Tracing to Deep LearningInnovations in Bio-Inspired Computing and Applications10.1007/978-3-030-96299-9_51(537-548)Online publication date: 22-Feb-2022
https://doi.org/10.1007/978-3-030-96299-9_51
Liu XHenderson KRego JJayasuriya SKoppal S(2021)Dense Lissajous sampling and interpolation for dynamic light-transportOptics Express10.1364/OE.42506129:12(18362)Online publication date: 27-May-2021
https://doi.org/10.1364/OE.425061
Marco JJarabo ANam JLiu XCosculluela MVelten AGutierrez D(2021)Virtual light transport matrices for non-line-of-sight imaging2021 IEEE/CVF International Conference on Computer Vision (ICCV)10.1109/ICCV48922.2021.00244(2420-2429)Online publication date: Oct-2021
https://doi.org/10.1109/ICCV48922.2021.00244
Henderson KLiu XFolden JTilmon BJayasuriya SKoppal S(2020)Design and Calibration of a Fast Flying-Dot Projector for Dynamic Light Transport AcquisitionIEEE Transactions on Computational Imaging10.1109/TCI.2020.29642466(529-543)Online publication date: 2020
https://doi.org/10.1109/TCI.2020.2964246
Qiu DZeng JKe ZSun WYang C(2020)Towards Geometry Guided Neural Relighting with Flash Photography2020 International Conference on 3D Vision (3DV)10.1109/3DV50981.2020.00124(1137-1146)Online publication date: Nov-2020
https://doi.org/10.1109/3DV50981.2020.00124
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents