skip to main content
article

TensorTextures: multilinear image-based rendering

Published: 01 August 2004 Publication History

Abstract

This paper introduces a tensor framework for image-based rendering. In particular, we develop an algorithm called TensorTextures that learns a parsimonious model of the bidirectional texture function (BTF) from observational data. Given an ensemble of images of a textured surface, our nonlinear, generative model explicitly represents the multifactor interaction implicit in the detailed appearance of the surface under varying photometric angles, including local (per-texel) reflectance, complex mesostructural self-occlusion, interreflection and self-shadowing, and other BTF-relevant phenomena. Mathematically, TensorTextures is based on multilinear algebra, the algebra of higher-order tensors, hence its name. It is computed through a decomposition known as the N-mode SVD, an extension to tensors of the conventional matrix singular value decomposition (SVD). We demonstrate the application of TensorTextures to the image-based rendering of natural and synthetic textured surfaces under continuously varying viewpoint and illumination conditions.

References

[1]
DANA, K. J., VAN GINNEKEN, B., NAYAR, S. K., AND KOENDERINK, J. J. 1999. Reflectance and texture of real-world surfaces. ACM Trans. Graphics 18, 1, 1--34.
[2]
FURUKAWA, R., KAWASAKI, H., IKEUCHI, K., AND SAKAUCHI, M. 2002. Appearance based object modelling using texture database: Acquisition, compression and rendering. In 8th Eurographics Workshop on Virtual Environments, 257--266.
[3]
GORTLER, S. J., GRZESZCZUK, R., SZELISKI, R., AND COHEN, M. F. 1996. The lumigraph. Computer Graphics (Proc. SIGGRAPH) 30, 43--45.
[4]
HAN, J. Y. AND PERLIN, K. 2003. Measuring bidirectional texture reflectance with a kaleidoscope. ACM Trans. Graphics (Proc. SIGGRAPH) 22, 741--748.
[5]
KOENDERINK, J. J., AND VAN DOORN, A. J. 1996. Illuminance texture due to surface mesostructure. Journal of the Optical Society of America 13, 3, 452--463.
[6]
KOLDA, T. G. 2001. Orthogonal tensor decompositions. SIAM J. Matrix Analysis and Applications 23, 1, 243--255.
[7]
KOUDELKA, M. L., MAGDA, S., BELHUMEUR, P. N., AND KRIEGMAN, D. J. 2003. Acquisition, compression, and synthesis of bidirectional texture functions. In Proc. 3rd Int. Workshop on Texture Analysis and Synthesis (Oct), Nice, France, 59--64.
[8]
KROONENBERG, P., ANDDE LEEUW, J. 1980. Principal component analysis of three-mode data by means of alternating least squares algorithms. Psychometrika 45, 69--97.
[9]
LATHAUWER, L. DE, MOOR, B. DE, AND VANDEWALLE, J. 2000. On the best rank-1 and rank-(R1, R2, …, Rn) approximation of higher-order tensors. SIAM J. Matrix Analysis and Applications 21, 4, 1324--1342.
[10]
LEVOY, M., AND HANRAHAN, R. 1996. Light field rendering. Computer Graphics (Proc. SIGGRAPH) 30, 31--42.
[11]
LIU, X., YU, Y., AND SHUM, H.-Y. 2001. Synthesizing bidirectional texture functions for real-world surfaces. ACM Trans. Graphics (Proc. SIGGRAPH) 20, 97--106.
[12]
MALZBENDER, T., GELB, D., AND WOLTERS, H. 2001. Polynomial texture maps. Computer Graphics (SIGGRAPH 01) (Aug), 519--528.
[13]
MATUSIK, W., PFISTER, H., BRAND, M., AND MCMILLAN, L. 2003. A data driven reflectance model. Computer Graphics (SIGGRAPH 03) 22, 3 (July), 759--769.
[14]
MESETH, J., MULLER, G., SATTLER, M., AND KLEIN, R. 2003. BTF rendering for virtual environments. In Proc. Virtual Concepts 2003 (Nov), 356--363.
[15]
SATTLER, M., SARLETTE, R., AND KLEIN, R. 2003. Efficient and realistic visualization of cloth. In Proc. Eurographics Symposium on Rendering, 167--177.
[16]
SUYKENS, F., BERGE, K. VON, LAGAE, A., AND KLEIN, R. 2003. Interactive rendering with bidirectional texture functions. Comp. Graphics Forum (Proc. Eurographics) 22, 3, 463--472.
[17]
TEO, P. AND HEEGER, P. 1994. Perceptual image distortion. In Proc. IEEE Conf. Image Processing (Nov), 982--986.
[18]
TONG, X., ZHANG, J., LIU, L., WANG, X., GUO, B., AND SHUM, H.-Y. 2002. Synthesis of bidirectional texture functions on arbitrary surfaces. ACM Trans. Graphics (Proc. SIGGRAPH) 21, 3, 665--672.
[19]
TUCKER, L. 1966. Some mathematical notes on three-mode factor analysis. Psychometrika 31, 279--311.
[20]
VASILESCU, M. A. O., AND TERZOPOULOS, D. 2002. Multilinear analysis of image ensembles: TensorFaces. In Proc. European Conf. Computer Vision, 447--460.
[21]
WANG, L., WANG, X., TONG, X., LIN, S., HU, S., GUO, B., AND SHUM, H. Y. 2003. View displacement mapping. ACM Trans. Graphics (Proc. SIGGRAPH) 22, 3, 334--339.

Cited By

View all

Index Terms

  1. TensorTextures: multilinear image-based rendering

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Transactions on Graphics
      ACM Transactions on Graphics  Volume 23, Issue 3
      August 2004
      684 pages
      ISSN:0730-0301
      EISSN:1557-7368
      DOI:10.1145/1015706
      Issue’s Table of Contents
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 01 August 2004
      Published in TOG Volume 23, Issue 3

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. Bidirectional Texture Function
      2. Image-Based Rendering
      3. Multilinear Algebra
      4. Statistical Learning
      5. Tensor Decomposition
      6. Tensors
      7. Textured Surface Rendering

      Qualifiers

      • Article

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)5
      • Downloads (Last 6 weeks)0
      Reflects downloads up to 17 Jan 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)HMK-CTA: A Hierarchical Multidimensional Representation for Visual DatasetsProceedings of the 50th Graphics Interface Conference10.1145/3670947.3670954(1-10)Online publication date: 3-Jun-2024
      • (2024)Causal Deep LearningPattern Recognition10.1007/978-3-031-78189-6_27(420-438)Online publication date: 11-Dec-2024
      • (2023)Group Sparse Reduced Rank Tensor Regression for Micro-Expression RecognitionIEICE Transactions on Information and Systems10.1587/transinf.2022EDL8073E106.D:4(575-578)Online publication date: 1-Apr-2023
      • (2022)Compact Poisson Filters for Fast Fluid SimulationACM SIGGRAPH 2022 Conference Proceedings10.1145/3528233.3530737(1-9)Online publication date: 27-Jul-2022
      • (2022)Multivariant spatial and geometric interpolation of field maps through model order reductionPhysical Review Accelerators and Beams10.1103/PhysRevAccelBeams.25.12460125:12Online publication date: 7-Dec-2022
      • (2022)Predefined-time convergent neural networks for solving the time-varying nonsingular multi-linear tensor equationsNeurocomputing10.1016/j.neucom.2021.11.108472(68-84)Online publication date: Feb-2022
      • (2022)Randomized algorithms for the computation of multilinear rank- approximationsJournal of Global Optimization10.1007/s10898-022-01182-887:2-4(373-403)Online publication date: 26-May-2022
      • (2022)TensoRF: Tensorial Radiance FieldsComputer Vision – ECCV 202210.1007/978-3-031-19824-3_20(333-350)Online publication date: 11-Nov-2022
      • (2021)Tensor Representations for Action RecognitionIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2021.3107160(1-1)Online publication date: 2021
      • (2021)CausalX: Causal eXplanations and Block Multilinear Factor Analysis2020 25th International Conference on Pattern Recognition (ICPR)10.1109/ICPR48806.2021.9412780(10736-10743)Online publication date: 10-Jan-2021
      • Show More Cited By

      View Options

      Login options

      Full Access

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media