research-article

The shading probe: fast appearance acquisition for mobile AR

Authors:

Kenny Mitchell,

Derek Nowrouzezahrai,

Jan KautzAuthors Info & Claims

SA '13: SIGGRAPH Asia 2013 Technical Briefs

Article No.: 20, Pages 1 - 4

https://doi.org/10.1145/2542355.2542380

Published: 19 November 2013 Publication History

Abstract

The ubiquity of mobile devices with powerful processors and integrated video cameras is re-opening the discussion on practical augmented reality (AR). Despite this technological convergence, several issues prevent reliable and immersive AR on these platforms. We address one such problem, the shading of virtual objects and determination of lighting that remains consistent with the surrounding environment. We design a novel light probe and exploit its structure to permit an efficient reformulation of the rendering equation that is suitable for fast shading on mobile devices. Unlike prior approaches, our shading probe directly captures the shading, and not the incident light, in a scene. As such, we avoid costly and unreliable radiometric calibration as well as side-stepping the need for complex shading algorithms. Moreover, we can tailor the shading probe's structure to better handle common lighting scenarios, such as outdoor settings. We achieve high-performance shading of virtual objects in an AR context, incorporating plausible local global-illumination effects, on mobile platforms.

References

[1]

Debevec, P., and Malik, J. 2008. Recovering high dynamic range radiance maps from photographs. In ACM SIGGRAPH 2008 Courses, ACM, 31.

Digital Library

[2]

Debevec, P., Graham, P., Busch, J., and Bolas, M. 2012. A single-shot light probe. In SIGGRAPH Talks.

Digital Library

[3]

Debevec, P. 2002. Image-based lighting. IEEE Comput. Graph. Appl. 22, 2 (Mar.), 26--34.

Digital Library

[4]

Diverdi, S., Withert, J., and Hllerert, T. 2008. Envisor: Online environment map construction for mixed reality. In Proc. IEEE VR 2008.

[5]

Fournier, A., Gunawan, A. S., and Romanzin, C. 1992. Common illumination between real and computer generated scenes. Tech. rep., University of British Columbia.

Digital Library

[6]

Goesele, M., Granier, X., Heidrich, W., and Seidel, H.-P. 2003. Accurate light source acquisition and rendering. In ACM Transactions on Graphics (Proc. SIGGRAPH), 621--630.

Digital Library

[7]

Jacobs, K., Nahmias, J.-D., Angus, C., Martinez, A. R., Loscos, C., and Steed, A. 2005. Automatic generation of consistent shadows for augmented reality. In Graphics Interface, 113--120.

Digital Library

[8]

Knecht, M., Traxler, C., Mattausch, O., Purgathofer, W., and Wimmer, M. 2010. Differential instant radiosity for mixed reality. In IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 99--107.

[9]

Knecht, M., Dünser, A., Traxler, C., Wimmer, M., and Grasset, R. 2011. A framework for perceptual studies in photorealistic augmented reality. In Proceedings of IEEE VR Workshop on Perceptual Illusions in Virtual Environments, 27--32.

[10]

Knecht, M., Traxler, C., Mattausch, O., and Wimmer, M. 2012. Reciprocal shading for mixed reality. Computers & Graphics 36, 7, 846--856.

Digital Library

[11]

Loos, B. J., Antani, L., Mitchell, K., Nowrouzezahrai, D., Jarosz, W., and Sloan, P.-P. 2011. Modular radiance transfer. ACM Trans. Graph. (ACM SIGGRAPH Asia) 30, 6.

Digital Library

[12]

Loos, B., James Nowrouzezahrai, D., Jarosz, W., and Sloan, P.-P. 2012. Delta radiance transfer. In ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games.

Digital Library

[13]

Nowrouzezahrai, D., Geiger, S., Mitchell, K., Sumner, R., Jarosz, W., and Gross, M. 2011. Light factorization for mixed-frequency shadows in augmented reality. In IEEE International Symposium on Mixed and Augmented Reality, 173--179.

Digital Library

[14]

Ramamoorthi, R. 2009. Precomputation-based rendering. Found. Trends. Comput. Graph. Vis. 3, 4 (Apr.), 281--369.

Digital Library

[15]

Ritschel, T., Grosch, T., Kim, M. H., Seidel, H.-P., Dachsbacher, C., and Kautz, J. 2008. Imperfect shadow maps for efficient computation of indirect illumination. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 27, 5, 129:1--129:8.

Digital Library

[16]

Schmalstieg, D., and Hollerer, T. 2013. Augmented Reality: Theory and Practice. Addison-Wesley.

[17]

Sloan, P.-P., Kautz, J., and Snyder, J. 2002. Precomputed radiance transfer for real-time rendering in dynamic, low-frequency lighting environments. ACM Transactions on Graphics (Proc. SIGGRAPH) 21, 3, 527--536.

Digital Library

[18]

Szeliski, R., and Shum, H.-Y. 1997. Creating full view panoramic image mosaics and environment maps. In SIGGRAPH '97, 251--258.

Digital Library

Cited By

Dorta Marques BGonzalez Clua EMontenegro ANader Vasconcelos C(2022)Spatially and color consistent environment lighting estimation using deep neural networks for mixed realityComputers and Graphics10.1016/j.cag.2021.08.007102:C(257-268)Online publication date: 1-Feb-2022
https://dl.acm.org/doi/10.1016/j.cag.2021.08.007
Swedish THenley CRaskar R(2021)Objects as Cameras: Estimating High-Frequency Illumination from Shadows2021 IEEE/CVF International Conference on Computer Vision (ICCV)10.1109/ICCV48922.2021.00259(2573-2582)Online publication date: Oct-2021
https://doi.org/10.1109/ICCV48922.2021.00259
Chen ZZheng QNiu PTang HPan G(2021)Indoor Lighting Estimation using an Event Camera2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR46437.2021.01452(14755-14765)Online publication date: Jun-2021
https://doi.org/10.1109/CVPR46437.2021.01452
Show More Cited By

Index Terms

The shading probe: fast appearance acquisition for mobile AR
1. Computing methodologies
  1. Computer graphics
    1. Image manipulation
      1. Texturing

Recommendations

A scalable framework for image-based material representations
Web3D '12: Proceedings of the 17th International Conference on 3D Web Technology

Complex material-light interaction is modeled mathematically in its most basic form through the 4D BRDF or the 6D spatially varying BRDF. To alleviate the overhead of calculating correct shading with a complex BRDF consisting of many parameters, many ...
Shading and inverse shading from direct illumination
NeLT: Object-Oriented Neural Light Transfer
This article presents object-oriented neural light transfer (NeLT), a novel neural representation of the dynamic light transportation between an object and the environment. Our method disentangles the global illumination of a scene into individual objects’...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SA '13: SIGGRAPH Asia 2013 Technical Briefs

November 2013

135 pages

ISBN:9781450326292

DOI:10.1145/2542355

Conference Chairs:
Baoquan Chen
Shenzhen Institute of Advanced Technology, China
,
Andrei Sharf
Ben Gurion University, Israel

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques

In-Cooperation

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 November 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Disney Research Zurich
University College London
Engineering and Physical Sciences Research Council

Conference

SA '13

Sponsor:

SIGGRAPH

SA '13: SIGGRAPH Asia 2013

November 19 - 22, 2013

Hong Kong, Hong Kong

Acceptance Rates

Overall Acceptance Rate 178 of 869 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
198
Total Downloads

Downloads (Last 12 months)11
Downloads (Last 6 weeks)1

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Dorta Marques BGonzalez Clua EMontenegro ANader Vasconcelos C(2022)Spatially and color consistent environment lighting estimation using deep neural networks for mixed realityComputers and Graphics10.1016/j.cag.2021.08.007102:C(257-268)Online publication date: 1-Feb-2022
https://dl.acm.org/doi/10.1016/j.cag.2021.08.007
Swedish THenley CRaskar R(2021)Objects as Cameras: Estimating High-Frequency Illumination from Shadows2021 IEEE/CVF International Conference on Computer Vision (ICCV)10.1109/ICCV48922.2021.00259(2573-2582)Online publication date: Oct-2021
https://doi.org/10.1109/ICCV48922.2021.00259
Chen ZZheng QNiu PTang HPan G(2021)Indoor Lighting Estimation using an Event Camera2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR46437.2021.01452(14755-14765)Online publication date: Jun-2021
https://doi.org/10.1109/CVPR46437.2021.01452
Gkitsas VZioulis NAlvarez FZarpalas DDaras P(2020)Deep Lighting Environment Map Estimation from Spherical Panoramas2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)10.1109/CVPRW50498.2020.00328(2719-2728)Online publication date: Jun-2020
https://doi.org/10.1109/CVPRW50498.2020.00328
Bi TMa JLiu YWeng DWang Y(2020)SIR-Net: Self-Supervised Transfer for Inverse Rendering via Deep Feature Fusion and Transformation From a Single ImageIEEE Access10.1109/ACCESS.2020.30352138(201861-201873)Online publication date: 2020
https://doi.org/10.1109/ACCESS.2020.3035213
Casas Cambra LFauconneau MKosek MMclister KMitchell K(2019)Enhanced Shadow Retargeting with Light-Source Estimation Using Flat Fresnel LensesComputers10.3390/computers80200298:2(29)Online publication date: 2-Apr-2019
https://doi.org/10.3390/computers8020029
Wei HLiu YXing GZhang YHuang W(2019)Simulating Shadow Interactions for Outdoor Augmented Reality With RGBD DataIEEE Access10.1109/ACCESS.2019.29209507(75292-75304)Online publication date: 2019
https://doi.org/10.1109/ACCESS.2019.2920950
Casas LFauconneau MKosek MMclister KMitchell KLaramee RTam GVidal F(2018)Image based proximate shadow retargetingProceedings of the Conference on Computer Graphics & Visual Computing10.2312/cgvc.20181206(43-50)Online publication date: 13-Sep-2018
https://dl.acm.org/doi/10.2312/cgvc.20181206
Walton DSteed AMorishima SItoh YShiratori TYue YLindeman R(2018)Dynamic HDR environment capture for mixed realityProceedings of the 24th ACM Symposium on Virtual Reality Software and Technology10.1145/3281505.3281531(1-11)Online publication date: 28-Nov-2018
https://dl.acm.org/doi/10.1145/3281505.3281531
Barreira JBessa MBarbosa LMagalhaes L(2018)A Context-Aware Method for Authentically Simulating Outdoors Shadows for Mobile Augmented RealityIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2017.267677724:3(1223-1231)Online publication date: 1-Mar-2018
https://doi.org/10.1109/TVCG.2017.2676777
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten