research-article

Cloud Baking: Collaborative Scene Illumination for Dynamic Web3D Scenes

Authors:

Lei ZhaoAuthors Info & Claims

ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), Volume 14, Issue 3s

Article No.: 59, Pages 1 - 20

https://doi.org/10.1145/3206431

Published: 15 June 2018 Publication History

Abstract

We propose Cloud Baking, a collaborative rendering architecture for dynamic Web3D scenes. In our architecture, the cloud renderer renders the scene with the global illumination (GI) information in a GI map; the web-based client renderer renders the scene with ambient lighting only and blends it with the GI map received from the cloud for the final scene. This approach allows the users to interact with the web scene and change the scene dynamically through the web interface end, yet move the computationally heavy tasks of global illumination computation to the cloud. A challenge we face is the interaction delay that causes the frames rendered on the cloud and the client to go out of sync. We propose to use 3D warping and a hole-filling algorithm designed for GI map to predict the late GI map. We show both quantitatively and visually the quality of the GI map produced using our method. Our prediction algorithm allows us to further reduce the frequency at which the GI map is computed and sent from the server, reducing both computational needs and bandwidth usage.

References

[1]

P. Bao and D. Gourlay. 2004. Remote walkthrough over mobile networks using 3-D image warping and streaming. IEE Proceedings-Vision, Image and Signal Processing 151, 4 (2004), 329--336.

[2]

Ken W. Brodlie, David A. Duce, Julian R. Gallop, J. P. R. B. Walton, and J. D. Wood. 2004. Distributed and collaborative visualization. Computer Graphics Forum 23, 2 (2004), 223--251.

[3]

Kuan-Ta Chen, Yu-Chun Chang, Po-Han Tseng, Chun-Ying Huang, and Chin-Laung Lei. 2011. Measuring the latency of cloud gaming systems. In Proceedings of the 19th ACM International Conference on Multimedia. ACM, Scottsdale, AZ, 1269--1272.

Digital Library

[4]

Luca Chittaro and Roberto Ranon. 2007. Web3D technologies in learning, education and training: Motivations, issues, opportunities. Computers 8 Education 49, 1 (2007), 3--18.

Digital Library

[5]

Sharon Choy, Bernard Wong, Gwendal Simon, and Catherine Rosenberg. 2012. The brewing storm in cloud gaming: A measurement study on cloud to end-user latency. In Proceedings of the 11th Annual Workshop on Network and Systems Support for Games. IEEE Press, 2.

Digital Library

[6]

Mark Claypool and Kajal Claypool. 2006. Latency and player actions in online games. Commun. ACM 49, 11 (2006), 40--45.

Digital Library

[7]

Cyril Crassin, David Luebke, Michael Mara, Morgan McGuire, Brent Oster, Peter Shirley, Peter-Pike Sloan, and Chris Wyman. 2015. CloudLight: A system for amortizing indirect lighting in real-time rendering. Journal of Computer Graphics Techniques 4, 4 (2015), 1--27.

[8]

Cyril Crassin, Fabrice Neyret, Miguel Sainz, Simon Green, and Elmar Eisemann. 2011. Interactive indirect illumination using voxel cone tracing. Computer Graphics Forum 30, 7 (2011), 1921--1930.

[9]

Carsten Dachsbacher and Marc Stamminger. 2005. Reflective shadow maps. In Proceedings of the 2005 Symposium on Interactive 3D Graphics and Games. ACM, 203--231.

Digital Library

[10]

Jos Dirksen. 2013. Learning Three.js: the JavaScript 3D Library for WebGL. Packt Publishing Ltd.

[11]

Florent Duguet and George Drettakis. 2004. Flexible point-based rendering on mobile devices. IEEE Computer Graphics and Applications 24, 4 (2004), 57--63.

Digital Library

[12]

Christoph Fehn. 2004. Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3D-TV. Proceedings of SPIE 5291, 2 (2004), 93--104.

[13]

Keith Gladstien. 2013. Flash Game Development In a Social, Mobile and 3D World. Cengage Learning.

Digital Library

[14]

Liwei Hsu. 2012. Web 3D simulation-based application in tourism education: A case study with Second Life. Journal of Hospitality, Leisure, Sport 8 Tourism Education 11, 2 (2012), 113--124.

[15]

Anton Kaplanyan and Carsten Dachsbacher. 2010. Cascaded light propagation volumes for real-time indirect illumination. In Proceedings of the 2010 ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games. ACM, 99--107.

Digital Library

[16]

David Koller, Michael Turitzin, Marc Levoy, Marco Tarini, Giuseppe Croccia, Paolo Cignoni, and Roberto Scopigno. 2004. Protected interactive 3D graphics via remote rendering. ACM Transactions on Graphics 23, 3 (2004), 695--703.

Digital Library

[17]

Kyungmin Lee, David Chu, Eduardo Cuervo, Johannes Kopf, Yury Degtyarev, Sergey Grizan, Alec Wolman, and Jason Flinn. 2015. Outatime: Using speculation to enable low-latency continuous interaction for mobile cloud gaming. In Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 151--165.

Digital Library

[18]

Javier Lluch, Rafael Gaitán, Emilio Camahort, and Roberto Vivó. 2005. Interactive three-dimensional rendering on mobile computer devices. In Proceedings of the 2005 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology. ACM, 254--257.

Digital Library

[19]

William R. Mark, Leonard McMillan, and Gary Bishop. 1997. Post-rendering 3D warping. In Proceedings of the 1997 Symposium on Interactive 3D Graphics. ACM, 7--ff.

Digital Library

[20]

Leonard McMillan Jr. 1997. An Image-Based Approach to Three-Dimensional Computer Graphics. Ph.D. Dissertation. University of North Carolina at Chapel Hill.

Digital Library

[21]

Yuji Mori, Norishige Fukushima, Tomohiro Yendo, Toshiaki Fujii, and Masayuki Tanimoto. 2009. View generation with 3D warping using depth information for FTV. Signal Processing: Image Communication 24, 1 (2009), 65--72. Special issue on advances in three-dimensional television and video.

Digital Library

[22]

Fiona Fui-Hoon Nah. 2004. A study on tolerable waiting time: How long are web users willing to wait? Behaviour 8 Information Technology 23, 3 (2004), 153--163.

[23]

Brian Paul, Sean Ahern, Wes Bethel, Eric Brugger, Rich Cook, Jamison Daniel, Ken Lewis, Jens Owen, and Dale Southard. 2008. Chromium renderserver: Scalable and open remote rendering infrastructure. IEEE Transactions on Visualization and Computer Graphics 14, 3 (2008), 627--639.

Digital Library

[24]

Jonathan Shade, Steven Gortler, Li-wei He, and Richard Szeliski. 1998. Layered depth images. In Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques. ACM, 231--242.

Digital Library

[25]

Shu Shi, Klara Nahrstedt, and Roy Campbell. 2012. A real-time remote rendering system for interactive mobile graphics. ACM Transactions on Multimedia Computing, Communications, and Applications 8, 3s (2012), 46.

Digital Library

[26]

Aljoscha Smolic, Karsten Muller, Kristina Dix, Philipp Merkle, Peter Kauff, and Thomas Wiegand. 2008. Intermediate view interpolation based on multiview video plus depth for advanced 3D video systems. In 2008 15th IEEE International Conference on Image Processing. 2448--2451.

[27]

Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. 2004. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing 13, 4 (2004), 600--612.

Digital Library

[28]

Laixiang Wen, Jinyuan Jia, and Shuang Liang. 2014. LPM: Lightweight progressive meshes towards smooth transmission of Web3D media over internet. In Proceedings of the 13th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and its Applications in Industry. ACM, 95--103.

Digital Library

[29]

Laixiang Wen, Ning Xie, and Jinyuan Jia. 2016. Client-driven strategy of large-scale scene streaming. In Proceedings of the International Conference on Multimedia Modeling. Springer, 93--103.

[30]

Minhui Zhu, Sebastien Mondet, Géraldine Morin, Wei Tsang Ooi, and Wei Cheng. 2011. Towards peer-assisted rendering in networked virtual environments. In Proceedings of the 19th International Conference on Multimedia, November 28--December 1. Scottsdale, AZ, 183--192.

Digital Library

[31]

Thomas Zinner, Thomas Zinner, Osama Abboud, and Tobias Hossfeld. 2010. Impact of frame rate and resolution on objective QoE metrics. In The 2nd International Workshop on Quality of Multimedia Experience (QoMEX’10). 29--34.

Cited By

Tatzgern WWeinrauch AStadlbauer PMueller JWinter MSteinberger M(2024)Radiance Caching with On-Surface Caches for Real-Time Global IlluminationProceedings of the ACM on Computer Graphics and Interactive Techniques10.1145/36753827:3(1-17)Online publication date: 9-Aug-2024
https://dl.acm.org/doi/10.1145/3675382
Azad FCandan KKapkiç ALi MLiu HMandal PSheth PArslan BChowell-puente GSabo JMuenich RRedondo Anton JSapino M(2024)(Vision Paper) A Vision for Spatio-Causal Situation Awareness, Forecasting, and PlanningACM Transactions on Spatial Algorithms and Systems10.1145/367255610:2(1-42)Online publication date: 12-Jun-2024
https://dl.acm.org/doi/10.1145/3672556
Bi WMa YHan YChen YTian DDu JChua TNgo CKa-Wei Lee RKumar RLauw H(2024)FusionRender: Harnessing WebGPU's Power for Enhanced Graphics Performance on Web BrowsersProceedings of the ACM Web Conference 202410.1145/3589334.3645395(2890-2901)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645395
Show More Cited By

Index Terms

Cloud Baking: Collaborative Scene Illumination for Dynamic Web3D Scenes

Recommendations

Lightweight WebSIM Rendering Framework Based on Cloud-Baking
SIGSIM-PADS '17: Proceedings of the 2017 ACM SIGSIM Conference on Principles of Advanced Discrete Simulation

Current cloud rendering systems are expensive in terms of data storage, rendering computation, and networking transmission. In this paper, we propose a novel framework for lightweight and realistic WebSIM rendering based on cloud baking. Different from ...
Ambient obscurance baking on the GPU
SA '13: SIGGRAPH Asia 2013 Technical Briefs

Ambient Occlusion and Ambient Obscurance are coarse approximations to global illumination from ambient lighting, commonly used in film and games. This paper describes a system that computes Ambient Obscurance over the vertices of complex polygon meshes. ...
Splatting indirect illumination
I3D '06: Proceedings of the 2006 symposium on Interactive 3D graphics and games

In this paper we present a novel method for plausible real-time rendering of indirect illumination effects for diffuse and non-diffuse surfaces. The scene geometry causing indirect illumination is captured by an extended shadow map, as proposed in ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Multimedia Computing, Communications, and Applications

ACM Transactions on Multimedia Computing, Communications, and Applications Volume 14, Issue 3s

Special Section on Delay-Sensitive Video Computing in the Cloud and Special Section on Extended MMSys-NOSSDAV Best Papers

June 2018

317 pages

ISSN:1551-6857

EISSN:1551-6865

DOI:10.1145/3233173

Editor:
Alberto Del Bimbo
University of Firenze, Italy

Issue’s Table of Contents

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 June 2018

Accepted: 01 April 2018

Revised: 01 February 2018

Received: 01 August 2017

Published in TOMM Volume 14, Issue 3s

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Fundamental Research Funds for Chinese Central Universities

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
430
Total Downloads

Downloads (Last 12 months)26
Downloads (Last 6 weeks)0

Reflects downloads up to 27 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Tatzgern WWeinrauch AStadlbauer PMueller JWinter MSteinberger M(2024)Radiance Caching with On-Surface Caches for Real-Time Global IlluminationProceedings of the ACM on Computer Graphics and Interactive Techniques10.1145/36753827:3(1-17)Online publication date: 9-Aug-2024
https://dl.acm.org/doi/10.1145/3675382
Azad FCandan KKapkiç ALi MLiu HMandal PSheth PArslan BChowell-puente GSabo JMuenich RRedondo Anton JSapino M(2024)(Vision Paper) A Vision for Spatio-Causal Situation Awareness, Forecasting, and PlanningACM Transactions on Spatial Algorithms and Systems10.1145/367255610:2(1-42)Online publication date: 12-Jun-2024
https://dl.acm.org/doi/10.1145/3672556
Bi WMa YHan YChen YTian DDu JChua TNgo CKa-Wei Lee RKumar RLauw H(2024)FusionRender: Harnessing WebGPU's Power for Enhanced Graphics Performance on Web BrowsersProceedings of the ACM Web Conference 202410.1145/3589334.3645395(2890-2901)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645395
Long HYang YLiu CJia J(2024)SRSSIS: Super-Resolution Screen Space Irradiance Sampling for Lightweight Collaborative Web3D Rendering ArchitectureComputer-Aided Design and Computer Graphics10.1007/978-981-99-9666-7_20(295-313)Online publication date: 7-Feb-2024
https://doi.org/10.1007/978-981-99-9666-7_20
Weinrauch ATatzgern WStadlbauer PCrickx AHladky JCoomans AWinter MMueller JSteinberger M(2023)Effect-based Multi-viewer Caching for Cloud-native RenderingACM Transactions on Graphics10.1145/359243142:4(1-16)Online publication date: 26-Jul-2023
https://dl.acm.org/doi/10.1145/3592431
Li LHuang YQiao XMeng YYu DRen PDustdar S(2023)Toward Distributed Collaborative Rendering Service for Immersive Mobile WebIEEE Network: The Magazine of Global Internetworking10.1109/MNET.133.220052438:3(137-145)Online publication date: 26-Jun-2023
https://dl.acm.org/doi/10.1109/MNET.133.2200524
Cai DHuang ZZhou DQian L(2023)Deadline Sensitive Cloud Computing Resource Scheduling Method for Scene Rendering2023 Fourth International Conference on Frontiers of Computers and Communication Engineering (FCCE)10.1109/FCCE58525.2023.00010(12-16)Online publication date: Jan-2023
https://doi.org/10.1109/FCCE58525.2023.00010
Yu GLiu CFang TJia JLin EHe YFu SWang LWei LHuang Q(2023)A survey of real-time rendering on Web3D applicationVirtual Reality & Intelligent Hardware10.1016/j.vrih.2022.04.0025:5(379-394)Online publication date: Oct-2023
https://doi.org/10.1016/j.vrih.2022.04.002
Liu CSong HFang TOu QYu GYou TYing M(2022)Web-Cloud Collaborative Mobile Online 3D Rendering SystemSecurity and Communication Networks10.1155/2022/47489462022Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1155/2022/4748946
Nyamtiga BHermawan ALuckyarno YKim TJung DKwak JYun J(2022)Edge-Computing-Assisted Virtual Reality Computation Offloading: An Empirical StudyIEEE Access10.1109/ACCESS.2022.320512010(95892-95907)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3205120
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents