skip to main content
research-article

The camera offset space: real-time potentially visible set computations for streaming rendering

Published: 08 November 2019 Publication History

Abstract

Potential visibility has historically always been of importance when rendering performance was insufficient. With the rise of virtual reality, rendering power may once again be insufficient, e.g., for integrated graphics of head-mounted displays. To tackle the issue of efficient potential visibility computations on modern graphics hardware, we introduce the camera offset space (COS). Opposite to how traditional visibility computations work---where one determines which pixels are covered by an object under all potential viewpoints---the COS describes under which camera movement a sample location is covered by a triangle. In this way, the COS opens up a new set of possibilities for visibility computations. By evaluating the pairwise relations of triangles in the COS, we show how to efficiently determine occluded triangles. Constructing the COS for all pixels of a rendered view leads to a complete potentially visible set (PVS) for complex scenes. By fusing triangles to larger occluders, including locations between pixel centers, and considering camera rotations, we describe an exact PVS algorithm that includes all viewing directions inside a view cell. Implementing the COS is a combination of real-time rendering and compute steps. We provide the first GPU PVS implementation that works without preprocessing, on-the-fly, on unconnected triangles. This opens the door to a new approach of rendering for virtual reality head-mounted displays and server-client settings for streaming 3D applications such as video games.

Supplementary Material

ZIP File (a231-hladky.zip)
Supplemental files.
MP4 File (a231-hladky.mp4)

References

[1]
John M. Airey, John H. Rohlf, and Frederick P. Brooks, Jr. 1990. Towards Image Realism with Interactive Update Rates in Complex Virtual Building Environments. SIGGRAPH Comput. Graph. 24, 2 (Feb. 1990), 41--50.
[2]
Jiri Bittner, Vlastimil Havran, and Pavel Slavik. 1998. Hierarchical visibility culling with occlusion trees. In Computer Graphics International, 1998. Proc. 207--219.
[3]
Jiří Bittner, Oliver Mattausch, Peter Wonka, Vlastimil Havran, and Michael Wimmer. 2009. Adaptive Global Visibility Sampling. ACM TOG 28, 3, Article 94 (July 2009), 10 pages.
[4]
Jiří Bittner, Michael Wimmer, Harald Piringer, and Werner Purgathofer. 2004. Coherent hierarchical culling: Hardware occlusion queries made useful. In CGF, Vol. 23. 615--624.
[5]
Chris Buehler, Michael Bosse, Leonard McMillan, Steven Gortler, and Michael Cohen. 2001. Unstructured Lumigraph Rendering. In Proc. SIGGRAPH (SIGGRAPH '01). 425--432.
[6]
Christopher A. Burns, Kayvon Fatahalian, and William R. Mark. 2010. A Lazy Object-space Shading Architecture with Decoupled Sampling. In Proc. High Performance Graphics (HPG '10). 19--28.
[7]
Chun-Fa Chang and Shyh-Haur Ger. 2002. Enhancing 3D Graphics on Mobile Devices by Image-Based Rendering. In Proc. of the Third IEEE Pacific Rim Conference on Multimedia: Advances in Multimedia Information Processing (PCM '02). 1105--1111.
[8]
Shenchang Eric Chen and Lance Williams. 1993. View Interpolation for Image Synthesis. In Proc. SIGGRAPH (SIGGRAPH '93). 279--288.
[9]
Daniel Cohen-Or, Yiorgos L Chrysanthou, Claudio T. Silva, and Frédo Durand. 2003. A survey of visibility for walkthrough applications. IEEE Transactions on Visualization and Computer Graphics 9, 3 (2003), 412--431.
[10]
Satyan Coorg and Seth Teller. 1999. Temporally Coherent Conservative Visibility. Comput. Geom. Theory Appl. 12, 1--2 (Feb. 1999), 105--124.
[11]
Thomas H Cormen. 2009. Introduction to algorithms. MIT press.
[12]
Wagner T. Correa, James T. Klosowski, and Claudio T. Silva. 2003. Visibility-Based Prefetching for Interactive Out-Of-Core Rendering. In Proc. PVG (PVG '03). 2-.
[13]
Paul E. Debevec, Camillo J. Taylor, and Jitendra Malik. 1996. Modeling and Rendering Architecture from Photographs: A Hybrid Geometry- and Image-based Approach. In Proc. SIGGRAPH (SIGGRAPH '96). 11--20.
[14]
Piotr Didyk, Elmar Eisemann, Tobias Ritschel, Karol Myszkowski, and Hans-Peter Seidel. 2010. Perceptually-motivated Real-time Temporal Upsampling of 3D Content for High-refresh-rate Displays. CGF (Proc. Eurographics 2010) 29, 2 (2010), 713--722.
[15]
Frédo Durand, George Drettakis, Joëlle Thollot, and Claude Puech. 2000. Conservative visibility preprocessing using extended projections. In Proc. Computer graphics and interactive techniques. 239--248.
[16]
Thomas A Funkhouser. 1996. Database management for interactive display of large architectural models. In Graphics Interface, Vol. 96. 1--8.
[17]
Craig Gotsman, Oded Sudarsky, and Jeffrey A Fayman. 1999. Optimized occlusion culling using five-dimensional subdivision. Computers & Graphics 23, 5 (1999), 645--654.
[18]
Naga K. Govindaraju, Avneesh Sud, Sung-Eui Yoon, and Dinesh Manocha. 2003. Interactive Visibility Culling in Complex Environments Using Occlusion-switches. In Proc. of the 2003 Symposium on Interactive 3D Graphics (I3D '03). 103--112.
[19]
Jon Hasselgren, Tomas Akenine-Möller, and Lennart Ohlsson. 2005. Conservative rasterization. GPU Gems 2 (2005), 677--690.
[20]
Karl E. Hillesland and J. C. Yang. 2016. Texel Shading. In EG 2016 - Short Papers, T. Bashford-Rogers and L. P. Santos (Eds.). The Eurographics Association.
[21]
Jozef Hladky, Hans-Peter Seidel, and Markus Steinberger. 2019. Tessellated Shading Streaming. Computer Graphics Forum 38, 4 (2019), 12.
[22]
Lichan Hong, Shigeru Muraki, Arie Kaufman, Dirk Bartz, and Taosong He. 1997. Virtual voyage: Interactive navigation in the human colon. In Proc. Computer graphics and interactive techniques. 27--34.
[23]
Junxian Huang, Feng Qian, Alexandre Gerber, Z. Morley Mao, Subhabrata Sen, and Oliver Spatscheck. 2012. A Close Examination of Performance and Power Characteristics of 4G LTE Networks. In Proc. of International Conference on Mobile Systems, Applications, and Services (MobiSys '12). 225--238.
[24]
Tom Hudson, Dinesh Manocha, Jonathan Cohen, Ming Lin, Kenneth Hoff, and Hansong Zhang. 1997. Accelerated occlusion culling using shadow frusta. In Proc. Computational geometry. 1--10.
[25]
WFH Jiménez, Claudio Esperança, and Antonio AF Oliveira. 2000. Efficient algorithms for computing conservative portal visibility information. In CGF, Vol. 19. 489--498.
[26]
Vladlen Koltun, Yiorgos Chrysanthou, and Daniel Cohen-Or. 2001. Hardware-accelerated from-region visibility using a dual ray space. In Rendering Techniques 2001. 205--215.
[27]
Johannes Kopf, Fabian Langguth, Daniel Scharstein, Richard Szeliski, and Michael Goesele. 2013. Image-based rendering in the gradient domain. ACM TOG 32, 6 (2013), 1--9.
[28]
Kyungmin Lee, David Chu, Eduardo Cuervo, Johannes Kopf, Alec Wolman, Yury Degtyarev, Sergey Grizan, and Jason Flinn. 2015. Outatime: Using Speculation to Enable Low-Latency Continuous Interaction for Mobile Cloud Gaming. GetMobile: Mobile Comp. and Comm. 19, 3 (Dec. 2015), 14--17.
[29]
Tommer Leyvand, Olga Sorkine, and Daniel Cohen-Or. 2003. Ray Space Factorization for From-region Visibility. ACM TOG 22, 3 (July 2003), 595--604.
[30]
Gerrit Lochmann, Bernhard Reinert, Tobias Ritschel, Stefan Müller, and Hans-Peter Seidel. 2014. Real-time Reflective and Refractive Novel-view Synthesis, Jan Bender, Arjan Kuijper, Tatiana von Landesberger, Holger Theisel, and Philipp Urban (Eds.). Eurographics Association, Darmstadt, Germany, 9--16.
[31]
David Luebke and Chris Georges. 1995. Portals and mirrors: Simple, fast evaluation of potentially visible sets. In Proc. Interactive 3D graphics. 105-ff.
[32]
William R. Mark, Leonard McMillan, and Gary Bishop. 1997. Post-rendering 3D warping. Symposium on Interactive 3D Graphics Figure 2 (1997), 7--16.
[33]
Joerg H. Mueller, Philip Voglreiter, Mark Dokter, Thomas Neff, Mina Makar, Markus Steinberger, and Dieter Schmalstieg. 2018. Shading Atlas Streaming. ACM TOG 37, 6, Article 199 (Dec. 2018), 16 pages.
[34]
Diego Nehab, Pedro V. Sander, Jason Lawrence, Natalya Tatarchuk, and John R. Isidoro. 2007. Accelerating Real-time Shading with Reverse Reprojection Caching. In Proc. Symposium on Graphics Hardware (GH '07). 25--35.
[35]
Marc Olano and Trey Greer. 1997. Triangle scan conversion using 2D homogeneous coordinates. In Proc. workshop on Graphics hardware. 89--95.
[36]
Bernhard Reinert, Johannes Kopf, Tobias Ritschel, Eduardo Cuervo, David Chu, and Hans-Peter Seidel. 2016. Proxy-guided Image-based Rendering for Mobile Devices. CGF 35, 7 (2016), 353--362.
[37]
Gernot Schaufler, Julie Dorsey, Xavier Decoret, and François X Sillion. 2000. Conservative volumetric visibility with occluder fusion. In Proc. Computer graphics and interactive techniques. 229--238.
[38]
Daniel Scherzer, Lei Yang, Oliver Mattausch, Diego Nehab, Pedro V. Sander, Michael Wimmer, and Elmar Eisemann. 2011. A Survey on Temporal Coherence Methods in Real-Time Rendering. In EUROGRAPHICS 2011 State of the Art Reports. 101--126.
[39]
Shu Shi and Cheng-Hsin Hsu. 2015. A Survey of Interactive Remote Rendering Systems. ACM Comput. Surv. 47, 4, Article 57 (May 2015), 29 pages.
[40]
Sudipta N. Sinha, Johannes Kopf, Michael Goesele, Daniel Scharstein, and Richard Szeliski. 2012. Image-based rendering for scenes with reflections. ACM TOG 31, 4 (2012), 1--10.
[41]
Pitchaya Sitthi-amorn, Jason Lawrence, Lei Yang, Pedro V. Sander, Diego Nehab, and Jiahe Xi. 2008. Automated Reprojection-based Pixel Shader Optimization. ACM TOG 27, 5, Article 127 (Dec. 2008), 11 pages.
[42]
Seth Teller and Pat Hanrahan. 1993. Global Visibility Algorithms for Illumination Computations. In Proc. SIGGRAPH (SIGGRAPH '93). 239--246.
[43]
Seth J Teller and Carlo H Séquin. 1991. Visibility preprocessing for interactive walkthroughs. In ACM SIGGRAPH Computer Graphics, Vol. 25. 61--70.
[44]
George Wolberg. 1998. Image morphing: a survey. The Visual Computer 14, 8--9 (1998), 360--372.
[45]
Peter Wonka, Michael Wimmer, and Dieter Schmalstieg. 2000. Visibility preprocessing with occluder fusion for urban walkthroughs. In Rendering Techniques 2000. 71--82.
[46]
P. Wonka, M. Wimmer, and F. X. Sillion. 2001. Instant visibility. CGF 20, 3 (9 2001).
[47]
Jason C. Yang, Justin Hensley, Holger Grün, and Nicolas Thibieroz. 2010. Real-time Concurrent Linked List Construction on the GPU. In Proc. EGSR (EGSR'10). 1297--1304.
[48]
Lei Yang, Yu-Chiu Tse, Pedro V. Sander, Jason Lawrence, Diego Nehab, Hugues Hoppe, and Clara L. Wilkins. 2011. Image-based Bidirectional Scene Reprojection. ACM TOG 30, 6, Article 150 (Dec. 2011), 10 pages.
[49]
Hansong Zhang, Dinesh Manocha, Tom Hudson, and Kenneth E. Hoff. 1997. Visibility culling using hierarchical occlusion maps. ACM, 77--88.
[50]
Henning Zimmer, Fabrice Rousselle, Wenzel Jakob, Oliver Wang, David Adler, Wojciech Jarosz, Olga Sorkine-hornung, and Alexander Sorkine-hornung. 2015. Path-space Motion Estimation and Decomposition for Robust Animation Filtering. Egsr 2015 34, 4 (2015), 12.

Cited By

View all

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics
ACM Transactions on Graphics  Volume 38, Issue 6
December 2019
1292 pages
ISSN:0730-0301
EISSN:1557-7368
DOI:10.1145/3355089
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 November 2019
Published in TOG Volume 38, Issue 6

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. GPU
  2. potentially visible set
  3. real-time rendering
  4. streaming rendering
  5. virtual reality
  6. visibility

Qualifiers

  • Research-article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)29
  • Downloads (Last 6 weeks)1
Reflects downloads up to 13 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)CloVR: Fast-Startup Low-Latency Cloud VRIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.337205930:5(2337-2346)Online publication date: 4-Mar-2024
  • (2024)Efficient and Robust From-Point VisibilityIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.329113830:8(5313-5327)Online publication date: 1-Aug-2024
  • (2024)Eye-tracking on virtual reality: a surveyVirtual Reality10.1007/s10055-023-00903-y28:1Online publication date: 5-Feb-2024
  • (2023)Visibility-Based R-Tree Spatial Index for Consistent Visualization in Indoor and Outdoor ScenesISPRS International Journal of Geo-Information10.3390/ijgi1212049812:12(498)Online publication date: 12-Dec-2023
  • (2023)Trim Regions for Online Computation of From-Region Potentially Visible SetsACM Transactions on Graphics10.1145/359243442:4(1-15)Online publication date: 26-Jul-2023
  • (2023)Potentially Visible Hidden-Volume Rendering for Multi-View WarpingACM Transactions on Graphics10.1145/359210842:4(1-11)Online publication date: 26-Jul-2023
  • (2023)PVLI: potentially visible layered image for real-time ray tracingThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-023-03007-539:8(3359-3372)Online publication date: 1-Aug-2023
  • (2023)Fine-Grained Web3D Culling-Transmitting-Rendering PipelineAdvances in Computer Graphics10.1007/978-3-031-50072-5_13(159-170)Online publication date: 28-Aug-2023
  • (2022)QuadStreamACM Transactions on Graphics10.1145/3550454.355552441:6(1-13)Online publication date: 30-Nov-2022
  • (2022)Meshlets and How to Shade Them: A Study on Texture‐Space ShadingComputer Graphics Forum10.1111/cgf.1447441:2(277-287)Online publication date: 24-May-2022
  • Show More Cited By

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media