Cited By
View all- Yang ZLiu BSong YYi LXiong YZhang ZYu X(2024)DirectL: Efficient Radiance Fields Rendering for 3D Light Field DisplaysACM Transactions on Graphics10.1145/368789743:6(1-19)Online publication date: 19-Dec-2024
Light Field Display Point Rendering
Article No.: 19, Pages 1 - 18
Abstract
Rendering for light field displays (LFDs) requires rendering of dozens or hundreds of views, which must then be combined into a single image on the display, making real-time LFD rendering extremely difficult. We introduce light field display point rendering (LFDPR), which meets these challenges by improving eye-based point rendering [Gavane and Watson 2023] with texture-based splatting, which avoids oversampling of triangles mapped to only a few texels; and with LFD-biased sampling, which adjusts horizontal and vertical triangle sampling to match the sampling of the LFD itself. To improve image quality, we introduce multiview mipmapping, which reduces texture aliasing even though compute shaders do not support hardware mipmapping. We also introduce angular supersampling and reconstruction to combat LFD view aliasing and crosstalk. The resulting LFDPR is 2-8x times faster than multiview rendering, with similar comparable quality.
Supplemental Material
ZIP File
Supplemental files
- Download
- 617.34 MB
References
[1]
2022. 3D Monitor without Galsses. https://aye3d.com/moniteur-3d/ [Online; accessed 01/04/2024].
[2]
2024. Dimenco - Products. https://www.dimenco.eu/sr-products [Online; accessed 01/04/2024].
[3]
2024. Documentation - Looking Glass Factory. https://docs.lookingglassfactory.com/ [Online; accessed 01/04/2024].
[4]
Edward H Adelson, James R Bergen, et al. 1991. The plenoptic function and the elements of early vision. Computational models of visual processing 1, 2 (1991), 3--20.
[5]
Tomas Akenine-Moller, Eric Haines, and Naty Hoffman. 2018. Real-time rendering. AK Peters/CRC Press.
[6]
Mojtaba Bemana, Karol Myszkowski, Hans-Peter Seidel, and Tobias Ritschel. 2020. X-Fields: Implicit Neural View-, Light- and Time-Image Interpolation. ACM Transactions on Graphics (Proc. SIGGRAPH Asia 2020) 39, 6 (2020). https://doi.org/10.1145/3414685.3417827
[7]
Franklin C Crow. 1977. The aliasing problem in computer-generated shaded images. Commun. ACM 20, 11 (1977), 799--805.
[8]
François De Sorbier, Vincent Nozick, and Hideo Saito. 2010. Gpu-based multi-view rendering. In Computer Games, Multimedia and Allied Technology. 7--13.
[9]
Laura Fink, Svenja Strobel, Linus Franke, and Marc Stamminger. 2023. Efficient Rendering for Light Field Displays using Tailored Projective Mappings. Proceedings of the ACM on Computer Graphics and Interactive Techniques 6, 1 (2023), 1--17.
[10]
John Flynn, Michael Broxton, Paul Debevec, Matthew DuVall, Graham Fyffe, Ryan Overbeck, Noah Snavely, and Richard Tucker. 2019. DeepView: View Synthesis With Learned Gradient Descent. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2362--2371. https://doi.org/10.1109/CVPR.2019.00247
[11]
Ajinkya Gavane and Benjamin Watson. 2022. Improving View Independent Rendering for Multiview Effects. (2022).
[12]
Ajinkya Gavane and Benjamin Watson. 2023. Eye-Based Point Rendering for Dynamic Multiview Effects. Proceedings of the ACM on Computer Graphics and Interactive Techniques 6, 1 (2023), 1--16.
[13]
Andreī Gershun. 1939. The light field. J. Mathematics and Physics 18, 1-4 (1939), 51--151.
[14]
Steven J Gortler, Radek Grzeszczuk, Richard Szeliski, and Michael F Cohen. 1996. The lumigraph. In Proceedings of the 23rd annual conference on Computer graphics and interactive techniques. ACM, 43--54.
[15]
Markus Gross and Hanspeter Pfister. 2011. Point-based graphics. Elsevier.
[16]
Jeff Heusser, John Montgomery, and Mike Seymour. 2018. Light Field Labs: 3D Holograms no glasses Deep Dive. https://www.fxguide.com/featured/light-field-labs-3d-holograms-no-glasses-deep-dive/
[17]
Thomas Hübner and Renato Pajarola. 2007. Single-pass multi-view volume rendering. (2007).
[18]
Thomas Hübner, Yanci Zhang, and Renato Pajarola. 2006. Multi-view point splatting. In Proceedings of the 4th international conference on Computer graphics and interactive techniques in Australasia and Southeast Asia. 285--294.
[19]
Frederic E Ives. 1903. Parallax stereogram and process of making same. US Patent 725,567.
[20]
Douglas Lanman and David Luebke. 2013. Near-eye light field displays. ACM Trans. on Graphics 32, 6 (2013), 220.
[21]
Marc Levoy and Pat Hanrahan. 1996. Light field rendering. In Proc. 23rd annual conference on Computer graphics and interactive techniques. ACM, 31--42.
[22]
Marc Levoy and Turner Whitted. 1985. The use of points as display primitives (Technical Report TR 85-022).
[23]
Rafal K Mantiuk, Dounia Hammou, and Param Hanji. 2023. HDR-VDP-3: A multi-metric for predicting image differences, quality and contrast distortions in high dynamic range and regular content. arXiv preprint arXiv:2304.13625 (2023).
[24]
Adam Marrs, Benjamin Watson, and Christopher G Healey. 2017. Real-Time View Independent Rasterization for Multi-View Rendering. In Eurographics (Short Papers). 17--20.
[25]
Kshitij Marwah, Gordon Wetzstein, Yosuke Bando, and Ramesh Raskar. 2013. Compressive light field photography using overcomplete dictionaries and optimized projections. ACM Transactions on Graphics (TOG) 32, 4 (2013), 1--12.
[26]
Wojciech Matusik and Hanspeter Pfister. 2004. 3D TV: a scalable system for real-time acquisition, transmission, and autostereoscopic display of dynamic scenes. ACM Transactions on Graphics (TOG) 23, 3 (2004), 814--824.
[27]
Morgan McGuire. 2017. Computer Graphics Archive. https://casual-effects.com/data
[28]
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. 2021. Nerf: Representing scenes as neural radiance fields for view synthesis. Commun. ACM 65, 1 (2021), 99--106.
[29]
Ren Ng, Marc Levoy, Mathieu Brédif, Gene Duval, Mark Horowitz, and Pat Hanrahan. 2005. Light field photography with a hand-held plenoptic camera. Ph. D. Dissertation. Stanford university.
[30]
Maria Perez-Ortiz, Aliaksei Mikhailiuk, Emin Zerman, Vedad Hulusic, Giuseppe Valenzise, and Rafał K Mantiuk. 2019. From pairwise comparisons and rating to a unified quality scale. IEEE Transactions on Image Processing 29 (2019), 1139--1151.
[31]
Matt Pharr, Wenzel Jakob, and Greg Humphreys. 2016. Physically based rendering: From theory to implementation. Morgan Kaufmann.
[32]
Tobias Ritschel, Elmar Eisemann, Inwoo Ha, James DK Kim, and Hans-Peter Seidel. 2011. Making imperfect shadow maps view-adaptive: High-quality global illumination in large dynamic scenes. In Computer Graphics Forum, Vol. 30. Wiley Online Library, 2258--2269.
[33]
Tobias Ritschel, Thomas Engelhardt, Thorsten Grosch, H-P Seidel, Jan Kautz, and Carsten Dachsbacher. 2009. Micro-rendering for scalable, parallel final gathering. ACM Transactions on Graphics (TOG) 28, 5 (2009), 1--8.
[34]
Tobias Ritschel, Thorsten Grosch, Min H Kim, H-P Seidel, Carsten Dachsbacher, and Jan Kautz. 2008. Imperfect shadow maps for efficient computation of indirect illumination. ACM transactions on graphics (tog) 27, 5 (2008), 1--8.
[35]
Markus Schütz, Bernhard Kerbl, and Michael Wimmer. 2021. Rendering Point Clouds with Compute Shaders and Vertex Order Optimization. arXiv preprint arXiv:2104.07526 (2021).
[36]
Mohammed Suhail, Carlos Esteves, Leonid Sigal, and Ameesh Makadia. 2022. Light field neural rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 8269--8279.
[37]
Colin Behrens Svenja Strobel. 2020. Coconut Scene.
[38]
Johannes Unterguggenberger, Bernhard Kerbl, Markus Steinberger, Dieter Schmalstieg, and Michael Wimmer. 2020. Fast Multi-View Rendering for Real-Time Applications. In EGPGV@ Eurographics/EuroVis. 13--23.
[39]
Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Andrew Tao, Jan Kautz, and Bryan Catanzaro. 2018. High-resolution image synthesis and semantic manipulation with conditional gans. In Proceedings of the IEEE conference on computer vision and pattern recognition. 8798--8807.
[40]
Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing 13, 4 (2004), 600--612.
[41]
Bennett Wilburn, Neel Joshi, Vaibhav Vaish, Eino-Ville Talvala, Emilio Antunez, Adam Barth, Andrew Adams, Mark Horowitz, and Marc Levoy. 2005. High performance imaging using large camera arrays. In ACM SIGGRAPH 2005 Papers. 765--776.
[42]
Suttisak Wizadwongsa, Pakkapon Phongthawee, Jiraphon Yenphraphai, and Supasorn Suwajanakorn. 2021. NeX: Real-time View Synthesis with Neural Basis Expansion. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[43]
Xunbo Yu, Xinzhu Sang, Duo Chen, Peng Wang, Xin Gao, Tianqi Zhao, Binbin Yan, Chongxiu Yu, Daxiong Xu, and Wenhua Dou. 2014. Autostereoscopic three-dimensional display with high dense views and the narrow structure pitch. Chinese Optics Letters 12, 6 (2014), 060008.
Recommendations
Multiple light field rendering
GRAPHITE '03: Proceedings of the 1st international conference on Computer graphics and interactive techniques in Australasia and South East AsiaA light field is a 4D function describing the radiance across a boundary between the volume containing a scene, and the disjoint volume in which the eyepoint may be placed. Light field rendering is the process of rendering novel views of a scene ...
Rendering for an interactive 360° light field display
SIGGRAPH '07: ACM SIGGRAPH 2007 papersWe describe a set of rendering techniques for an autostereoscopic light field display able to present interactive 3D graphics to multiple simultaneous viewers 360 degrees around the display. The display consists of a high-speed video projector, a ...
Eye-Based Point Rendering for Dynamic Multiview Effects
Eye-based point rendering (EPR) can make multiview effects much more practical by adding eye (camera) buffer resolution efficiencies to improved view-independent rendering (iVIR). We demonstrate this very successfully by applying EPR to dynamic cube-...
Comments
Information & Contributors
Information
Published In
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 13 May 2024
Published in PACMCGIT Volume 7, Issue 1
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed
Data Availability
Supplemental files https://dl.acm.org/doi/10.1145/3651300#a19-gavane-supplemental.zip
Funding Sources
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 145Total Downloads
- Downloads (Last 12 months)145
- Downloads (Last 6 weeks)22
Reflects downloads up to 03 Mar 2025
Other Metrics
Citations
Cited By
View all- Yang ZLiu BSong YYi LXiong YZhang ZYu X(2024)DirectL: Efficient Radiance Fields Rendering for 3D Light Field DisplaysACM Transactions on Graphics10.1145/368789743:6(1-19)Online publication date: 19-Dec-2024
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in