skip to main content
10.1145/3388536.3407878acmconferencesArticle/Chapter ViewAbstractPublication PagessiggraphConference Proceedingsconference-collections
abstract

DeepView Immersive Light Field Video

Published: 17 August 2020 Publication History

Abstract

This Immersive Pavilion installation introduces our new system for capturing, reconstructing, compressing, and rendering light field video content. By leveraging DeepView, a recently introduced view synthesis algorithm, our system can reconstruct challenging scenes with view-dependent reflections, semi-transparent surfaces, and near-field objects as close as 34 cm to the surface of our 46 camera capture rig. Improving upon past light field video systems that required specialized storage and graphics hardware for playback, our compressed videos can be rendered in a web browser or on mobile VR headsets while being streamed over a gigabit network connection. This makes ours the first system to encode high quality light field video at sufficiently low bandwidth for internet streaming.

Supplementary Material

VTT File (3388536.3407878.vtt)
MP4 File (3388536.3407878.mp4)
Presentation video

References

[1]
Michael Broxton, John Flynn, Ryan Overbeck, Daniel Erickson, Peter Hedman, Matthew DuVall, Jason Dourgarian, Jay Busch, Matt Whalen, and Paul Debevec. 2020. Immersive Light Field Video with a Layered Mesh Representation. 39, 4 (2020), 86:1–86:15.
[2]
John Flynn, Michael Broxton, Paul Debevec, Matthew DuVall, Graham Fyffe, Ryan Overbeck, Noah Snavely, and Richard Tucker. 2019. DeepView: View Synthesis With Learned Gradient Descent. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2367–2376.
[3]
Ben Mildenhall, Pratul P. Srinivasan, Rodrigo Ortiz-Cayon, Nima Khademi Kalantari, Ravi Ramamoorthi, Ren Ng, and Abhishek Kar. 2019. Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelines. ACM Trans. Graph. 38, 4, Article 29 (July 2019), 14 pages. https://doi.org/10.1145/3306346.3322980
[4]
Tinghui Zhou, Richard Tucker, John Flynn, Graham Fyffe, and Noah Snavely. 2018. Stereo Magnification: Learning View Synthesis Using Multiplane Images. ACM Trans. Graph. 37, 4, Article 65 (July 2018), 12 pages. https://doi.org/10.1145/3197517.3201323

Cited By

View all
  • (2024)Beyond Photometric Consistency: Geometry-Based Occlusion-Aware Unsupervised Light Field Disparity EstimationIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2023.328905635:11(15660-15674)Online publication date: Nov-2024
  • (2024)OmniLocalRF: Omnidirectional Local Radiance Fields from Dynamic Videos2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.00656(6871-6880)Online publication date: 16-Jun-2024
  • (2023)LiTrix: A Lightweight Live Light Field Video Scheme for Metaverse Stereoscopic ApplicationsIEEE Internet of Things Magazine10.1109/IOTM.001.22001886:2(137-142)Online publication date: Jun-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SIGGRAPH '20: ACM SIGGRAPH 2020 Immersive Pavilion
August 2020
36 pages
ISBN:9781450379687
DOI:10.1145/3388536
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 August 2020

Check for updates

Author Tags

  1. augmented reality
  2. light field
  3. view synthesis
  4. virtual reality
  5. volumetric imaging

Qualifiers

  • Abstract
  • Research
  • Refereed limited

Conference

SIGGRAPH '20
Sponsor:

Acceptance Rates

Overall Acceptance Rate 1,822 of 8,601 submissions, 21%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)7
  • Downloads (Last 6 weeks)1
Reflects downloads up to 19 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Beyond Photometric Consistency: Geometry-Based Occlusion-Aware Unsupervised Light Field Disparity EstimationIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2023.328905635:11(15660-15674)Online publication date: Nov-2024
  • (2024)OmniLocalRF: Omnidirectional Local Radiance Fields from Dynamic Videos2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.00656(6871-6880)Online publication date: 16-Jun-2024
  • (2023)LiTrix: A Lightweight Live Light Field Video Scheme for Metaverse Stereoscopic ApplicationsIEEE Internet of Things Magazine10.1109/IOTM.001.22001886:2(137-142)Online publication date: Jun-2023
  • (2023)Balanced Spherical Grid for Egocentric View Synthesis2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52729.2023.01592(12663-12673)Online publication date: Jun-2023
  • (2023)OpenDIBR: Open Real-Time Depth-Image-Based renderer of light field videos for VRMultimedia Tools and Applications10.1007/s11042-023-16250-883:9(25797-25815)Online publication date: 23-Aug-2023
  • (2022)SILVRProceedings of the 13th ACM Multimedia Systems Conference10.1145/3524273.3532890(221-226)Online publication date: 14-Jun-2022
  • (2022)A Portable Multiscopic Camera for Novel View and Time Synthesis in Dynamic Scenes2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)10.1109/IROS47612.2022.9982040(2409-2416)Online publication date: 23-Oct-2022
  • (2022)SADN: Learned Light Field Image Compression with Spatial-Angular DecorrelationICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP43922.2022.9747377(1870-1874)Online publication date: 23-May-2022
  • (2022)Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52688.2022.00539(5460-5469)Online publication date: Jun-2022
  • (2021)6DIVE: 6 Degrees-of-Freedom Immersive Video EditorFrontiers in Virtual Reality10.3389/frvir.2021.6768952Online publication date: 14-Jun-2021
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media