skip to main content
10.1145/3641521.3664412acmconferencesArticle/Chapter ViewAbstractPublication PagessiggraphConference Proceedingsconference-collections
abstract

Intractable Live Free-Viewpoint Video with Haptic Feedback

Published: 17 July 2024 Publication History

Abstract

We present a novel interactable free-viewpoint video (FVV), which generates photo-realistic and editable volumetric content with high degree of freedom. On the one hand, an enhanced visual hull guided neural representation with higher performance is proposed, an-easy-to-use sparse multi-camera capture system is used, as well as a high performance VH-NeRF is proposed for fast generation of photo-realistic FVV results for live streaming. On the other hand, for monocular panorama video input, high-precision depth and coarse dynamic mesh are calculated, which are used for novel view synthesis of right perspective views with motion parallax on the fly. For better interaction with high realism, dual-channel stereohaptics is implemented and attached on VR headsets to obtain haptic feedback. Last, our FVV solution can do effective compression and transmission on both multi-perspective videos and panorama video with depth data, as well as real-time rendering on consumer-grade hardware. To the best of our knowledge, our work is the first interactable FVV solution with high visual quality and real-time haptic feedback. It is established that the user can do intuitive manipulation for immersive experiences in VR/AR applications.

Supplemental Material

MOV File
Accompanied Video for Paper-Interactable Live Free-Viewpoint Video with Haptic Feedback

References

[1]
Ali Israr, Siyan Zhao, Kyna McIntosh, Zachary Schwemler, Adam Fritz, John Mars, Job Bedford, Christian Frisson, Ivan Huerta, Maggie Kosek, Babis Koniaris, and Kenny Mitchell. 2016. Stereohaptics: a haptic interaction toolkit for tangible virtual experiences. SIGGRAPH 2016 Studio (2016).
[2]
Shaohui Jiao, Yuzhong Chen, Zhaoliang Liu, Danying Wang, Wen Zhou, Li Zhang, and Yue Wang. 2023. Photo-Realistic Streamable Free-Viewpoint Video. In ACM SIGGRAPH 2023 Posters, SIGGRAPH 2023, Los Angeles, CA, USA, August 6-10, 2023. ACM.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SIGGRAPH '24: ACM SIGGRAPH 2024 Immersive Pavilion
July 2024
39 pages
ISBN:9798400705274
DOI:10.1145/3641521
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 July 2024

Check for updates

Qualifiers

  • Abstract
  • Research
  • Refereed limited

Conference

SIGGRAPH '24
Sponsor:

Acceptance Rates

Overall Acceptance Rate 1,822 of 8,601 submissions, 21%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 167
    Total Downloads
  • Downloads (Last 12 months)167
  • Downloads (Last 6 weeks)15
Reflects downloads up to 02 Mar 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media