skip to main content
10.1145/3652212.3652227acmconferencesArticle/Chapter ViewAbstractPublication PagesmmsysConference Proceedingsconference-collections
research-article
Open access

A Comparative Study of K-Planes vs. V-PCC for 6-DoF Volumetric Video Representation

Published: 15 April 2024 Publication History

Abstract

With NeRF, neural scene representations have gained increased popularity in recent years. To date, many models have been designed to represent dynamic scenes that can be explored in 6 degrees-of-freedom (6-DoF) in immersive applications such as virtual reality (VR), augmented reality (AR), and mixed reality (MR). In this paper, we aim to evaluate how newer neural representations of 6-DoF video compare with more-traditional point cloud-based representations in terms of their representation and transmission efficiency. We design a new methodology for fair comparison between K-Planes, anew dynamic neural scene representation model, and video-based point cloud compression (V-PCC). We conduct extensive experiments using three datasets with a total of 11 sequences with different characteristics. Results show that the current K-Planes models excel for moderately dynamic content, but struggle with highly dynamic scenes. In addition, in emulated volumetric data capture scenarios, the recorded point cloud data can be highly noisy, and the visual quality of views rendered by trained K-Planes models are significantly better than V-PCC.

References

[1]
2018. VMAF: The Journey Continues. https://medium.com/netflix-techblog/vmaf-the-journey-continues-44b51ee9ed12.
[2]
2019. zip - package and compress (archive) files. https://manpages.ubuntu.com/manpages/focal/man1/zip.1.html.
[3]
2020. Bjontegaard_metric. https://github.com/Anserw/Bjontegaard_metric.
[4]
2023. bjontegaard. https://pypi.org/project/bjontegaard/.
[5]
2023. Blender 3.5. https://www.blender.org/.
[6]
2023. Free | Amily Animations | Blender Demo. https://www.blender.org/download/demo-files/.
[7]
2023. Free | Piggy Animations | Vfx Grace. https://blendermarket.com/products/piggy-animations-vfx-grace.
[8]
2023. TORCH.AMP. https://pytorch.org/docs/stable/amp.html.
[9]
2024. Draco. https://google.github.io/draco/.
[10]
Gisle Bjontegaard. 2001. Calculation of average PSNR differences between RD-curves. VCEG-M33 (2001).
[11]
Anpei Chen, Zexiang Xu, Andreas Geiger, Jingyi Yu, and Hao Su. 2022. TensoRF: Tensorial Radiance Fields. In European Conference on Computer Vision (ECCV).
[12]
Samuel Rhys Cox, May Lim, and Wei Tsang Ooi. 2023. VOLVQAD: An MPEG V-PCC Volumetric Video Quality Assessment Dataset. In Proceedings of the 14th Conference on ACM Multimedia Systems. 357--362.
[13]
Eugene d'Eon, Bob Harrison, Taos Myers, and Philip A Chou. 2017. 8i voxelized full bodies-a voxelized point cloud dataset. ISO/IEC JTC1/SC29 Joint WG11/WG1 (MPEG/JPEG) input document WG11M40059/WG1M74006 7, 8 (2017), 11.
[14]
Sara Fridovich-Keil, Giacomo Meanti, Frederik Rahbæk Warburg, Benjamin Recht, and Angjoo Kanazawa. 2023. K-Planes for Radiance Fields in Space, Time, and Appearance. arXiv:2301.10241 [cs.CV]
[15]
D Graziosi, O Nakagami, S Kuma, A Zaghetto, T Suzuki, and A Tabatabai. 2020. An overview of ongoing point cloud compression standardization activities: Video-based (V-PCC) and geometry-based (G-PCC). APSIPA Transactions on Signal and Information Processing 9 (2020), e13.
[16]
Dan Grois, Detlev Marpe, Amit Mulayoff, Benaya Itzhaky, and Ofer Hadar. 2013. Performance comparison of h.265/mpeg-hevc, vp9, and h.264/mpeg-avc encoders. In 2013 Picture Coding Symposium (PCS). IEEE, 394--397.
[17]
Simon NB Gunkel, Rick Hindriks, Karim M El Assal, Hans M Stokking, Sylvie Dijkstra-Soudarissanane, Frank ter Haar, and Omar Niamut. 2021. VRComm: an end-to-end web system for real-time photorealistic social VR communication. In Proceedings of the 12th ACM Multimedia Systems Conference. 65--79.
[18]
Bo Han, Yu Liu, and Feng Qian. 2020. ViVo: Visibility-aware mobile volumetric video streaming. In Proceedings of the 26th annual international conference on mobile computing and networking. 1--13.
[19]
Henry Haugsten Hansen, Sayed Muchallil, Carsten Griwodz, Vetle Sillerud, and Fredrik Johanssen. 2020. Dense lidar point clouds from room-scale scans. In Proceedings of the 11th ACM Multimedia Systems Conference. 88--98.
[20]
Branislav Jenco. 2022. Virtual LiDAR error models in point cloud compression. Master's thesis.
[21]
Tianye Li, Mira Slavcheva, Michael Zollhoefer, Simon Green, Christoph Lassner, Changil Kim, Tanner Schmidt, Steven Lovegrove, Michael Goesele, Richard Newcombe, et al. 2022. Neural 3d video synthesis from multi-view video. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5521--5531.
[22]
Rufael Mekuria, Kees Blom, and Pablo Cesar. 2016. Design, implementation, and evaluation of a point cloud codec for tele-immersive video. IEEE Transactions on Circuits and Systems for Video Technology 27, 4 (2016), 828--842.
[23]
Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. 2020. NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. In ECCV.
[24]
Albert Pumarola, Enric Corona, Gerard Pons-Moll, and Francesc Moreno-Noguer. 2021. D-nerf: Neural radiance fields for dynamic scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 10318--10327.
[25]
Sebastian Schwarz, Gaëlle Martin-Cocher, David Flynn, and Madhukar Budagavi. 2018. Common test conditions for point cloud compression. Document ISO/IEC JTC1/SC29/WG11 w17766, Ljubljana, Slovenia (2018).
[26]
Sebastian Schwarz, Marius Preda, Vittorio Baroncini, Madhukar Budagavi, Pablo Cesar, Philip A Chou, Robert A Cohen, Maja Krivokuća, Sébastien Lasserre, Zhu Li, et al. 2018. Emerging MPEG standards for point cloud compression. IEEE Journal on Emerging and Selected Topics in Circuits and Systems 9, 1 (2018), 133--148.
[27]
Shishir Subramanyam, Irene Viola, Jack Jansen, Evangelos Alexiou, Alan Hanjalic, and Pablo Cesar. 2022. Evaluating the impact of tiled user-adaptive real-time point cloud streaming on vr remote communication. In Proceedings of the 30th ACM International Conference on Multimedia. 3094--3103.
[28]
Jeroen Van Der Hooft, Tim Wauters, Filip De Turck, Christian Timmerer, and Hermann Hellwagner. 2019. Towards 6dof http adaptive streaming through point cloud compression. In Proceedings of the 27th ACM International Conference on Multimedia. 2405--2413.
[29]
Irene Viola, Jack Jansen, Shishir Subramanyam, Ignacio Reimat, and Pablo Cesar. 2023. VR2Gather: A Collaborative, Social Virtual Reality System for Adaptive, Multiparty Real-Time Communication. IEEE MultiMedia 30, 2 (2023), 48--59.
[30]
Thomas Wiegand, Gary J Sullivan, Gisle Bjontegaard, and Ajay Luthra. 2003. Overview of the H. 264/AVC video coding standard. IEEE Transactions on circuits and systems for video technology 13, 7 (2003), 560--576.
[31]
Emin Zerman, Cagri Ozcinar, Pan Gao, and Aljosa Smolic. 2020. Textured mesh vs coloured point cloud: A subjective study for volumetric video compression. In Twelfth International Conference on Quality of Multimedia Experience (QoMEX).

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
MMVE '24: Proceedings of the 16th International Workshop on Immersive Mixed and Virtual Environment Systems
April 2024
101 pages
ISBN:9798400706189
DOI:10.1145/3652212
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 April 2024

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. 6-DoF
  2. neural scene representations
  3. point cloud
  4. volumetric videos

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Funding Sources

Conference

MMSys '24
Sponsor:

Acceptance Rates

Overall Acceptance Rate 26 of 44 submissions, 59%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 157
    Total Downloads
  • Downloads (Last 12 months)157
  • Downloads (Last 6 weeks)27
Reflects downloads up to 17 Jan 2025

Other Metrics

Citations

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media