Cited By
View all- Yu DChen RLi XXiao MZhang GLiu Y(2023)A GPU-Enabled Real-Time Framework for Compressing and Rendering Volumetric VideosIEEE Transactions on Computers10.1109/TC.2023.334310473:3(789-800)Online publication date: 14-Dec-2023
patchVVC: A Real-time Compression Framework for Streaming Volumetric Videos
Pages 119 - 129
Abstract
Nowadays, volumetric video has emerged as an attractive multimedia application, which provides highly immersive watching experiences. However, streaming the volumetric video demands prohibitively high bandwidth. Thus, effectively compressing its underlying point cloud frames is essential to deploying the volumetric videos. The existing compression techniques are either 3D-based or 2D-based, but they still have drawbacks when being deployed in practice. The 2D-based methods compress the videos in an effective but slow manner, while the 3D-based methods feature high coding speeds but low compression ratios. In this paper, we propose patchVVC, a 3D-based compression framework that reaches both a high compression ratio and a real-time decoding speed. More importantly, patchVVC is designed based on point cloud patches, which makes it friendly to an field of view adaptive streaming system that further reduces the bandwidth demands. The evaluation shows patchVCC achieves the real-time decoding speed and the comparable compression ratios as the representative 2D-based scheme, V-PCC, in an FoV-adaptive streaming scenario.
References
[1]
Yasuhiro Aoki, Hunter Goforth, Rangaprasad Arun Srivatsan, and Simon Lucey. 2019. Pointnetlk: Robust & efficient point cloud registration using pointnet. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 7163--7172.
[2]
Paul J Besl and Neil D McKay. 1992. Method for Registration of 3-D Shapes. Sensor fusion IV: control paradigms and data structures 1611 (1992), 586--606.
[3]
Benjamin Bross, Ye-Kui Wang, Yan Ye, Shan Liu, Jianle Chen, Gary J. Sullivan, and Jens-Rainer Ohm. 2021. Overview of the Versatile Video Coding (VVC) Standard and its Applications. IEEE Transactions on Circuits and Systems for Video Technology 31, 10 (2021), 3736--3764.
[4]
Seonghwa Choi, Anh-Duc Nguyen, Jinwoo Kim, Sewoong Ahn, and Sanghoon Lee. 2019. Point Cloud Deformation for Single Image 3d Reconstruction. In 2019 IEEE International Conference on Image Processing. 2379--2383.
[5]
Cisco. 2020. Cisco Annual Internet Report (2018--2023) White Paper. https://www.cisco.com/c/en/us/solutions/collateral/executive-perspectives/annual-internet-report/white-paper-c11-741490.html. Online; accessed 31 March 2022.
[6]
Yann Collet and Murray Kucherawy. 2018. Zstandard Compression and the Application/Zstd Media Type. RFC 8478 (2018).
[7]
Ricardo L De Queiroz and Philip A Chou. 2016. Compression of 3D point clouds Using a Region-Adaptive Hierarchical Transform. IEEE Transactions on Image Processing 25, 8 (2016), 3947--3956.
[8]
Ricardo L de Queiroz and Philip A Chou. 2017. Motion-Compensated Compression of Dynamic Voxelized Point Clouds. IEEE Transactions on Image Processing 26, 8 (2017), 3886--3895.
[9]
Eugene d'Eon, Bob Harrison, Taos Myers, and Philip A Chou. 2017. 8i Voxelized Full Bodies-A Voxelized Point Cloud Dataset. ISO/IEC JTC1/SC29 Joint WG11/WG1 (MPEG/JPEG) input document WG11M40059/WG1M74006 7 (2017), 8.
[10]
Martin Ester, Hans-Peter Kriegel, Jörg Sander, and Xiaowei Xu. 1996. A Density-based Algorithm for Discovering Clusters in Large Spatial Databases with Noise. In The Second International Conference on Knowledge Discovery and Data Mining, Vol. 96. 226--231.
[11]
Pierre-Marie Gandoin and Olivier Devillers. 2002. Progressive Lossless Compression of Arbitrary Simplicial Complexes. ACM Transactions on Graphics 21, 3 (2002), 372--379.
[12]
Serhan Gül, Dimitri Podborski, Thomas Buchholz, Thomas Schierl, and Cornelius Hellge. 2020. Low-Latency Cloud-based Volumetric Video Streaming using Head Motion Prediction. In Proceedings of the 30th ACM Workshop on Network and Operating Systems Support for Digital Audio and Video. 27--33.
[13]
Serhan Gül, Dimitri Podborski, Jangwoo Son, Gurdeep Singh Bhullar, Thomas Buchholz, Thomas Schierl, and Cornelius Hellge. 2020. Cloud Rendering-based Volumetric Video Streaming System for Mixed Reality Services. In Proceedings of the 11th ACM Multimedia Systems Conference. 357--360.
[14]
Dorina hanou, Philip A Chou, and Pascal Frossard. 2016. Graph-based Compression of Dynamic 3D Point Cloud Sequences. IEEE Transactions on Image Processing 25, 4 (2016), 1765--1778.
[15]
Mohammad Hosseini and Christian Timmerer. 2018. Dynamic Adaptive Point Cloud Streaming. In Proceedings of the 23rd Packet Video Workshop. 25--30.
[16]
Yan Huang, Jingliang Peng, C-C Jay Kuo, and M Gopi. 2006. Octree-based Progressive Geometry Coding of Point Clouds. In Proceedings of the 3rd Eurographics. 103--110.
[17]
K Krishna and M Narasimha Murty. 1999. Genetic K-means Algorithm. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 29, 3 (1999), 433--439.
[18]
Kyungjin Lee, Juheon Yi, Youngki Lee, Sunghyun Choi, and Young Min Kim. 2020. GROOT: A Real-time Sstreaming System of High-Fidelity Volumetric Videos. In Proceedings of the 26th Annual International Conference on Mobile Computing and Networking. 1--14.
[19]
Li Li, Zhu Li, Shan Liu, and Houqiang Li. 2019. Occupancy-Map-Based Rate Distortion Optimization for Video-based Point Cloud Compression. In 2019 IEEE International Conference on Image Processing. 3167--3171.
[20]
Li Li, Zhu Li, Vladyslav Zakharchenko, Jianle Chen, and Houqiang Li. 2019. Advanced 3D Motion Prediction for Video-based Dynamic Point Cloud Compression. IEEE Transactions on Image Processing 29 (2019), 289--302.
[21]
Rufael Mekuria, Kees Blom, and Pablo Cesar. 2016. Design, Implementation, and Evaluation of a Point Cloud Codec for Tele-Immersive Video. IEEE Transactions on Circuits and Systems for Video Technology 27, 4 (2016), 828--842.
[22]
Rufael Mekuria and Pablo Cesar. 2016. MP3DG-PCC, Open Source Software Framework for Implementation and Evaluation of Point Cloud Compression. In Proceedings of the 24th ACM International Conference on Multimedia. 1222--1226.
[23]
Jounsup Park, Philip A Chou, and Jenq-Neng Hwang. 2019. Rate-Utility Optimized Streaming of Volumetric Media for Augmented Reality. IEEE Journal on Emerging and Selected Topics in Circuits and Systems 9, 1 (2019), 149--162.
[24]
Feng Qian, Bo Han, Jarrell Pair, and Vijay Gopalakrishnan. 2019. Toward Practical Volumetric Video Streaming on Commodity Smartphones. In Proceedings of the 20th International Workshop on Mobile Computing Systems and Applications. 135--140.
[25]
ITUR Rec. 1995. BT 601: Studio encoding parameters of digital television for standard 4: 3 and wide-screen 16: 9 aspect ratios. ITU-R Rec. BT 656 (1995).
[26]
Ruwen Schnabel and Reinhard Klein. 2006. Octree-based Point-Cloud Compression. In Proceedings of the 3rd Eurographics. 111--120.
[27]
Sebastian Schwarz, Gaëlle Martin-Cocher, David Flynn, and Madhukar Budagavi. 2018. Common Test Conditions for Point Cloud Compression. Document ISO/IEC JTC1/SC29/WG11 w17766, Ljubljana, Slovenia (2018).
[28]
Sebastian Schwarz, Marius Preda, Vittorio Baroncini, Madhukar Budagavi, Pablo Cesar, Philip A Chou, Robert A Cohen, Maja Krivokuća, Sébastien Lasserre, Zhu Li, et al. 2018. Emerging MPEG standards for Point Cloud Compression. IEEE Journal on Emerging and Selected Topics in Circuits and Systems 9, 1 (2018), 133--148.
[29]
Aleksandr Segal, Dirk Haehnel, and Sebastian Thrun. 2009. Generalized-icp. 435.
[30]
Jacopo Serafin and Giorgio Grisetti. 2015. NICP: Dense Normal based Point Cloud Registration. In 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems. 742--749.
[31]
Yiting Shao, Zhaobin Zhang, Zhu Li, Kui Fan, and Ge Li. 2017. Attribute Compression of 3D Point Clouds Using Laplacian Sparsity Optimized Graph Transform. In 2017 IEEE Visual Communications and Image Processing. 1--4.
[32]
Subramanyam Shishir, Hanjalic Alan, and Cesar Pablo. 2020. User Centered Adaptive Streaming of Dynamic Point Clouds with Low Complexity Tiling. In Proceedings of the 28th ACM International Conference on Multimedia. 3669--3677.
[33]
Hubert P. H. Shum, Edmond S. L. Ho, Yang Jiang, and Shu Takagi. 2013. Real-Time Posture Reconstruction for Microsoft Kinect. IEEE Transactions on Cybernetics 43, 5 (2013), 1357--1369.
[34]
Gary J Sullivan, Jens-Rainer Ohm, Woo-Jin Han, and Thomas Wiegand. 2012. Overview of the High Efficiency Video Coding (HEVC) Standard. IEEE Transactions on Circuits and Systems for Video Technology 22, 12 (2012), 1649--1668.
[35]
Jeroen van der Hooft, Tim Wauters, Filip De Turck, Christian Timmerer, and Hermann Hellwagner. 2019. Towards 6DoF HTTP Adaptive Streaming Through Point Cloud Compression. In Proceedings of the 27th ACM International Conference on Multimedia. 2405--2413.
[36]
Yue Wang and Justin M Solomon. 2019. Deep Closest Point: Learning Representations for Point Cloud Registration. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 3523--3532.
[37]
Thomas Wiegand, Gary J Sullivan, Gisle Bjontegaard, and Ajay Luthra. 2003. Overview of the H. 264/AVC Video Coding Standard. IEEE Transactions on Circuits and Systems for Video Technology 13, 7 (2003), 560--576.
[38]
Yiqun Xu, Wei Hu, Shanshe Wang, Xinfeng Zhang, Shiqi Wang, Siwei Ma, Zongming Guo, and Wen Gao. 2020. Predictive Generalized Graph Fourier Transform for Attribute Compression of Dynamic Point clouds. IEEE Transactions on Circuits and Systems for Video Technology 31, 5 (2020), 1968--1982.
[39]
Xiangyu Yue, Bichen Wu, Sanjit A. Seshia, Kurt Keutzer, and Alberto L. Sangiovanni-Vincentelli. 2018. A LiDAR Point Cloud Generator: From a Virtual World to Autonomous Driving. In Proceedings of the 2018 ACM on International Conference on Multimedia Retrieval. 458--464.
[40]
Anlan Zhang, Chendong Wang, Bo Han, and Feng Qian. 2022. YuZu : Neural-Enhanced Volumetric Video Streaming. In 19th USENIX Symposium on Networked Systems Design and Implementation. 137--154.
[41]
Cha Zhang, Dinei Florencio, and Charles Loop. 2014. Point Cloud Attribute Compression with Graph Transform. In 2014 IEEE International Conference on Image Processing. 2066--2070.
[42]
Wenjie Zhu, Zhan Ma, Yiling Xu, Li Li, and Zhu Li. 2021. View-Dependent Dynamic Point Cloud Compression. IEEE Transactions on Circuits and Systems for Video Technology 31, 2 (2021), 765--781.
Index Terms
- patchVVC: A Real-time Compression Framework for Streaming Volumetric Videos
Recommendations
Enabling Low Bit-Rate MPEG V-PCC-encoded Volumetric Video Streaming with 3D Sub-sampling
MMSys '23: Proceedings of the 14th ACM Multimedia Systems ConferenceMPEG's Video-based Point Cloud Compression (V-PCC) is a recent new standard for volumetric video compression. By mapping a 3D dynamic point cloud to a 2D image sequence, V-PCC can rely on state-of-the-art video codecs to achieve high compression rate ...
Volumetric Video Compression Through Neural-based Representation
MMVE '24: Proceedings of the 16th International Workshop on Immersive Mixed and Virtual Environment SystemsVolumetric video offers immersive exploration and interaction in 3D space, revolutionizing visual storytelling. Recently, Neural Radiance Fields (NeRF) have emerged as a powerful neural-based technique for generating high-fidelity images from 3D scenes. ...
SSIM-based error-resilient rate-distortion optimization of H.264/AVC video coding for wireless streaming
The SSIM-based rate-distortion optimization (RDO) has been verified to be an effective tool for H.264/AVC to promote the perceptual video coding performance. However, the current SSIM-based RDO is not efficient for improving the perceptual quality of ...
Comments
Information & Contributors
Information
Published In
June 2023
495 pages
ISBN:9798400701481
DOI:10.1145/3587819
- Co-chairs:
- Shervin Shirmohammadi,
- Mohamed Hefeeda,
- Program Co-chairs:
- Roger Zimmermann,
- Carsten Griwodz,
- Mea Wang
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
In-Cooperation
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 08 June 2023
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- National Natural Science Foundation of China
- Natural Science Foundation of Shandong Province, China
Conference
MMSys '23
Sponsor:
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 415Total Downloads
- Downloads (Last 12 months)177
- Downloads (Last 6 weeks)16
Reflects downloads up to 28 Feb 2025
Other Metrics
Citations
Cited By
View all- Yu DChen RLi XXiao MZhang GLiu Y(2023)A GPU-Enabled Real-Time Framework for Compressing and Rendering Volumetric VideosIEEE Transactions on Computers10.1109/TC.2023.334310473:3(789-800)Online publication date: 14-Dec-2023
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in