Pagoda: Privacy Protection for Volumetric Video Streaming through Poisson Diffusion Model
Authors: 
Rui Lu, Lai Wei, Shuntao Zhu, Chuang Hu, Dan WangAuthors Info & Claims
Rui Lu, Lai Wei, Shuntao Zhu, Chuang Hu, Dan WangAuthors Info & ClaimsPages 1689 - 1697
Abstract
With the increasing popularity of 3D volumetric video applications, e.g., metaverse, AR/VR, etc., there is a growing need to protect users' privacy while sharing their experiences during streaming. In this paper, we show that the existing privacy-preserving approaches for dense point clouds suffer a massive computation cost and degrade the quality of the streaming experience. We design Pagoda, a new PrivAcy-preservinG VOlumetric ViDeo StreAming incorporating the MPEG V-PCC standard, which protects different domain privacy information of dense point cloud, and maintains high throughput. The core idea is to content-aware transform the privacy attribute information to the geometry domain and content-agnostic protect the geometry information by adding Poisson noise perturbations. These perturbations can be denoised through a Poisson diffusion probabilistic model we design to deploy on the cloud. Users only need to encrypt a small amount of high-sensitive information and achieve secure streaming. Our designs ensure the dense point clouds can be transmitted in high quality and the attackers cannot reconstruct the original one. We evaluate Pagoda using three volumetric video datasets. The results show that Pagoda outperforms existing privacy-preserving baselines for 75.6% protection capability improvement, 4.27 times streaming quality, and 26 times latency reduction.
References
[1]
Crypto Library 8.7. [n.,d.]. Free C Class Library of Cryptographic Schemes. https://www.cryptopp.com/
[2]
Evangelos Alexiou and Touradj Ebrahimi. 2020. Towards a point cloud structural similarity metric. In Proc. of IEEE International Conference on Multimedia and Expo Workshops (ICMEW'20). LA, CA, USA.
[3]
PyTorch C API. [n.,d.]. PyTorch C API documentation. https://pytorch.org/cppdocs/
[4]
Frank Cangialosi, Neil Agarwal, Venkat Arun, et al. 2022. Privid: Practical, Privacy-Preserving Video Analytics Queries. In Proc. of USENIX Symposium on Networked Systems Design and Implementation (NSDI'22). Renton, WA, USA.
[5]
Loop Charles, Cai Qin, et al. [n.,d.]. Microsoft Voxelized Upper Bodies - A Voxelized Point Cloud Dataset. https://plenodb.jpeg.org/pc/microsoft
[6]
Florinel-Alin Croitoru, Vlad Hondru, Radu Tudor Ionescu, and Mubarak Shah. 2023. Diffusion models in vision: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence (2023).
[7]
Sandeep Dsouza, Victor Bahl, Lixiang Ao, et al. 2020. Amadeus: Scalable, Privacy-Preserving Live Video Analytics. arXiv preprint:2011.05163 (2020).
[8]
Danillo Graziosi, Alexandre Zaghetto, Ali Tabatabai, and Vladyslav Zakharchenko. 2021. Synchronization of decoded frames before point cloud reconstruction. US Patent App. 17/066,434.
[9]
Meng-Hao Guo, Jun-Xiong Cai, Zheng-Ning Liu, Tai-Jiang Mu, et al. 2021. Pct: Point cloud transformer. Computational Visual Media, Vol. 7 (2021), 187--199.
[10]
Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, Vol. 33 (2020), 6840--6851.
[11]
Wei Hu, Zeqing Fu, and Zongming Guo. 2019. Local frequency interpretation and non-local self-similarity on graph for point cloud inpainting. IEEE Transactions on Image Processing, Vol. 28, 8 (2019), 4087--4100.
[12]
Glenn Jocher. 2020. YOLOv5 by Ultralytics. https://doi.org/10.5281/zenodo.3908559
[13]
Navid Ali Khan, Sarfraz Nawaz Brohi, and NZ Jhanjhi. 2020. UAV's applications, architecture, security issues and attack scenarios: A survey. In Proc. of Intelligent Computing and Innovation on Data Science (ICTIDS'19). Petaling Jaya, Malaysia.
[14]
Mate Kisantal, Zbigniew Wojna, Jakub Murawski, et al. 2019. Augmentation for small object detection. arXiv preprint:1902.07296 (2019).
[15]
Maja Krivokuca, Philip A Chou, and Patrick Savill. 2018. 8i voxelized surface light field (8iVSLF) dataset. (July 2018).
[16]
Qinya Li, Zhenzhe Zheng, Fan Wu, and Guihai Chen. 2020. Generative adversarial networks-based privacy-preserving 3D reconstruction. In Proc. of IEEE/ACM International Symposium on Quality of Service (IWQoS'20). Virtual Event.
[17]
Hongbin Liu, Jinyuan Jia, and Neil Zhenqiang Gong. 2021. Pointguard: Provably robust 3d point cloud classification. In Proc. of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR'21). Virtual Event.
[18]
Wei Liu, Dragomir Anguelov, Dumitru Erhan, et al. 2016. Ssd: Single shot multibox detector. In Proc. of European Conference on Computer Vision (ECCV'16). Amsterdam, Netherlands.
[19]
Juwei Lu et al. 2009. On conversion from color to gray-scale images for face detection. In Proc. of IEEE/CVF IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW'09). Miami, FL.
[20]
Rui Lu, Siping Shi, Dan Wang, Chuang Hu, and Bihai Zhang. 2022. Preva: Protecting Inference Privacy through Policy-based Video-frame Transformation. In Proc. of IEEE/ACM Symposium on Edge Computing (SEC'22). Seattle, WA, USA.
[21]
Shitong Luo and Wei Hu. 2021. Score-based point cloud denoising. In Proc. of the IEEE/CVF International Conference on Computer Vision (ICCV'21). Virtual Event.
[22]
Meta. [n.,d.]. Oculus Quest II. https://www.meta.com/quest/quest-pro/
[23]
Gabriel Meynet, Yana Nehmé, Julie Digne, et al. 2020. PCQM: A full-reference quality metric for colored 3D point clouds. In Proc. of IEEE International Conference on Quality of Multimedia Experience (QoMEX'20). Athlone, Ireland.
[24]
MPEGGroup. [n.,d.]. GitHub - MPEGGroup/mpeg-pcc-tmc2: Video codec based point cloud compression. https://github.com/MPEGGroup/mpeg-pcc-tmc2
[25]
Jianbing Ni, Kuan Zhang, and Athanasios V Vasilakos. 2020. Security and privacy for mobile edge caching: Challenges and solutions. IEEE Wireless Communications, Vol. 28, 3 (2020), 77--83.
[26]
Alexander Quinn Nichol and Prafulla Dhariwal. 2021. Improved denoising diffusion probabilistic models. In Proc. of International Conference on Machine Learning (ICML'21). Virtual.
[27]
Tribhuvanesh Orekondy, Mario Fritz, et al. 2018. Connecting pixels to privacy and utility: Automatic redaction of private information in images. In Proc. of IEEE/CFV International Conference on Computer Vision (CVPR'18). SLC,UT,USA.
[28]
Ozgur Oyman. 2021. Methods for timed metadata priority rank signaling for point clouds. US Patent App. 17/032,630.
[29]
F Ozge Unel, Burak O Ozkalayci, and Cevahir Cigla. 2019. The power of tiling for small object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW'19). Long Beach, CA.
[30]
Jiahao Pang and Gene Cheung. 2017. Graph Laplacian regularization for image denoising: Analysis in the continuous domain. IEEE Transactions on Image Processing, Vol. 26, 4 (2017), 1770--1785.
[31]
Adam Paszke, Sam Gross, et al. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems 32. Curran Associates, Inc., 8024--8035. http://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf
[32]
Rishabh Poddar, Ganesh Ananthanarayanan, Srinath Setty, Stavros Volos, et al. 2020. Visor: Privacy-preserving video analytics as a cloud service. In Proc. of USENIX Conference on Security Symposium (2020). Virtual Event.
[33]
Joseph Redmon and Ali Farhadi. 2018. Yolov3: An incremental improvement. arXiv preprint:1804.02767 (2018).
[34]
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolutional networks for biomedical image segmentation. In Proc. of Medical Image Computing and Computer-Assisted Intervention (MICCAI'15). Munich, Germany.
[35]
Rasool S Salman, Alaa K Farhan, and Ali Shakir. 2022. Lightweight modifications in the Advanced Encryption Standard (AES) for IoT applications: a comparative survey. In Proc. of IEEE International Conference on Computer Science and Software Engineering (CSASE'22). Duhok, Kurdistan Region, Iraq.
[36]
Sebastian Schwarz and Mika Pesonen. 2019. Real-time decoding and AR playback of the emerging MPEG video-based point cloud compression standard. Nokia Technologies; IBC: Helsinki, Finland (2019).
[37]
Sebastian Schwarz, Marius Preda, Vittorio Baroncini, Madhukar Budagavi, Pablo Cesar, Philip A Chou, Robert A Cohen, Maja Krivokuća, Zhu Li, et al. 2018. Emerging MPEG standards for point cloud compression. IEEE Journal on Emerging and Selected Topics in Circuits and Systems, Vol. 9, 1 (2018), 133--148.
[38]
Zhenbo Shi, Zhi Chen, Zhenbo Xu, Wei Yang, Zhidong Yu, and Liusheng Huang. 2022. Shape Prior Guided Attack: Sparser Perturbations on 3D Point Clouds. In Proc. of Conference on Artificial Intelligence (AAAI'22). Montreal, Canada.
[39]
Warit Sirichotedumrong, Takahiro Maekawa, Yuma Kinoshita, and Hitoshi Kiya. 2019. Privacy-preserving deep neural networks with pixel-based image encryption considering data augmentation in the encrypted domain. In Proc. of IEEE International Conference on Image Processing (ICIP'19). Taipei, Taiwan.
[40]
Jiaming Song, Chenlin Meng, and Stefano Ermon. 2020. Denoising diffusion implicit models. arXiv preprint: 2010.02502 (2020).
[41]
Qianru Sun, Ayush Tewari, Weipeng Xu, Mario Fritz, et al. 2018. A hybrid model for identity obfuscation by face replacement. In Proc. of the European conference on computer vision (ECCV'18). Munich, Germany.
[42]
Zhongze Tang, Xianglong Feng, Yi Xie, Huy Phan, et al. 2020. VVSec: Securing Volumetric Video Streaming via Benign Use of Adversarial Perturbation. In Proc. of ACM International Conference on Multimedia(MM'20). SEA, WA, USA.
[43]
Dong Tian et al. 2017. Geometric distortion metrics for point cloud compression. In Proc. of IEEE International Conference on Image Processing (ICIP'17). China.
[44]
Praneeth Vepakomma, Abhishek Singh, et al. 2020. NoPeek: Information leakage reduction to share activations in distributed deep learning. In Proc. of IEEE International Conference on Data Mining Workshops (ICDMW'20). Sorrento, Italy.
[45]
Nishant Vishwamitra et al. 2017. Blur vs. block: Investigating the effectiveness of privacy-enhancing obfuscation for images. In Proc. of IEEE/CVF International Conference on Computer Vision Workshops (CVPRW'17). Honolulu, HI, USA.
[46]
Ivaylo Vladimirov et al. 2022. Security and Privacy Protection Obstacles with 3D Reconstructed Models of People in Applications and the Metaverse: A Survey. In Proc. of International Scientific Conference on Information, Communication and Energy Systems and Technologies (ICEST'22). Ni?, Serbia.
[47]
Zhou Wang, Eero P Simoncelli, and Alan C Bovik. 2003. Multiscale structural similarity for image quality assessment. In Proc. of IEEE Asilomar Conference on Signals, Systems & Computers (ACSSC'03). Pacific Grove, CA, USA.
[48]
Xingxing Wei, Huanqian Yan, and Bo Li. 2022. Sparse black-box video attack with reinforcement learning. International Journal of Computer Vision, Vol. 130, 6 (2022), 1459--1473.
[49]
Xingxing Wei, Jun Zhu, Sha Yuan, and Hang Su. 2019. Sparse adversarial perturbations for videos. In Proc. of AAAI'19. Honolulu, HI, USA.
[50]
Yutong Xie, Minne Yuan, Bin Dong, and Quanzheng Li. 2023. Diffusion Model for Generative Image Denoising. arXiv preprint:2302.02398 (2023).
[51]
Jinrui Xing, Hui Yuan, Chen Chen, and Tian Guo. 2022. Wiener Filter-Based Point Cloud Adaptive Denoising for Video-based Point Cloud Compression. In Proc. of International Workshop on Advances in Point Cloud Compression, Processing and Analysis (APCCPA'22). Lisbon,Portugal.
[52]
Qi Yang, Zhan Ma, Yiling Xu, Zhu Li, and Jun Sun. 2020. Inferring point cloud quality via graph similarity. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 44, 6 (2020), 3015--3029.
[53]
Xu Yi, Lu Yao, and Wen Ziyu. 2017. Owlii Dynamic human mesh sequence dataset.
[54]
Yikuan Yu et al. 2020. Point Encoder GAN: A deep learning model for 3D point cloud inpainting. Neurocomputing, Vol. 384 (2020), 192--199.
[55]
Xiaohui Zeng et al. 2022. LION: Latent Point Diffusion Models for 3D Shape Generation. In Proc. of Advances in Neural Information Processing Systems (NeurIPS'22). New Orleans, Louisiana, USA.
[56]
Bihai Zhang, Siping Shi, Dan Wang, and Chuang Hu. 2022. EPC: a video analytics system with efficient edge-side privacy control. In Proc. of Workshop on Mobility in the Evolving Internet Architecture (MobiArch'22). Sydney, Australia.
Index Terms
- Pagoda: Privacy Protection for Volumetric Video Streaming through Poisson Diffusion Model
Recommendations
Anonymizing Streaming Data for Privacy Protection
ICDE '08: Proceedings of the 2008 IEEE 24th International Conference on Data EngineeringIn many applications, transaction data arrive in the form of high speed data streams. These data contain a lot of information about customers, not just transactions, and thus have to be carefully managed to protect customers' privacy. This paper ...
Privacy-Preserving Data Publishing Based on De-clustering
ICIS '08: Proceedings of the Seventh IEEE/ACIS International Conference on Computer and Information Science (icis 2008)In recent years, privacy preservation has become a serious concern in publication of personal data because of the wide availability of personal data. In the literature, we know that the degree of privacy protection is really determined by the number of ...
Location Recommendation with Privacy Protection
ISMSI '19: Proceedings of the 2019 3rd International Conference on Intelligent Systems, Metaheuristics & Swarm IntelligenceWith the development of Internet technology, users pay more and more attention to the privacy of personal location data. In order to cover up the user's original check-in data information and prevent attackers from using the user's friend relationship ...
Comments
Information & Contributors
Information
Published In
October 2023
9913 pages
ISBN:9798400701085
DOI:10.1145/3581783
- General Chairs:
- Abdulmotaleb El Saddik,
- Tao Mei,
- Rita Cucchiara,
- Program Chairs:
- Marco Bertini,
- Diana Patricia Tobon Vallejo,
- Pradeep K. Atrey,
- M. Shamim Hossain
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 27 October 2023
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- Innovation and Technology Commission
- CRF
- Fundamental Research Funds for the Central Universities
- Innovation and Technology Fund
- Research Grants Council General Research Fund
Conference
MM '23
Sponsor:
MM '23: The 31st ACM International Conference on Multimedia
October 29 - November 3, 2023
Ottawa ON, Canada
Acceptance Rates
Overall Acceptance Rate 2,145 of 8,556 submissions, 25%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 272Total Downloads
- Downloads (Last 12 months)143
- Downloads (Last 6 weeks)9
Reflects downloads up to 05 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in