Rui Lu
ABSTRACT
With the increasing popularity of 3D volumetric video applications, e.g., metaverse, AR/VR, etc., there is a growing need to protect users' privacy while sharing their experiences during streaming. In this paper, we show that the existing privacy-preserving approaches for dense point clouds suffer a massive computation cost and degrade the quality of the streaming experience. We design Pagoda, a new PrivAcy-preservinG VOlumetric ViDeo StreAming incorporating the MPEG V-PCC standard, which protects different domain privacy information of dense point cloud, and maintains high throughput. The core idea is to content-aware transform the privacy attribute information to the geometry domain and content-agnostic protect the geometry information by adding Poisson noise perturbations. These perturbations can be denoised through a Poisson diffusion probabilistic model we design to deploy on the cloud. Users only need to encrypt a small amount of high-sensitive information and achieve secure streaming. Our designs ensure the dense point clouds can be transmitted in high quality and the attackers cannot reconstruct the original one. We evaluate Pagoda using three volumetric video datasets. The results show that Pagoda outperforms existing privacy-preserving baselines for 75.6% protection capability improvement, 4.27 times streaming quality, and 26 times latency reduction.
- Crypto Library 8.7. [n.,d.]. Free C Class Library of Cryptographic Schemes. https://www.cryptopp.com/Google Scholar
- Evangelos Alexiou and Touradj Ebrahimi. 2020. Towards a point cloud structural similarity metric. In Proc. of IEEE International Conference on Multimedia and Expo Workshops (ICMEW'20). LA, CA, USA.Google ScholarCross Ref
- PyTorch C API. [n.,d.]. PyTorch C API documentation. https://pytorch.org/cppdocs/Google Scholar
- Frank Cangialosi, Neil Agarwal, Venkat Arun, et al. 2022. Privid: Practical, Privacy-Preserving Video Analytics Queries. In Proc. of USENIX Symposium on Networked Systems Design and Implementation (NSDI'22). Renton, WA, USA.Google Scholar
- Loop Charles, Cai Qin, et al. [n.,d.]. Microsoft Voxelized Upper Bodies - A Voxelized Point Cloud Dataset. https://plenodb.jpeg.org/pc/microsoftGoogle Scholar
- Florinel-Alin Croitoru, Vlad Hondru, Radu Tudor Ionescu, and Mubarak Shah. 2023. Diffusion models in vision: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence (2023).Google ScholarDigital Library
- Sandeep Dsouza, Victor Bahl, Lixiang Ao, et al. 2020. Amadeus: Scalable, Privacy-Preserving Live Video Analytics. arXiv preprint:2011.05163 (2020).Google Scholar
- Danillo Graziosi, Alexandre Zaghetto, Ali Tabatabai, and Vladyslav Zakharchenko. 2021. Synchronization of decoded frames before point cloud reconstruction. US Patent App. 17/066,434.Google Scholar
- Meng-Hao Guo, Jun-Xiong Cai, Zheng-Ning Liu, Tai-Jiang Mu, et al. 2021. Pct: Point cloud transformer. Computational Visual Media, Vol. 7 (2021), 187--199.Google ScholarCross Ref
- Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, Vol. 33 (2020), 6840--6851.Google Scholar
- Wei Hu, Zeqing Fu, and Zongming Guo. 2019. Local frequency interpretation and non-local self-similarity on graph for point cloud inpainting. IEEE Transactions on Image Processing, Vol. 28, 8 (2019), 4087--4100.Google ScholarCross Ref
- Glenn Jocher. 2020. YOLOv5 by Ultralytics. https://doi.org/10.5281/zenodo.3908559Google ScholarCross Ref
- Navid Ali Khan, Sarfraz Nawaz Brohi, and NZ Jhanjhi. 2020. UAV's applications, architecture, security issues and attack scenarios: A survey. In Proc. of Intelligent Computing and Innovation on Data Science (ICTIDS'19). Petaling Jaya, Malaysia.Google ScholarCross Ref
- Mate Kisantal, Zbigniew Wojna, Jakub Murawski, et al. 2019. Augmentation for small object detection. arXiv preprint:1902.07296 (2019).Google Scholar
- Maja Krivokuca, Philip A Chou, and Patrick Savill. 2018. 8i voxelized surface light field (8iVSLF) dataset. (July 2018).Google Scholar
- Qinya Li, Zhenzhe Zheng, Fan Wu, and Guihai Chen. 2020. Generative adversarial networks-based privacy-preserving 3D reconstruction. In Proc. of IEEE/ACM International Symposium on Quality of Service (IWQoS'20). Virtual Event.Google ScholarCross Ref
- Hongbin Liu, Jinyuan Jia, and Neil Zhenqiang Gong. 2021. Pointguard: Provably robust 3d point cloud classification. In Proc. of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR'21). Virtual Event.Google ScholarCross Ref
- Wei Liu, Dragomir Anguelov, Dumitru Erhan, et al. 2016. Ssd: Single shot multibox detector. In Proc. of European Conference on Computer Vision (ECCV'16). Amsterdam, Netherlands.Google ScholarCross Ref
- Juwei Lu et al. 2009. On conversion from color to gray-scale images for face detection. In Proc. of IEEE/CVF IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW'09). Miami, FL.Google Scholar
- Rui Lu, Siping Shi, Dan Wang, Chuang Hu, and Bihai Zhang. 2022. Preva: Protecting Inference Privacy through Policy-based Video-frame Transformation. In Proc. of IEEE/ACM Symposium on Edge Computing (SEC'22). Seattle, WA, USA.Google ScholarCross Ref
- Shitong Luo and Wei Hu. 2021. Score-based point cloud denoising. In Proc. of the IEEE/CVF International Conference on Computer Vision (ICCV'21). Virtual Event.Google ScholarCross Ref
- Meta. [n.,d.]. Oculus Quest II. https://www.meta.com/quest/quest-pro/Google Scholar
- Gabriel Meynet, Yana Nehmé, Julie Digne, et al. 2020. PCQM: A full-reference quality metric for colored 3D point clouds. In Proc. of IEEE International Conference on Quality of Multimedia Experience (QoMEX'20). Athlone, Ireland.Google ScholarCross Ref
- MPEGGroup. [n.,d.]. GitHub - MPEGGroup/mpeg-pcc-tmc2: Video codec based point cloud compression. https://github.com/MPEGGroup/mpeg-pcc-tmc2Google Scholar
- Jianbing Ni, Kuan Zhang, and Athanasios V Vasilakos. 2020. Security and privacy for mobile edge caching: Challenges and solutions. IEEE Wireless Communications, Vol. 28, 3 (2020), 77--83.Google ScholarDigital Library
- Alexander Quinn Nichol and Prafulla Dhariwal. 2021. Improved denoising diffusion probabilistic models. In Proc. of International Conference on Machine Learning (ICML'21). Virtual.Google Scholar
- Tribhuvanesh Orekondy, Mario Fritz, et al. 2018. Connecting pixels to privacy and utility: Automatic redaction of private information in images. In Proc. of IEEE/CFV International Conference on Computer Vision (CVPR'18). SLC,UT,USA.Google ScholarCross Ref
- Ozgur Oyman. 2021. Methods for timed metadata priority rank signaling for point clouds. US Patent App. 17/032,630.Google Scholar
- F Ozge Unel, Burak O Ozkalayci, and Cevahir Cigla. 2019. The power of tiling for small object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW'19). Long Beach, CA.Google ScholarCross Ref
- Jiahao Pang and Gene Cheung. 2017. Graph Laplacian regularization for image denoising: Analysis in the continuous domain. IEEE Transactions on Image Processing, Vol. 26, 4 (2017), 1770--1785.Google ScholarDigital Library
- Adam Paszke, Sam Gross, et al. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems 32. Curran Associates, Inc., 8024--8035. http://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdfGoogle Scholar
- Rishabh Poddar, Ganesh Ananthanarayanan, Srinath Setty, Stavros Volos, et al. 2020. Visor: Privacy-preserving video analytics as a cloud service. In Proc. of USENIX Conference on Security Symposium (2020). Virtual Event.Google Scholar
- Joseph Redmon and Ali Farhadi. 2018. Yolov3: An incremental improvement. arXiv preprint:1804.02767 (2018).Google Scholar
- Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolutional networks for biomedical image segmentation. In Proc. of Medical Image Computing and Computer-Assisted Intervention (MICCAI'15). Munich, Germany.Google ScholarCross Ref
- Rasool S Salman, Alaa K Farhan, and Ali Shakir. 2022. Lightweight modifications in the Advanced Encryption Standard (AES) for IoT applications: a comparative survey. In Proc. of IEEE International Conference on Computer Science and Software Engineering (CSASE'22). Duhok, Kurdistan Region, Iraq.Google ScholarCross Ref
- Sebastian Schwarz and Mika Pesonen. 2019. Real-time decoding and AR playback of the emerging MPEG video-based point cloud compression standard. Nokia Technologies; IBC: Helsinki, Finland (2019).Google Scholar
- Sebastian Schwarz, Marius Preda, Vittorio Baroncini, Madhukar Budagavi, Pablo Cesar, Philip A Chou, Robert A Cohen, Maja Krivokuća, Zhu Li, et al. 2018. Emerging MPEG standards for point cloud compression. IEEE Journal on Emerging and Selected Topics in Circuits and Systems, Vol. 9, 1 (2018), 133--148.Google ScholarCross Ref
- Zhenbo Shi, Zhi Chen, Zhenbo Xu, Wei Yang, Zhidong Yu, and Liusheng Huang. 2022. Shape Prior Guided Attack: Sparser Perturbations on 3D Point Clouds. In Proc. of Conference on Artificial Intelligence (AAAI'22). Montreal, Canada.Google ScholarCross Ref
- Warit Sirichotedumrong, Takahiro Maekawa, Yuma Kinoshita, and Hitoshi Kiya. 2019. Privacy-preserving deep neural networks with pixel-based image encryption considering data augmentation in the encrypted domain. In Proc. of IEEE International Conference on Image Processing (ICIP'19). Taipei, Taiwan.Google ScholarCross Ref
- Jiaming Song, Chenlin Meng, and Stefano Ermon. 2020. Denoising diffusion implicit models. arXiv preprint: 2010.02502 (2020).Google Scholar
- Qianru Sun, Ayush Tewari, Weipeng Xu, Mario Fritz, et al. 2018. A hybrid model for identity obfuscation by face replacement. In Proc. of the European conference on computer vision (ECCV'18). Munich, Germany.Google ScholarDigital Library
- Zhongze Tang, Xianglong Feng, Yi Xie, Huy Phan, et al. 2020. VVSec: Securing Volumetric Video Streaming via Benign Use of Adversarial Perturbation. In Proc. of ACM International Conference on Multimedia(MM'20). SEA, WA, USA.Google ScholarDigital Library
- Dong Tian et al. 2017. Geometric distortion metrics for point cloud compression. In Proc. of IEEE International Conference on Image Processing (ICIP'17). China.Google Scholar
- Praneeth Vepakomma, Abhishek Singh, et al. 2020. NoPeek: Information leakage reduction to share activations in distributed deep learning. In Proc. of IEEE International Conference on Data Mining Workshops (ICDMW'20). Sorrento, Italy.Google ScholarCross Ref
- Nishant Vishwamitra et al. 2017. Blur vs. block: Investigating the effectiveness of privacy-enhancing obfuscation for images. In Proc. of IEEE/CVF International Conference on Computer Vision Workshops (CVPRW'17). Honolulu, HI, USA.Google Scholar
- Ivaylo Vladimirov et al. 2022. Security and Privacy Protection Obstacles with 3D Reconstructed Models of People in Applications and the Metaverse: A Survey. In Proc. of International Scientific Conference on Information, Communication and Energy Systems and Technologies (ICEST'22). Ni?, Serbia.Google Scholar
- Zhou Wang, Eero P Simoncelli, and Alan C Bovik. 2003. Multiscale structural similarity for image quality assessment. In Proc. of IEEE Asilomar Conference on Signals, Systems & Computers (ACSSC'03). Pacific Grove, CA, USA.Google ScholarCross Ref
- Xingxing Wei, Huanqian Yan, and Bo Li. 2022. Sparse black-box video attack with reinforcement learning. International Journal of Computer Vision, Vol. 130, 6 (2022), 1459--1473.Google ScholarDigital Library
- Xingxing Wei, Jun Zhu, Sha Yuan, and Hang Su. 2019. Sparse adversarial perturbations for videos. In Proc. of AAAI'19. Honolulu, HI, USA.Google ScholarDigital Library
- Yutong Xie, Minne Yuan, Bin Dong, and Quanzheng Li. 2023. Diffusion Model for Generative Image Denoising. arXiv preprint:2302.02398 (2023).Google Scholar
- Jinrui Xing, Hui Yuan, Chen Chen, and Tian Guo. 2022. Wiener Filter-Based Point Cloud Adaptive Denoising for Video-based Point Cloud Compression. In Proc. of International Workshop on Advances in Point Cloud Compression, Processing and Analysis (APCCPA'22). Lisbon,Portugal.Google ScholarDigital Library
- Qi Yang, Zhan Ma, Yiling Xu, Zhu Li, and Jun Sun. 2020. Inferring point cloud quality via graph similarity. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 44, 6 (2020), 3015--3029.Google ScholarCross Ref
- Xu Yi, Lu Yao, and Wen Ziyu. 2017. Owlii Dynamic human mesh sequence dataset.Google Scholar
- Yikuan Yu et al. 2020. Point Encoder GAN: A deep learning model for 3D point cloud inpainting. Neurocomputing, Vol. 384 (2020), 192--199.Google ScholarDigital Library
- Xiaohui Zeng et al. 2022. LION: Latent Point Diffusion Models for 3D Shape Generation. In Proc. of Advances in Neural Information Processing Systems (NeurIPS'22). New Orleans, Louisiana, USA.Google Scholar
- Bihai Zhang, Siping Shi, Dan Wang, and Chuang Hu. 2022. EPC: a video analytics system with efficient edge-side privacy control. In Proc. of Workshop on Mobility in the Evolving Internet Architecture (MobiArch'22). Sydney, Australia.Google ScholarDigital Library
Index Terms
- Pagoda: Privacy Protection for Volumetric Video Streaming through Poisson Diffusion Model
Recommendations
Anonymizing Streaming Data for Privacy Protection
ICDE '08: Proceedings of the 2008 IEEE 24th International Conference on Data EngineeringIn many applications, transaction data arrive in the form of high speed data streams. These data contain a lot of information about customers, not just transactions, and thus have to be carefully managed to protect customers' privacy. This paper ...
Privacy-Preserving Data Publishing Based on De-clustering
ICIS '08: Proceedings of the Seventh IEEE/ACIS International Conference on Computer and Information Science (icis 2008)In recent years, privacy preservation has become a serious concern in publication of personal data because of the wide availability of personal data. In the literature, we know that the degree of privacy protection is really determined by the number of ...
Location Recommendation with Privacy Protection
ISMSI '19: Proceedings of the 2019 3rd International Conference on Intelligent Systems, Metaheuristics & Swarm IntelligenceWith the development of Internet technology, users pay more and more attention to the privacy of personal location data. In order to cover up the user's original check-in data information and prevent attackers from using the user's friend relationship ...
Comments