Yujie Zhang
ABSTRACT
To address the point cloud quality assessment (PCQA) problem, GraphSIM was proposed via jointly considering geometrical and color features, which shows compelling performance in multiple distortion detection. However, GraphSIM does not take into account the mutiscale characteristics of human perception. In this paper, we propose a multiscale PCQA model, called Multiscale Graph Similarity (MS-GraphSIM), that can better predict human subjective perception. First, exploring the multiscale processing method used in image processing, we introduce a multiscale representation of point clouds based on graph signal processing. Second, we extend GraphSIM into multiscale version based on the proposed multiscale representation. Specifically, MS-GraphSIM constructs a multiscale representation for each local patch extracted from the reference point cloud or the distorted point cloud, and then fuses GraphSIM at different scales to obtain an overall quality score. Experiment results demonstrate that the proposed MS-GraphSIM outperforms the state-of-the-art PCQA metrics over two fairly large and independent databases. Ablation studies further prove the proposed MS-GraphSIM is robust to different model hyperparameter settings. The code is available at https://github.com/zyj1318053/MS_GraphSIM.
- Edward H Adelson, Charles H Anderson, James R Bergen, Peter J Burt, and Joan M Ogden. Pyramid methods in image processing. RCA engineer, 29(6):33--41, 1984.Google Scholar
- E. Alexiou and T. Ebrahimi. Point cloud quality assessment metric based on angular similarity. In 2018 IEEE International Conference on Multimedia and Expo (ICME), pages 1--6, 2018.Google ScholarCross Ref
- Evangelos Alexiou and Touradj Ebrahimi. Exploiting user interactivity in quality assessment of point cloud imaging. In 2019 Eleventh International Conference on Quality of Multimedia Experience (QoMEX), pages 1--6, 2019.Google ScholarCross Ref
- Evangelos Alexiou and Touradj Ebrahimi. Towards a point cloud structural similarity metric. In 2020 IEEE International Conference on Multimedia Expo Workshops (ICMEW), pages 1--6, 2020.Google ScholarCross Ref
- Evangelos Alexiou, Irene Viola, Tomás M. Borges, Tiago A. Fonseca, Ricardo L. de Queiroz, and Touradj Ebrahimi. A comprehensive study of the rate-distortion performance in mpeg point cloud compression. APSIPA Transactions on Signal and Information Processing, 8:e27, 2019.Google ScholarCross Ref
- R. Q. Charles, H. Su, M. Kaichun, and L. J. Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 77--85, 2017.Google ScholarCross Ref
- S. Chen, D. Tian, C. Feng, A. Vetro, and J.. Fast resampling of three-dimensional point clouds via graphs. IEEE Transactions on Signal Processing, 66(3):666--681, 2018.Google ScholarCross Ref
- Rafael Diniz, Pedro Garcia Freitas, and Mylène C. Q. Farias. Towards a point cloud quality assessment model using local binary patterns. In 2020 Twelfth International Conference on Quality of Multimedia Experience (QoMEX), pages 1--6, 2020.Google ScholarCross Ref
- Rafael Diniz, Pedro Garcia Freitas, and Mylène C.Q. Farias. Multi-distance point cloud quality assessment. In 2020 IEEE International Conference on Image Processing (ICIP), pages 3443--3447, 2020.Google ScholarCross Ref
- D. Groux-Leclet, J. Lentremy, G. Caron, and E. Mouaddib. Efficient reproduction of heritage buildings: a new way to exploit 3d point cloud slicing : An example with the amiens gothic cathedral. In 2018 3rd Digital Heritage International Congress (DigitalHERITAGE) held jointly with 2018 24th International Conference on Virtual Systems Multimedia (VSMM 2018), pages 1--4, 2018.Google ScholarCross Ref
- W. Hu, X. Gao, G. Cheung, and Z. Guo. Feature graph learning for 3d point cloud denoising. IEEE Transactions on Signal Processing, 68:2841--2856, 2020.Google ScholarCross Ref
- Wei Hu, Jiahao Pang, Xianming Liu, Dong Tian, Chia-Wen Lin, and Anthony Vetro. Graph signal processing for geometric data and beyond: Theory and applications, arXiv prepeint arXiv: 2008.01918, 2021.Google Scholar
- G. Jang, Y. Shin, J. Park, and M. Baeg. Real-time point-cloud data transmission for teleoperation using h.264/avc. In 2014 IEEE International Symposium on Safety, Security, and Rescue Robotics (2014), pages 1--6, 2014.Google ScholarCross Ref
- Alireza Javaheri, Catarina Brites, Fernando Pereira, and João Ascenso. A generalized hausdorff distance based quality metric for point cloud geometry. In 2020 Twelfth International Conference on Quality of Multimedia Experience (QoMEX), pages 1--6, 2020.Google ScholarCross Ref
- Alireza Javaheri, Catarina Brites, Fernando Pereira, and João Ascenso. Mahalanobis based point to distribution metric for point cloud geometry quality evaluation. IEEE Signal Processing Letters, 27:1350--1354, 2020.Google ScholarCross Ref
- Alireza Javaheri, Catarina Brites, Fernando Manuel Bernardo Pereira, and Joao M. Ascenso. Point cloud rendering after coding: Impacts on subjective and objective quality. IEEE Transactions on Multimedia, pages 1--1, 2020.Google ScholarCross Ref
- Q. Liu, H. Yuan, J. Hou, R. Hamzaoui, and H. Su. Model-based joint bit allocation between geometry and color for video-based 3d point cloud compression. IEEE Transactions on Multimedia, pages 1--1, 2020.Google Scholar
- Yipeng Liu, Qi Yang, Yiling Xu, and Le Yang. Point cloud quality assessment: Large-scale dataset construction and learning-based no-reference approach, arXiv prepeint arXiv: 2012.11895, 2020.Google Scholar
- G. Meynet, J. Digne, and G. Lavoué. Pc-msdm: A quality metric for 3d point clouds. In 2019 Eleventh International Conference on Quality of Multimedia Experience (QoMEX), pages 1--3, 2019.Google ScholarCross Ref
- G. Meynet, Y. Nehmé, J. Digne, and G. Lavoué. Pcqm: A full-reference quality metric for colored 3d point clouds. In 2020 Twelfth International Conference on Quality of Multimedia Experience (QoMEX), pages 1--6, 2020.Google ScholarCross Ref
- Carsten Moenning and Neil A. Dodgson. Fast Marching farthest point sampling. Technical Report UCAM-CL-TR-562, University of Cambridge, Computer Laboratory, April 2003.Google Scholar
- Charles R. Qi, Li Yi, Hao Su, and Leonidas J. Guibas. Pointnet+: Deep hierarchical feature learning on point sets in a metric space, arXiv prepeint arXiv: 1706.02413, 2017.Google Scholar
- S. Schwarz, M. Preda, V. Baroncini, M. Budagavi, P. Cesar, P. A. Chou, R. A. Cohen, M. Krivoku?a, S. Lasserre, Z. Li, J. Llach, K. Mammou, R. Mekuria, O. Nakagami, E. Siahaan, A. Tabatabai, A. M. Tourapis, and V. Zakharchenko. Emerging mpeg standards for point cloud compression. IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 9(1):133--148, 2019.Google ScholarCross Ref
- D. I. Shuman, S. K. Narang, P. Frossard, A. Ortega, and P. Vandergheynst. The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains. IEEE Signal Processing Magazine, 30(3):83--98, 2013.Google ScholarCross Ref
- Aljoscha Smolic. An overview of 3d video and free viewpoint video. In Xiaoyi Jiang and Nicolai Petkov, editors, Computer Analysis of Images and Patterns, pages 1--8, Berlin, Heidelberg, 2009. Springer Berlin Heidelberg. Google ScholarDigital Library
- K. Sugimoto, R. A. Cohen, D. Tian, and A. Vetro. Trends in efficient representation of 3d point clouds. In 2017 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), pages 364--369, 2017.Google ScholarCross Ref
- Irene Viola and Pablo Cesar. A reduced reference metric for visual quality evaluation of point cloud contents. IEEE Signal Processing Letters, 27:1660--1664, 2020.Google ScholarCross Ref
- Irene Viola, Shishir Subramanyam, and Pablo Cesar. A color-based objective quality metric for point cloud contents. In 2020 Twelfth International Conference on Quality of Multimedia Experience (QoMEX), pages 1--6, 2020.Google ScholarCross Ref
- Z. Wang, E. P. Simoncelli, and A. C. Bovik. Multiscale structural similarity for image quality assessment. In The Thrity-Seventh Asilomar Conference on Signals, Systems Computers, 2003, volume 2, pages 1398--1402 Vol.2, 2003.Google ScholarCross Ref
- Q. Yang, H. Chen, Z. Ma, Y. Xu, R. Tang, and J. Sun. Predicting the perceptual quality of point cloud: A 3d-to-2d projection-based exploration. IEEE Transactions on Multimedia, pages 1--1, 2020.Google ScholarCross Ref
- Q. Yang, Z. Ma, Y. Xu, Z. Li, and J. Sun. Inferring point cloud quality via graph similarity. IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 1--1, 2020.Google Scholar
- Qi Yang, Siheng Chen, Yiling Xu, Jun Sun, M. Salman Asif, and Zhan Ma. Point cloud distortion quantification based on potential energy for human and machine perception, arXiv prepeint arXiv: 2103.02850, 2021.Google Scholar
- Zhou Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):600--612, 2004. Google ScholarDigital Library
Index Terms
- MS-GraphSIM: Inferring Point Cloud Quality via Multiscale Graph Similarity
Recommendations
Multiscale representation and compression of 3-D point data
A compact representation scheme is presented for 3-D point data. To describe underlying surface from raw point samples, we dyadically divide a 3-D domain enclosing whole points. Then, local points in each cube are approximated by a plane patch, yielding ...
Shape classification via image-based multiscale description
We introduce a new multiscale Fourier-based object description in 2-D space using a low-pass Gaussian filter (LPGF) and a high-pass Gaussian filter (HPGF), separately. Using the LPGF at different scales (standard deviation) represents the inner and ...
A robust 3D point cloud watermarking method based on the graph Fourier transform
AbstractMany modern applications make use of 3D modeling and / or reconstruction of complex objects, such as historical monuments and entire urban centers. One of the most common representations of these 3D models is by point clouds, which are a dense set ...
Comments