No-reference Omnidirectional Image Quality Assessment via Equirectangular Convolution and Hierarchically Fusion
Authors: 
Junhao Chen, Yuzhen NiuAuthors Info & Claims
Junhao Chen, Yuzhen NiuAuthors Info & ClaimsPages 643 - 648
Abstract
The number of omnidirectional images and videos, which have gained much attention due to their large viewing angles and interactive viewing experience, has grown significantly. Thus omnidirectional image quality assessment (OIQA) has become more critical. In this paper, we propose an Equi-StairNet, an end-to-end no-reference (NR) image quality assessment method that fuses high and low-level visual features and predicts directly on the equirectangular projection (ERP). The Equirectangular convolution kernels (EquiConvs) were adopted, which take into account the geometric deformations of the ERP. An Equirectangular convolution kernel samples the points on the spherical surface by mapping the points of the spherical points to the ERP. Through depth-wise and point-wise convolutions, a staircase structure network is adopted to hierarchically fuse feature mappings from the stage. Experimental results show that the proposed method outperforms the state-of-the-art OIQA methods.
References
[1]
Fernandez-Labrador C, Facil J M, Perez-Yus A, . Corners for Layout: End-to-end layout recovery from 360 images[J]. IEEE Robotics and Automation Letters, 2020, 5(2): 1255-1262.
[2]
Sun Y, Lu A, Yu L. Weighted-to-spherically-uniform quality evaluation for omnidirectional video[J]. IEEE Signal Processing Letters, 2017, 24(9): 1408-1412.
[3]
Sun W, Min X, Zhai G, MC360IQA: A multi-channel CNN for blind 360-degree image quality assessment[J]. IEEE Journal of Selected Topics in Signal Processing, 2019, 14(1): 64-77.
[4]
Xu J, Zhou W, Chen Z. Blind omnidirectional image quality assessment with viewport oriented graph convolutional networks[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2020, 31(5): 1724-1737.
[5]
Sendjasni A, Larabi M C, Cheikh F A. Perceptually-weighted CNN for 360-degree image quality assessment using visual scan-path and JND[C]//2021 IEEE International Conference on Image Processing (ICIP). IEEE, 2021: 1439-1443.
[6]
Jiang H, Jiang G, Yu M, Cubemap-based perception-driven blind quality assessment for 360-degree images[J]. IEEE Transactions on Image Processing, 2021, 30: 2364-2377.
[7]
Dai J, Qi H, Xiong Y, Deformable convolutional networks[C]//Proceedings of the IEEE International Conference on Computer Vision (ICCV). 2017: 764-773.
[8]
Sun W, Min X, Zhai G, Blind quality assessment for in-the-wild images via hierarchical feature fusion and iterative mixed database training[J]. arXiv preprint arXiv:2105.14550, 2021.
[9]
Sun W, Wang T, Min X, Deep learning based full-reference and no-reference quality assessment models for compressed ugc videos[C]//2021 IEEE International Conference on Multimedia & Expo Workshops (ICMEW). IEEE, 2021: 1-6.
[10]
He K, Zhang X, Ren S, Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2016: 770-778.
[11]
Sun W, Gu K, Ma S, A large-scale compressed 360-degree spherical image database: From subjective quality evaluation to objective model comparison[C]//2018 IEEE 20th International Workshop on Multimedia Signal Processing (MMSP). IEEE, 2018: 1-6.
[12]
Howard A G, Zhu M, Chen B, Mobilenets: Efficient convolutional neural networks for mobile vision applications[J]. arXiv preprint arXiv:1704.04861, 2017.
[13]
Kingma D P, Ba J. Adam: A method for stochastic optimization[J]. arXiv preprint arXiv:1412.6980, 2014.
[14]
Zheng X, Jiang G, Yu M, Segmented spherical projection-based blind omnidirectional image quality assessment[J]. IEEE Access, 2020, 8: 31647-31659.
[15]
Wang Z, Bovik A C, Sheikh H R, Image quality assessment: from error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4): 600-612.
[16]
Yu M, Lakshman H, Girod B. A framework to evaluate omnidirectional video coding schemes[C]//2015 IEEE International Symposium on Mixed and Augmented Reality. 2015: 31-36.
[17]
Zakharchenko V, Choi K P, Park J H. Quality metric for spherical panoramic video[C]//Optics and Photonics for Information Processing X. SPIE, 2016, 9970: 57-65.
[18]
Mittal A, Moorthy A K, Bovik A C. No-reference image quality assessment in the spatial domain[J]. IEEE Transactions on Image Processing, 2012, 21(12): 4695-4708.
[19]
Xue W, Zhang L, Mou X. Learning without human scores for blind image quality assessment[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2013: 995-1002.
[20]
Min X, Gu K, Zhai G, Blind quality assessment based on pseudo-reference image[J]. IEEE Transactions on Multimedia, 2017, 20(8): 2049-2062.
[21]
Ma K, Liu W, Liu T, dipIQ: Blind image quality assessment by learning-to-rank discriminable image pairs[J]. IEEE Transactions on Image Processing, 2017, 26(8): 3951-3964.
[22]
Liu Y, Yu H, Huang B, Blind omnidirectional image quality assessment based on structure and natural features[J]. IEEE Transactions on Instrumentation and Measurement, 2021, 70: 1-11.
Index Terms
- No-reference Omnidirectional Image Quality Assessment via Equirectangular Convolution and Hierarchically Fusion
Recommendations
Kurtosis-based no-reference quality assessment of JPEG2000 images
No-reference (NR) image quality assessment (QA) presumes no prior knowledge of reference (distortion-free) images and seeks to quantitatively predict visual quality solely from the distorted images. We develop kurtosis-based NR quality measures for ...
No-reference image quality assessment based on gradient histogram response
A NR image quality assessment based on gradient histogram response (GHR) is proposed.A test image is preprocessed to produce a noise image object and a blur image object.GHR is the gradient histogram variation of an image object under a local ...
Non-distortion-specific no-reference image quality assessment
Over the last two decades, there has been a surge of interest in the research of image quality assessment due to its wide applicability to many domains. In general, the aim of image quality assessment algorithms is to evaluate the perceptual quality of ...
Comments
Information & Contributors
Information
Published In
September 2022
1221 pages
ISBN:9781450396899
DOI:10.1145/3573942
Copyright © 2022 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 16 May 2023
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
Conference
AIPR 2022
AIPR 2022: 2022 5th International Conference on Artificial Intelligence and Pattern Recognition
September 23 - 25, 2022
Xiamen, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 45Total Downloads
- Downloads (Last 12 months)18
- Downloads (Last 6 weeks)3
Reflects downloads up to 01 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format