Ruitao Chen
ABSTRACT
High dynamic range (HDR) omnidirectional image system can provide users with an immersive visual experience, but its coding, transmission and visualization processes will cause corresponding coding distortion, tone mapping distortion and mixed distortion, thereby reducing the quality of HDR omnidirectional image (HOI), and affecting the quality of user experience. Different from ordinary 2D images, omnidirectional images are usually viewed using head mounted display (HMD). Therefore, this paper proposes an HOI quality assessment method based on viewport feature learning, which includes a preprocessing module, feature extraction module and quality regression module. Firstly, in order to be consistent with what is observed on the HMD, viewport images are generated from HOIs as input images by the viewport sampling. Afterwards, they are input in parallel to feature extraction modules. Considering the distortion aliasing of images in channels and spaces, a triplet attention mechanism is used to capture joint features on spaces and channels. Finally, considering the interaction between different viewports, the quality regression module aggregates the features of different viewports to obtain the final quality score. Experimental results show that the proposed method achieves excellent performance on an HDR omnidirectional image database.
- Abderrezzaq Sendjasni, Mohamed-Chaker Larabi. (2022, October). Transfer Learning from Vision Transformers or ConvNets for 360-Degree Images Quality Assessmentƒ. In 2022 IEEE International Conference on Image Processing (ICIP) (pp. 4133-4137). IEEE. https://doi.org/10.1109/ICIP46576.2022.9897480Google Scholar
Cross Ref
- Xin Huang, Qi Zhang, Ying Feng, Hongdong Li, Xuan Wang, Qing Wang l. (2022, June). Hdr-nerf: High dynamic range neural radiance fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 18398-18408). https://doi.org/10.1109/CVPR52688.2022.01785Google ScholarCross Ref
- Guangcong Wang, Yinuo Yang, Chen Chang Loy, Ziwei Liu. (2022, October). Stylelight: Hdr panorama generation for lighting estimation and editing. In European Conference on Computer Vision (pp. 477-492). https://doi.org/10.1007/978-3-031-19784-0_28Google ScholarDigital Library
- Giljoo Nam, Haebom Lee , Sungsoo Oh, Min H. Kim. (2015). Measuring color defects in flat panel displays using HDR imaging and appearance modeling. IEEE Transactions on Instrumentation and Measurement, 65(2), 297-304. https://doi.org/10.1109/TIM.2015.2485341Google ScholarCross Ref
- Pengfei Chen, Leida Li, Xinfeng Zhang, Shanshe Wang, Allen Tan. (2019). Blind quality index for tone-mapped images based on luminance partition. Pattern Recognition, 89, 108-118. https://doi.org/10.1016/j.patcog.2019.01.010Google ScholarCross Ref
- Yang Song, Gangyi Jiang, Hao Jiang, Mei Yu, Feng Shao, Zongju Peng. (2017, September). A new tone-mapped image quality assessment approach for high dynamic range imaging system. In 2017 IEEE International Conference on Image Processing (ICIP) (pp. 1012-1016). IEEE. https://doi.org/10.1109/ICIP.2017.8296434Google ScholarDigital Library
- Guanghui Yue, Chunping Hou, Ke Gu, Shasha Mao, Wenjun Zhang. (2017). Biologically inspired blind quality assessment of tone-mapped images. IEEE Transactions on Industrial Electronics, 65(3), 2525-2536. https://doi.org/10.1109/TIE.2017.2739708Google ScholarCross Ref
- Hao Jiang, Gangyi Jiang, Mei Yu, Yun Zhang, You Yang, Zongju Peng, Fen Chen, Qingbo Zhang. (2021). Cubemap-based perception-driven blind quality assessment for 360-degree images. IEEE Transactions on Image Processing, 30, 2364-2377. https://doi.org/10.1109/TIP.2021.3052073Google ScholarDigital Library
- Miska M. Hannuksela, Ye-Kui Wang, (2021). An overview of omnidirectional media format (OMAF). Proceedings of the IEEE, 109(9), 1590-1606. https://doi.org/10.1109/JPROC.2021.3063544Google ScholarCross Ref
- Wei Sun, Xiongkuo Min, Guangtao Zhai, Ke Gu, Huiyu Duan, Siwei Ma. (2019). MC360IQA: A multi-channel CNN for blind 360-degree image quality assessment. IEEE Journal of Selected Topics in Signal Processing, 14(1), 64-77. https://doi.org/10.1109/JSTSP.2019.2955024Google ScholarCross Ref
- Yumeng Xia, Yongfang Wang, Ye Peng. (2019, December). Blind panoramic image quality assessment via the asymmetric mechanism of human brain. In 2019 IEEE Visual Communications and Image Processing (VCIP) (pp. 1-4). IEEE. https://doi.org/10.1109/VCIP47243.2019.8965887Google ScholarCross Ref
- Xuelei Zheng, Gangyi Jiang, Mei Yu, Hao Jiang. (2020). Segmented spherical projection-based blind omnidirectional image quality assessment. IEEE Access, 8, 31647-31659. https://doi.org/10.1109/ACCESS.2020.2972158Google ScholarCross Ref
- Abderrezzaq Sendjasni, Mohamed-Chaker Larabi. (2022, July). SAL-360IQA: A saliency weighted patch-based cnn model for 360-degree images quality assessment. In 2022 IEEE International Conference on Multimedia and Expo Workshops (ICMEW) (pp. 1-6). IEEE. https://doi.org/10.1109/ICMEW56448.2022.9859468Google ScholarCross Ref
- Jiahua Xu, Wei Zhou, Zhibo Chen. (2020). Blind omnidirectional image quality assessment with viewport oriented graph convolutional networks. IEEE Transactions on Circuits and Systems for Video Technology, 31(5), 1724-1737. https://doi.org/10.1109/TCSVT.2020.3015186Google ScholarDigital Library
- Ke Gu, Shiqi Wang, Guangtao Zhai, Siwei Ma, Xiaokang Yang, Siwei Lin, Wenjun Zhang, Wen Gao. (2016). Blind quality assessment of tone-mapped images via analysis of information, naturalness, and structure. IEEE Transactions on Multimedia, 18(3), 432-443. https://doi.org/10.1109/TMM.2016.2518868Google ScholarDigital Library
- Debarati Kundu, Deepti Ghadiyaram, Alan C. Bovik, and Brian L. Evans. (2017). No-reference quality assessment of tone-mapped HDR pictures. IEEE Transactions on Image Processing, 26(6), 2957-2971. https://doi.org/10.1109/TIP.2017.2685941Google ScholarDigital Library
- Biwei Chi, Mei Yu, Gangyi Jiang, Zhouyan He, Zongju Peng, Fen Chen. (2020). Blind tone mapped image quality assessment with image segmentation and visual perception. Journal of Visual Communication and Image Representation, 67, 102752. https://doi.org/10.1016/j.jvcir.2020.102752Google ScholarDigital Library
- Liuyan Cao, Gangyi Jiang, Zhidi Jiang, Mei Yu, Yubin Qi, Yo-song Ho. (2021). Quality measurement for high dynamic range omnidirectional image systems. IEEE Transactions on Instrumentation and Measurement, 70, 1-15. https://doi.org/10.1109/TIM.2021.3093940Google ScholarCross Ref
- Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun. (2016, June). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778). https://doi.org/10.1109/CVPR.2016.90Google ScholarCross Ref
- Rajeev Ranjan, Vishal M. Patel, Rama Chellappa. (2017). Hyperface: A deep multi-task learning framework for face detection, landmark localization, pose estimation, and gender recognition. IEEE transactions on pattern analysis and machine intelligence, 41(1), 121-135. https://doi.org/10.1109/TPAMI.2017.2781233Google ScholarDigital Library
- Diganta Misra, Trikay Nalamada, Ajay Uppili Arasanipalai, Qibin Hou (2021, January). Rotate to attend: Convolutional triplet attention module. In Proceedings of the IEEE/CVF winter conference on applications of computer vision (pp. 3139-3148). https://doi.org/10.1109/WACV48630.2021.00318Google ScholarCross Ref
- Yubin Qi, Gangyi Jiang, Mei Yu, Yun Zhang, Yo-Sung Ho. (2020). Viewport perception based blind stereoscopic omnidirectional image quality assessment. IEEE Transactions on Circuits and Systems for Video Technology, 31(10), 3926-3941. https://doi.org/10.1109/TCSVT.2020.3043349Google ScholarCross Ref
- Peter J. Huber(2004). Robust statistics (Vol. 523). John Wiley & Sons.Google Scholar
- Diederik P. Kingma, Jimmy Ba (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. https://doi.org/10.48550/arXiv.1412.6980Google ScholarCross Ref
Index Terms
- Visual Quality Assessment of HDR Omnidirectional Image System Based on Viewport Feature Learning
Recommendations
Objective HDR image quality assessment
Although there is a lot of literature about generating HDR images, very limited research has been conducted on the issue of HDR image quality based on people's feeling and preferences. The goal of this paper is to identify an objective quality ...
Quality assessment for real out-of-focus blurred images
Our method can accurately evaluate the quality of the out-of-focus blurred images.Our method is based on GM and PC-based structure extraction and saliency detection.Our method is not only effective but also efficient. Images are vulnerable to different ...
Omnidirectional image processing using geodesic metric
ICIP'09: Proceedings of the 16th IEEE international conference on Image processingDue to distorsions of catadioptric sensors, omnidirectional images can not be treated as classical images. If the equivalence between central catadioptric images and spherical images is now well known and used, spherical analysis often leads to complex ...
Comments