Abstract
Image quality assessment aims to design effective models to automatically predict the perceptual quality score of a given image that is consistent with human cognition. In this paper, we propose a novel end-to-end multi-level fusion based deep convolutional neural network for full-reference image quality assessment (FR-IQA), codenamed MF-IQA. In MF-IQA, we first extract features with the help of edge feature fusion for both distorted images and the corresponding reference images. Afterwards, we apply multi-level feature fusion to evaluate a number of local quality indices, and then they would be pooled into a global quality score. With the proposed multi-level fusion and edge feature fusion strategy, the input images and the corresponding feature maps can be better learned and thus help produce more accurate and meaningful visual perceptual predictions. The experimental results and statistical comparisons on three IQA datasets demonstrate that our framework achieves the state-of-the-art prediction accuracy in contrast to most existing algorithms.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Bosse, S., Maniry, D., Müller, K.R., Wiegand, T., Samek, W.: Deep neural networks for no-reference and full-reference image quality assessment. IEEE TIP (2017)
Bosse, S., Siekmann, M., Samek, W., Wiegand, T.: A perceptually relevant shearlet-based adaptation of the PSNR. In: IEEE ICIP (2017)
Canny, J.: A computational approach to edge detection. IEEE PAMI 8, 679–698 (1986)
Cao, J., et al.: DO-CONV: depthwise over-parameterized convolutional layer. arXiv preprint arXiv:2006.12030 (2020)
Ding, K., Ma, K., Wang, S., Simoncelli, E.P.: Image quality assessment: unifying structure and texture similarity. arXiv preprint arXiv:2004.07728 (2020)
Gao, F., Wang, Y., Li, P., Tan, M., Yu, J., Zhu, Y.: Deepsim: deep similarity for image quality assessment. Neurocomputing (2017)
Girod, B.: What’s wrong with mean-squared error? Digital images and human vision (1993)
Larson, E.C., Chandler, D.M.: Most apparent distortion: full-reference image quality assessment and the role of strategy. SPIE IS&T J. Electron. Imaging (2010)
Ma, X., Jiang, X.: Multimedia image quality assessment based on deep feature extraction. Multimedia Tools Appl. 79(47), 35209–35220 (2019). https://doi.org/10.1007/s11042-019-7571-y
Ponomarenko, N., et al.: Color image database tid2013: peculiarities and preliminary results. In: IEEE EUVIP (2013)
Prashnani, E., Cai, H., Mostofi, Y., Sen, P.: Pieapp: perceptual image-error assessment through pairwise preference. In: IEEE CVPR (2018)
Sara, U., Akter, M., Uddin, M.S.: Image quality assessment through FSIM, SSIM, MSE and PSNR-a comparative study. JCC 07, 8–18 (2019)
Sheikh, H.R., Bovik, A.C.: Image information and visual quality. IEEE TIP 15, 430–444 (2006)
Sheikh, H.R., Sabir, M.F., Bovik, A.C.: A statistical evaluation of recent full reference image quality assessment algorithms. IEEE TIP 15, 3440–3451 (2006)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE TIP 13, 600–612 (2004)
Wang, Z., Li, Q.: Information content weighting for perceptual image quality assessment. IEEE TIP 20, 1185–1198 (2010)
Xue, W., Zhang, L., Mou, X., Bovik, A.C.: Gradient magnitude similarity deviation: a highly efficient perceptual image quality index. IEEE TIP 23, 684–695 (2013)
Zhang, L., Li, H.: Sr-sim: A fast and high performance IQA index based on spectral residual. In: IEEE ICIP (2012)
Zhang, L., Zhang, L., Mou, X., Zhang, D.: FSIM: a feature similarity index for image quality assessment. IEEE TIP 20, 2378–2386 (2011)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Guo, Q., Wen, J. (2021). Multi-level Fusion Based Deep Convolutional Network for Image Quality Assessment. In: Del Bimbo, A., et al. Pattern Recognition. ICPR International Workshops and Challenges. ICPR 2021. Lecture Notes in Computer Science(), vol 12666. Springer, Cham. https://doi.org/10.1007/978-3-030-68780-9_51
Download citation
DOI: https://doi.org/10.1007/978-3-030-68780-9_51
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-68779-3
Online ISBN: 978-3-030-68780-9
eBook Packages: Computer ScienceComputer Science (R0)
