Skip to main content

Advertisement

SpringerLink
Log in

Real distorted images quality assessment based on multi-layer visual perception mechanism and high-level semantics

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Most of the existing image quality assessment (IQA) methods are directed to artificially synthesized distorted images, in which the types and characteristics of distortion are different from those in the real world. In view of the fact that the existing non-reference IQA methods can not accurately evaluate the quality of the real distortion image, combined with the theoretical analysis of multi-layer visual perception mechanism, we propose a real image distortion IQA method based on image underlying features and high-level semantics. Considering non-linear hierarchical structure of human visual perception, firstly, k-means clustering algorithm is performed according to the underlying feature indexs of the image so that the used image database can be divided into several groups, which aims to improve the accuracy of predicted quality score. Secondly, the deep convolutional neural network (DCNN) is used to extract the first-grade high-level semantic features in each group. Then, second-grade high-level semantic features that can provide better representation of image features are obtained by performing multiple statistical functions on first-grade high-level semantics. Besides, we establish an effective high-capacity regressor with high-level semantics and subjective mean opinion scores (MOS) values of the human eyes. The experimental results show that the proposed model on the KonIQ-10 k image database can predict the quality score effectively and achieve a high consistency with the corresponding MOS value, which is helpful for the subsequent image enhancement.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Bianco S, Celona L, Napoletano P, Schettini R (2016) On the use of deep learning for blind image quality assessment. Signal Image & Video Processing 3:1–8

    Google Scholar 

  2. Bosse S, Maniry D, Wiegand T, Samek W (2016) A deep neural network for image quality assessment. IEEE International Conference on Image Processing.

  3. Cai W, Wei Z (2020) PiiGAN: generative adversarial networks for pluralistic image Inpainting. IEEE Access 8:48451–48463

    Article  Google Scholar 

  4. Dosovitskiy A, Brox T Generating images with perceptual similarity metrics based on deep networks. NIPS

  5. Fan DP, Cheng MM, Liu Y, et al (2017) Structure-measure: a new way to evaluate foreground maps[C] // IEEE international conference on computer vision

  6. Fan DP, Gong C, Cao Y, Ren B, Cheng MM, Borji A (2018) Enhanced-alignment measure for binary fore-ground map evaluation[C]// computer vision and. Pattern Recognition

  7. Fan DP, Zhang SC, Wu YH, et al (2019) Scoot: A perceptual metric for facial sketches[J]. Proceedings/ IEEE International Conference on Computer Vision. IEEE International Conference on Computer Vision

  8. Gao F, Wang Y, Li P, Tan M, Yu J, Zhu Y (2017) Deepsim: deep similarity for image quality assessment. Neuro-computing.:S092523121–S097301480

  9. Ghadiyaram D, Bovik AC (2015) Feature maps driven no-reference image quality prediction of authentically distorted images. Proceedings of SPIE - The International Society for Optical Engineering. https://doi.org/10.1117/12.2084807

  10. Hawkins J (2006) The future of AI[M]. Shaanxi Science and Technology Press, Shanxi

    Google Scholar 

  11. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition, 770–778. Computer Vision and Pattern Recognition, 2015. https://doi.org/10.1109/CVPR.2016.90

  12. He N, Xie K, Li T (2017) A review of image quality evaluation. Journal of Beijing Institute of Graphic Communication 25(02):47–50

    Google Scholar 

  13. Jia Y.e.a. (2014) Caffe: Convolutional architecture for fast feature embedding. In: ACM MM.; pp. 675–678.

  14. Johnson J, Alahi A, Fei-Fei L (2016) Perceptual losses for real-time style transfer and super-resolution. ECCV

  15. Kang L, Ye P, Li Y, Doermann D (1733–1740) Convolutional neural networks for no-reference image quality assessment. In CVPR 2014

  16. Kim J, Lee S (2017) Fully deep blind image quality predictor. IEEE Journal of Selected Topics in Signal Processing 11(1):206–220

    Article  Google Scholar 

  17. Li D, Jiang T, Jiang M (2017) Exploiting high-level semantics for no-reference image quality assessment of realistic blur images. Proceedings of the 2017 ACM on Multimedia Conference 2017. https://doi.org/10.1145/3123266.3123322.

  18. Lin H, Hosu V, Saupe, D (2018) “KonIQ-10k: Towards an ecologically valid and large-scale IQA database,” Computer Vision and Pattern Recognition.

  19. Liu L, Liu B, Huang H, Bovik AC (2014) No-reference image quality assessment based on spatial and spectral entropies. Signal Process Image Commun 29(8):856–863

    Article  Google Scholar 

  20. Liu X, Wang X, Zhang T (2015) An image quality evaluation method based on visual underlying features. Optical. 41(03):280–284

    Google Scholar 

  21. Lu P, Lin G, Zou G (2018) Study on non-reference image quality evaluation method based on information entropy and deep learning. Application Research of Computer 35(11):3508–3512

    Google Scholar 

  22. Mansouri A, Mahmoudi-Aznaveh A (2019) SSVD: Struc-tural SVD-based image quality assessment. Image Communication, Signal Processing

    Google Scholar 

  23. Margolin R, Zelnik-Manor L, Tal A (2014) How to evaluate foreground maps[C]// IEEE conference on computer vision and pattern. IEEE

    Google Scholar 

  24. Mittal A, Moorthy AK, Bovik AC (2012) No-reference image quality assessment in the spatial domain. IEEE Transactions on Image Processing A Publication of the IEEE Signal Processing Society. 21(12):4695–4708

    Article  MathSciNet  Google Scholar 

  25. Moorthy K, Bovik, Conrad A (2011) Blind image quality assessment: from natural scene statistics to perceptual quality. IEEE Trans Image Process 20(12):3350–3364

    Article  MathSciNet  Google Scholar 

  26. Saad MA, Bovik AC, Charrier C (2012) Blind image quality assessment: a natural scene statistics approach in the DCT domain. IEEE Transactions on Image Processing A Publication of the IEEE Signal Processing Society 21(8):3339.0

    Article  MathSciNet  Google Scholar 

  27. Siahaan E, Hanjalic A (2016) A RediJ Augmenting blind image quality assess-ment using image semantics In IEEE International Symposium on Multimedia:307–312

  28. Simonyan K (2014) Zisserman. A. Very deep convolu-tional networks for large-scale image recognition, Computer Science

    Google Scholar 

  29. Sun J, Li H (2017) Automatic image annota-tion based on multi-feature fusion and PLSA-GMM. Measurement & Control Technology 36(04):31–35+39

    Google Scholar 

  30. Sun C, Li H, Li W (2016) No-reference image quality assessment based on global and local content perception. In Visual Communications and Image Processing

  31. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D et al. (2015) Going deeper with convolutions. Computer Vision and Pattern Recognition.; pp. 1–9.

  32. Tang M (2018) Research on real image quality evaluation method based on subjective perception. Beijing University of Posts and Telecommunications

  33. Varga, D. ;Saupe, D. ; Sziranyi, T. Deeprn: (2018) A Content preserving deep architecture for blind image quality assessment. 2018 IEEE International Conference on Multimedia and Expo (ICME)

  34. Wang YX, Hebert M (2016) Learning to learn: model regression networks for easy small sample learning. Computer Vision – ECCV

  35. Yang L (2017) Research on non-reference quality evaluation based on real distortion image. Beijing University of Posts and Telecommunica-tions

  36. Yang L, Du H, Xu J, Yong L (2016) Blind image quality assessment on authentically distorted images with perceptual features. IEEE International Conference on Image Processing

  37. You H, Tian S, Yu L, Lv Y (Feb. 2020) Pixel-level remote sensing image recognition based on bidirectional word vectors. in IEEE Transactions on Geoscience and Remote Sensing 58(2):1281–1293

    Article  Google Scholar 

  38. Yu S, Wu S, Jiang F, Li L, Xie Y, Wang L (2017) A shallow convolutional neural network for blind image sharpness assessment. PLoS One 12(5):e0176632

    Article  Google Scholar 

  39. Zhang R et al. (2018) The unreasonable effectiveness of deep features as a perceptual metric." CVPR 2018.

Download references

Acknowledgements

This work was supported by Shanghai Education Development Foundation and Shanghai Municipal Education Commission.

Author information

Authors and Affiliations

  1. University of Shanghai for Science and Technology, Shanghai, 200093, China

    Xiaohong Wang & Yunjie Pang

  2. Shanghai Publishing and Printing College, Shanghai, 200093, China

    Xiangcai Ma

Authors
  1. Xiaohong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yunjie Pang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiangcai Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaohong Wang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

ESM 1

(PNG 2 kb)

ESM 2

(PNG 2 kb)

ESM 3

(PNG 12 kb)

ESM 4

(PNG 5 kb)

ESM 5

(PNG 7 kb)

ESM 6

(PNG 4 kb)

ESM 7

(PNG 36 kb)

ESM 8

(PNG 14 kb)

ESM 9

(PNG 6 kb)

ESM 10

(PNG 9 kb)

ESM 11

(PNG 6 kb)

ESM 12

(PNG 8 kb)

ESM 13

(PNG 12 kb)

ESM 14

(PNG 10 kb)

ESM 15

(PNG 26 kb)

ESM 16

(PNG 12 kb)

ESM 17

(PNG 42 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, X., Pang, Y. & Ma, X. Real distorted images quality assessment based on multi-layer visual perception mechanism and high-level semantics. Multimed Tools Appl 79, 25905–25920 (2020). https://doi.org/10.1007/s11042-020-09222-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09222-9

Keywords

Search

Navigation