Skip to main content
SpringerLink
Log in

DRIQA-NR: no-reference image quality assessment based on disentangled representation

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Due to the characterization capabilities of deep features, image quality assessment (IQA) methods based on convolutional neural networks (CNNs) have been proposed. However, the existing CNN-based IQA does not make full use of deep features. So, we propose a novel no-reference image quality assessment based on disentangled representation (DRIQA-NR), which decomposes the deep features extracted from distorted images into content features and distortion information features. The content features are used to restore the input image. To eventually predict the quality of the image, features extracted from the restored image and the distorted image are merged with the distortion information feature. In addition, the distortion information features can also be used to improve the performance of full-reference image quality assessment. Experiments on LIVE, CSIQ and TID2013 suggest that the method proposed achieves favorable performance against other methods, with higher average Spearman’s rank-order correlation coefficient, Pearson’s linear correlation coefficient, Kendall rank-order correlation coefficient values and lower root mean square error.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Zhou, W., Bovik, A.C., et al.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  2. Bosse, S., et al.: Deep neural networks for no-reference and full-reference image quality assessment. IEEE Trans. Image Process. 27(1), 206–219 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  3. Seo, S., Ki, S.: Deep HVS-IQA Net: Human Visual System Inspired Deep Image Quality Assessment Networks (2019). arXiv:1902.05316

  4. Narwaria, M., Lin, W., et al.: Fourier transform-based scalable image quality measure. IEEE Trans. Image Process. Publ. IEEE Signal Process. Soc. 21(8), 3364–3377 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  5. Ma, L., et al.: Reduced-reference image quality assessment using reorganized DCT-based image representation. IEEE Trans. Multimed. 13(4), 824–829 (2011)

    Article  Google Scholar 

  6. Mittal, A., Moorthy, A.K., et al.: No-reference image quality assessment in the spatial domain. IEEE Trans. Image Process. Publ. IEEE Signal Process. Soc. 21(12), 4695 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  7. Liu, X., Weijer, J., et al.: Rankiqa: learning from rankings for no-reference image quality assessment. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1040–1049 (2017)

  8. Lin, K.Y., Wang, G.: Hallucinated-IQA: no-reference image quality assessment via adversarial learning. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 732–741 (2018)

  9. Krizhevsky, A., et al.: ImageNet classification with deep convolutional neural networks. In: 2012 Conference on Neural Information Processing Systems, pp. 1097–1105 (2012)

  10. Goodfellow, I.J., Pouget-Abadie, J., et al.: Generative adversarial nets. In: 2014 Conference on Neural Information Processing Systems, pp. 2672–2680 (2014)

  11. Bromley, J., Guyon, I., et al.: Signature verification using a “Siamese” time delay neural network. In: 7th Annual Neural Information Processing Systems Conference, pp. 737–744 (1993)

  12. Hughes, L.H., Schmitt, M., et al.: Identifying corresponding patches in SAR and optical images with a pseudo-Siamese CNN. IEEE Geosci. Remote Sens. Lett. 15(5), 784–788 (2018)

    Article  Google Scholar 

  13. Zhu, J., Park, T., et al.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: 2017 IEEE International Conference on Computer Vision, pp. 2242–2251 (2017)

  14. Zhang, J., Sclaroff, S., et al.: Minimum barrier salient object detection at 80 Fps. In: 2015 IEEE International Conference on Computer Vision, pp. 1401-1412 (2015)

  15. Bae, S.H., Kim, M.: A DCT-based total JND profile for spatio-temporal and foveated masking effects. IEEE Trans. Circuits Syst. Video Technol. 27(6), 1 (2017)

    Article  Google Scholar 

  16. Bae, S.H., Kim, J., et al.: HEVC-based perceptually adaptive video coding using a DCT-based local distortion detection probability model. IEEE Trans. Image Process. 25(7), 3343–3357 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  17. Saad, M.A., Bovik, A.C., et al.: Blind image quality assessment: a natural scene statistics approach in the DCT domain. IEEE Trans. Image Process. 21(8), 3339–3352 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  18. Le, K., Peng, Y., et al.: Simultaneous estimation of image quality and distortion via multi-task convolutional neural networks. In: 2015 IEEE International Conference on Image Processing, pp. 2791–2795 (2015)

  19. Ma, K., Liu X., et al.: Blind image quality assessment by learning from multiple annotators. In: 2019 IEEE International Conference on Image Processing (ICIP) (2019)

  20. Xia, W., et al.: Domain fingerprints for no-reference image quality assessment. IEEE Trans. Circuits Syst. Video Technol. 31(4), 1332–1341 (2021)

    Article  Google Scholar 

  21. Zhu, H., Li L., et al.: MetaIQA: deep meta-learning for no-reference image quality assessment. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 14131–14140 (2020)

  22. Weixia, Z.,Kede M., et al.: Task-Specific Normalization for Continual Learning of Blind Image Quality Models (2021). arXiv:2107.13429

  23. Lu, B., Chen, J.C., et al.: Unsupervised domain-specific deblurring via disentangled representations. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10225–10234 (2019)

  24. Du, W., Chen, H., et al.: Learning invariant representation for unsupervised image restoration. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14471–14480 (2020)

  25. Wang, L., Wu, Q., et al.: Blind tone-mapped image quality assessment and enhancement via disentangled representation learning. In: 2020 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), pp. 1096–1102 (2020)

  26. Bianco, S., et al.: Disentangling image distortions in deep feature space. Pattern Recogn. Lett. 148, 128–135 (2021)

    Article  Google Scholar 

  27. Bao, J., et al.: Towards open-set identity preserving face synthesis. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6713–6722 (2018)

  28. Min, X., Gu, K., et al.: Blind quality assessment based on pseudo-reference image. IEEE Trans. Multimed. 20(8), 2049–2062 (2018)

    Article  Google Scholar 

  29. Ponomarenko, M., et al.: Color image database HTID for verification of no-reference metrics: peculiarities and preliminary results. In: 2021 9th European workshop on visual information processing (EUVIP), pp. 1–6 (2021)

Download references

Author information

Authors and Affiliations

  1. Soochow University, Suzhou, Jiangsu, China

    Zhidan Ye, Yilin Wu, Dong Liao, Tianqi Yu, Jianfeng Yang & Jianling Hu

  2. Wuxi University, Wuxi, Jiangsu, China

    Jianling Hu

Authors
  1. Zhidan Ye
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yilin Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dong Liao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tianqi Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jianfeng Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jianling Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jianling Hu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ye, Z., Wu, Y., Liao, D. et al. DRIQA-NR: no-reference image quality assessment based on disentangled representation. SIViP 17, 661–669 (2023). https://doi.org/10.1007/s11760-022-02273-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-022-02273-5

Keywords

Search

Navigation