Skip to main content
Springer Nature Link
Log in

Hierarchical multi-scale stereoscopic image quality assessment based on visual mechanism

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

The recent advance on stereoscopic image quality assessment (SIQA) models has been remarkably improved due to the pervasive application of convolutional neural network (CNN). Although the current CNN-based methods have achieved good results, these methods only extract single scale features at the same level. And some CNN-based methods directly take left and right images as an input of the network ignoring the visual fusion mechanism. In this work, a hierarchical multi-scale no-reference SIQA method is proposed based on dilated convolution. Multi-scale module constructed by standard convolution will lead to a sharp increase in the number of model parameters. On the contrary, the dilated convolution can restrain the increase in the number of model parameters and enlarge the receptive field. Therefore, dilated convolution is used to simulate the multi-scale characteristics of human vision. In addition, instead of left and right images, the cyclopean image generated by a new method is used as the input of the network. Experimental results on four public databases show that the proposed model is superior to the state-of-the-art SIQA methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Chen, L., Zhao, J.: Robust contourlet-based blind watermarking for depth-image-based rendering 3d images. Sig. Proc. Image Commun. 54, 56–65 (2017)

    Article  Google Scholar 

  2. Khan, S., Appina, B., Channappayya, S.S.: Full-reference stereo image quality assessment using natural stereo scene statistics. IEEE Sig. Proc. Lett. 22(11), 1985–1989 (2015)

    Article  Google Scholar 

  3. Wang, X., Kwong, S., Zhang, Y., Zhang, Y.: Considering binocular spatial sensitivity in stereoscopic image quality assessment. In: 2011 Visual Communications and Image Processing (VCIP), pp. 1–4 (2011)

  4. Shao, F., Li, K., Lin, W., Jiang, G., Yu, M., Dai, Q.: Full-reference quality assessment of stereoscopic images by learning binocular receptive field properties. IEEE Trans. Image Proc. 24(10), 2971–2983 (2015)

    Article  MathSciNet  Google Scholar 

  5. Lin, Y.H., Wu, J.L.: Quality assessment of stereoscopic 3d image compression by binocular integration behaviors. IEEE Trans. Image Proc. 23(4), 1527–1542 (2014)

    Article  MathSciNet  Google Scholar 

  6. Wang, Z., Bovik, A., Sheikh, H., Simoncelli, E.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Proc. 13(4), 600–612 (2004)

    Article  Google Scholar 

  7. Chen, M.J., Su, C.C., Kwon, D.K., Cormack, L.K., Bovik, A.C.: Full-reference quality assessment of stereopairs accounting for rivalry. Sig. Proc. Image Comm. 28(9), 1143–1155 (2013)

    Article  Google Scholar 

  8. Maalouf, A., Larabi, M.C.: Cyclop: A stereo color image quality assessment metric. In: 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1161–1164 (2011)

  9. Li, Q., Wang, Z.: Reduced-reference image quality assessment using divisive normalization-based image representation. IEEE J. Select. Top. Sig. Proc. 3(2), 202–211 (2009)

    Article  Google Scholar 

  10. Ma, J., An, P., Shen, L., Li, K.: Reduced-reference stereoscopic image quality assessment using natural scene statistics and structural degradation. IEEE Access 6, 2768–2780 (2018)

    Article  Google Scholar 

  11. Wu, J., Lin, W., Shi, G., Xu, L.: Reduced-reference image quality assessment with local binary structural pattern. In: 2014 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 898–901 (2014)

  12. Liu, T.J., Lin, C.T., Liu, H.H., Pei, S.C.: Blind stereoscopic image quality assessment based on hierarchical learning. IEEE Access 7, 8058–8069 (2019)

    Article  Google Scholar 

  13. Liu, Y., Kong, F., Zhen, Z.: Toward a quality predictor for stereoscopic images via analysis of human binocular visual perception. IEEE Access 7, 69283–69291 (2019)

    Article  Google Scholar 

  14. Yang, J., Sim, K., Gao, X., Lu, W., Meng, Q., Li, B.: A blind stereoscopic image quality evaluator with segmented stacked autoencoders considering the whole visual perception route. IEEE Trans. Image Proc. 28(3), 1314–1328 (2019)

    Article  MathSciNet  Google Scholar 

  15. Chen, L., Zhao, J.: No-reference quality assessment for stereoscopic 3d images based on binocular visual perception. In: 2018 IEEE International Symposium on Haptic, Audio and Visual Environments and Games (HAVE), pp. 1–5 (2018)

  16. Yang, J., Sim, K., Lu, W., Jiang, B.: Predicting stereoscopic image quality via stacked auto-encoders based on stereopsis formation. IEEE Trans. Multim. 21(7), 1750–1761 (2019)

    Article  Google Scholar 

  17. Sang, Q., Gu, T., Li, C., Wu, X.: Stereoscopic image quality assessment via convolutional neural networks. In: 2017 International Smart Cities Conference (ISC2), pp. 1–2 (2017)

  18. Xu, X., Shi, B., Gu, Z., Deng, R., Chen, X., Krylov, A.S., Ding, Y.: 3d no-reference image quality assessment via transfer learning and saliency-guided feature consolidation. IEEE Access 7, 85286–85297 (2019)

    Article  Google Scholar 

  19. Li, S., Wang, M.: No-reference stereoscopic image quality assessment based on convolutional neural network with a long-term feature fusion. In: 2020 IEEE International Conference on Visual Communications and Image Processing (VCIP), pp. 318–321 (2020)

  20. Zhang, W., Qu, C., Ma, L., Guan, J., Huang, R.: Learning structure of stereoscopic image for no-reference quality assessment with convolutional neural network. Pattern Recog. 59, 176–187 (2016)

    Article  Google Scholar 

  21. Ding, Y., Deng, R., Xie, X., Xu, X., Zhao, Y., Chen, X., Krylov, A.S.: No-reference stereoscopic image quality assessment using convolutional neural network for adaptive feature extraction. IEEE Access 6, 37595–37603 (2018)

    Article  Google Scholar 

  22. Fang, Y., Yan, J., Liu, X., Wang, J.: Stereoscopic image quality assessment by deep convolutional neural network. J. Vis. Comm. Image Represent. 58, 400–406 (2019)

    Article  Google Scholar 

  23. Oh, H., Ahn, S., Kim, J., Lee, S.: Blind deep s3d image quality evaluation via local to global feature aggregation. IEEE Trans. Image Proc. 26(10), 4923–4936 (2017)

    Article  MathSciNet  Google Scholar 

  24. Ge, B., Guo, L., Zhang, T., Zhu, D., Hu, X., Han, J., Liu, T.: Construction of multi-scale brain networks via dicccol landmarks. In: 2013 IEEE 10th International Symposium on Biomedical Imaging, pp. 680–683 (2013)

  25. Yu, F., Koltun, V.: Multi-scale context aggregation by dilated convolutions. In: ICLR (2016)

  26. Meegan, D.V., Stelmach, L.B., Tam, W.J.: Unequal weighting of monocular inputs in binocular combination: implications for the compression of stereoscopic imagery. J. Exp. Psychol. Appl. 7(2), 143–153 (2001)

    Article  Google Scholar 

  27. Lu, K., Zhu, W.: Stereoscopic image quality assessment based on cyclopean image. In: 2016 IEEE 14th Intl Conf on Dependable, Autonomic and Secure Computing, 14th Intl Conf on Pervasive Intelligence and Computing, 2nd Intl Conf on Big Data Intelligence and Computing and Cyber Science and Technology Congress, pp. 420–423 (2016)

  28. Ding, J., Klein, S.A., Levi, D.M.: Binocular combination of phase and contrast explained by a gain-control and gain-enhancement model. J. Vis. 13(2), 1–37 (2013)

    Google Scholar 

  29. Sun, G., Ding, Y., Deng, R., Zhao, Y., Chen, X., Krylov, A.S.: Stereoscopic image quality assessment by considering binocular visual mechanisms. IEEE Access 6, 51337–51347 (2018)

    Article  Google Scholar 

  30. Moorthy, A.K., Su, C.C., Mittal, A., Bovik, A.C.: Subjective evaluation of stereoscopic image quality. Sig. Proc. Image Comm. 28(8), 870–883 (2013)

    Article  Google Scholar 

  31. Wang, J., Zhou, W.: Perceptual quality of asymmetrically distorted stereoscopic images:the role of image distortion types. In: International Workshop on Video Processing I & Quality Metrics for Consumer Electronics, pp. 1–6 (2014)

  32. Wang, J., Rehman, A., Zeng, K., Wang, S., Wang, Z.: Quality prediction of asymmetrically distorted stereoscopic 3d images. IEEE Trans. Image Proc. 24(11), 3400–3414 (2015)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China under Grant 61971306, 61520106002, 61471262.

Author information

Authors and Affiliations

  1. School of Electrical and Information Engineering, Tianjin University, Nankai District, Tianjin, 300072, China

    Yongli Chang, Sumei Li & Ping Zhao

Authors
  1. Yongli Chang
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Sumei Li
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Ping Zhao
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Sumei Li.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chang, Y., Li, S. & Zhao, P. Hierarchical multi-scale stereoscopic image quality assessment based on visual mechanism. SIViP 16, 1177–1185 (2022). https://doi.org/10.1007/s11760-021-02068-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-021-02068-0

Keywords