Skip to main content
SpringerLink
Log in

Reduced-reference quality assessment of DIBR-synthesized images based on multi-scale edge intensity similarity

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Depth-image-based-rendering (DIBR) plays an important role in view synthesis for free-viewpoint videos. The warping process in DIBR causes geometric displacement, which distributes intensively around edges, and the subsequent rendering process results in the impairment of edges. Traditional 2D image quality metrics are limited in the quality evaluation of DIBR-synthesized images. In this paper, we present a reduced-reference quality metric for DIBR-synthesized images by only extracting several feature values, namely multi-scale Edge Intensity Similarity (EIS). The original and synthesized images are first downsampled to generate images with different resolutions. Then an edge detection process is conducted on each scale and the edge intensity is calculated. The similarity of the edge intensity between each downsampled original image and the corresponding synthesized image is computed. Finally, the average similarity is calculated as the quality score of the DIBR-synthesized image. Experiments conducted on IRCCyN/IVC DIBR image and video databases demonstrate that the proposed method overall outperforms traditional 2D and existing DIBR-targeted quality metrics.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Battisti F, Bosc E, Carli M, Callet PL (2015) Objective image quality assessment of 3D synthesized views. Sig Process.: Image Commun 30:78–88

    Google Scholar 

  2. Bosc E, Pépion R, Callet PL, Koppel M, Nya PN, Cedex R (2011) Towards a new quality metric for 3-D synthesized view assessment. IEEE J Select Top Signal Process 5(7):1332–1343

    Article  Google Scholar 

  3. Bosc E, Pepion R, Callet PL, Koppel M, Ndijiki-Nya P, Morin L, Pressigout M (2011) Perceived quality of DIBR-based synthesized views. SPIE Optics+ Photonics 8135(1):8135–16

    Google Scholar 

  4. Canny JF (1986) A computational approach to edge detection. IEEE Trans Parr Anal Mach Intell 8(6):679–698

    Article  Google Scholar 

  5. Cermak G, Thorpe L, Pinson M (2009) Test plan for evaluation of video quality models for use with high definition TV content. Video Quality Experts Group

  6. Chandler D, Hemami S (2007) VSNR: A wavelet-based visual signal-to-noise ratio for natural images. IEEE Trans Image Process 16(9):2284–2298

    Article  MathSciNet  Google Scholar 

  7. Conze PH, Robert P, Morin L (2012) Objective view synthesis quality assessment. Electron Imag Int Society for Optics and Photonics 8288:8288–8256

    Google Scholar 

  8. Duds RO, Hart PE (1973) Pattern classification and scene analysis. Wiley, New York

    Google Scholar 

  9. Fan YC, Huang PK, Shen DW (2013) 3DTV depth map reconstruction based on structured light scheme. IEEE Int Instrum Meas Technol Conf (I2MTC), pp 835–838

  10. Fehn C (2004) Depth-image-based rendering (DIBR), compression and transmission for a new approach on 3-d-TV. SPIE Conf Stereoscopic Displays and Virtual Reality Systems X(5291):93–104

    Google Scholar 

  11. Gu K, Zhai GT, Yang XK, Zhang WJ (2013) A new reduced-reference image quality assessment using structural degradation model. IEEE International Symposium on Circuits and Systems, pp 1095–1098

  12. Gu K, Zhai GT, Yang XK, Zhang WJ (2014) An efficient color image quality metric with local-tuned-global model. IEEE Int Conf Image Process, pp 506–510

  13. Gu K, Wang SQ, Zhai GT, Lin WS, Yang XK, Zhang WJ (2016) Analysis of distortion distribution for pooling in image quality prediction. IEEE Trans Broadcasting 62:446–456

    Article  Google Scholar 

  14. Gu K, Zhou J, Qiao JF, Zhai GT, Lin WS, Bovik AC (2017) No-reference quality assessment of screen content pictures. IEEE Trans Image Process 26(8):4005–4018

    Article  MathSciNet  Google Scholar 

  15. Gu K, Tao DC, Qiao JF, Lin WS (2017) Learning a no-reference quality assessment model of enhanced images with big data. IEEE Trans Neural Netw Learning Syst, pp 1–13

  16. Hsu CC, Lin CW, Fang YM, Lin WS (2014) Objective quality assessment for image retargeting based on perceptual geometric distortion and information loss. IEEE Journal of Selected Topics in Signal Processing 8(3):377–389

    Article  Google Scholar 

  17. Huszak A (2017) Advanced free viewpoint video streaming techniques. Multimed Tools Appl, https://doi.org/10.1007/s11042-015-3048-9

  18. ITU-T p.910 (1999) Subjective video quality assessment methods for multimedia applications

  19. Jain R, Kasturi R, Schunck BG (1995) Machine vision. McGraw-Hill, New York

    Google Scholar 

  20. Jung YJ, Kim HG, Ro YM (2016) Critical binocular asymmetry measure for perceptual quality assessment of synthesized stereo 3D images in view synthesis. IEEE Trans Circuits Syst Video Technol 26(7):1201–1214

    Article  Google Scholar 

  21. Kang LW, Hsu CC, Zhuang BQ, Lin CW, Yeh CH (2015) Learning-based joint super-resolution and deblocking for a highly compressed image. IEEE Trans Multimedia 17(7):921–934

    Article  Google Scholar 

  22. Kim HG, Ro YM (2016) Measurement of critical temporal inconsistency for quality assessment of synthesized video. IEEE Int Conf Image Process, pp 1027–1031

  23. Koppel M, Nya PN, Doshkov D, Lakshman H, Merkle P, Mller K, Wiegand T (2010) Temporally consistent handling of disocclusions with texture synthesis for depth-image-based rendering. IEEE Int Conf Image Process, pp 1809–1812

  24. Larson EC, Chandler DM (2010) Most apparent distortion: Full reference image quality assessment and the role of strategy. J Electr Imag 19(1):001006:1–21

    Google Scholar 

  25. Li B, Duan LY, Lin CW, Huang TJ, Gao W (2015) Depth-preserving warping for stereo image retargeting. IEEE Trans Image Process 24(9):2811–2826

    Article  MathSciNet  Google Scholar 

  26. Li LD, Cai H, Zhang YB, Lin WS, Kot AC, Sun XM (2016) Sparse representation based image quality index with adaptive sub-dictionaries. IEEE Trans Image Process 25(8):3775–3786

    MathSciNet  Google Scholar 

  27. Li LD, Wu D, Wu JJ, Li HL, Lin WS, Kot AC (2016) Image sharpness assessment by sparse representation. IEEE Trans Multimedia 18(6):1085–1097

    Article  Google Scholar 

  28. Li LD, Zhou Y, Lin WS, Wu JJ, Zhang XF, Chen BJ (2016) No-reference quality assessment of deblocked image. Neurocomputing 177:572–584

    Article  Google Scholar 

  29. Liang Y, Liu YJ, Gutierrez D (2016) Objective quality prediction of image retargeting algorithms. IEEE Trans Vis Comput Graph 23(2):1099–1110

    Article  Google Scholar 

  30. Liu YJ, Luo X, Xuan YM, Chen WF, Fu XL (2011) Image retargeting quality assessment. Computer Graphics Forum 30(2):583–592

    Article  Google Scholar 

  31. Liu AM, Lin WS, Narwaria M (2012) Image quality assessment based on gradient similarity. IEEE Trans Image Process 21(4):1500–1512

    Article  MathSciNet  MATH  Google Scholar 

  32. Mori Y, Fukushima N, Yendo T, Fujii T, Tanimoto M (2009) View generation with 3-D warping using depth information for FTV. Signal Processing: Image Commu 24:65–72

    Google Scholar 

  33. Muller K, Smolic A, Dix K, Merkle P, Kauff P, Wiegand T (2008) View synthesis for advanced 3-D video systems. J Image Video Process, Article ID 438148

  34. Narwaria M, Lin WS, McLoughlin IV, Emmanuel S, Chia LT (2012) Fourier transform-based scalable image quality measure. IEEE Trans Image Process 21(8):3364–3377

    Article  MathSciNet  MATH  Google Scholar 

  35. Nya PN, Koppel M, Doshkov D, Lakshman H, Merkle P, Muller K, Wiegand T (2010) Depth image based rendering with advanced texture synthesis. IEEE Int Conf Multimedia Expo, pp 424–429

  36. Purica A, Valenzise G, Popescu BP, Dufaux F (2016) Using region-of-interest for quality evaluation of DIBR-based view synthesis methods. IEEE Int Workshop on Quality of Multimedia Experience

  37. Ren TW, Wu GS (2010) Automatic image retargeting evaluation based on user perception. IEEE Int Conf Image Process, pp 1569–1572

  38. Roberts LG (1965) Machine perception of three-dimensional solids. Optical and Electro-Optical Information Process 20:159–197

  39. Ryu A, Kim S, Sohn K (2014) Synthesis quality prediction model based on distortion intolerance. IEEE Int Conf Image Process pp 585–589

  40. Stankovic DS, Kukolj D, Callet PL (2015) DIBR Synthesized image quality assessment based on morphological wavelets. IEEE Int Workshop on Quality of Multimedia Experience, pp 1–6

  41. Stankovic DS, Kukolj D, Callet PL (2015) DIBR Synthesized image quality assessment based on morphological pyramids. The True Vision-Capture, Transmission and Display of 3D Video, pp 1–4

  42. Stankovic DS, Kukolj D, Callet PL (2016) Multi-scale synthesized view assessment based on morphological pyramids. J Electr Eng 67(1):3–11

    Google Scholar 

  43. Telea A (2004) An image inpainting technique based on the fast marching method. J Graphics, GPU Game Tools 9(1):23–34

    Article  Google Scholar 

  44. Wang Z, Li Q (2011) Information content weighting for perceptual image quality assessment. IEEE Trans Image Process 20(5):1185–1198

    Article  MathSciNet  MATH  Google Scholar 

  45. Wang Z, Simoncelli EP (2005) Reduced-reference image quality assessment using a wavelet-domain natural image statistic model. SPIE 5666:149–159

    Google Scholar 

  46. Wang Z, Simoncelli EP, Bovik AC (2003) Multiscale structural similarity for image quality assessment. Asilomar Conf Signals, Syst Comput 2:1398–1402

    Google Scholar 

  47. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  Google Scholar 

  48. Wu JJ, Lin WS, Shi GM, Liu A (2013) Perceptual quality metric with internal generative mechanism. IEEE Trans Image Process 22(1):43–54

    Article  MathSciNet  MATH  Google Scholar 

  49. Wu JJ, Lin WS, Shi GM, Liu AM (2013) Reduced-reference image quality assessment with visual information fidelity. IEEE Trans Multimedia 15(7):1700–1705

    Article  Google Scholar 

  50. Xie X, Zaitsev Y, Velsquez-Garca LF, Teller SJ, Livermore C (2014) Scalable, MEMS-enabled, vibrational tactile actuators for high resolution tactile displays. J Micromech Microeng 24(12):125014

    Article  Google Scholar 

  51. Xie X, Zaitsev Y, Velasquez-Garcia LF, Teller SJ, Livermore C (2014) Compact, scalable, high-resolution, MEMS-enabled tactile displays. In: Proceedings of solid-state sensors, actuators, and microsystems workshop, pp 127–130

  52. Xue WF, Zhang L, Mou XQ, Bovik AC (2014) Gradient magnitude similarity deviation: a highly efficient perceptual image quality index. IEEE Trans Image Process 23(2):684–695

    Article  MathSciNet  MATH  Google Scholar 

  53. Yang Y, Wang X, Liu Q, Xu ML, Wu W (2017) User models of subjective image quality assessment on virtual viewpoint in free-viewpoint video system. Multimed Tools Appl, https://doi.org/10.1007/s11042-014-2321-7

  54. Zhang L, Zhang L, Mou XQ, Zhang D (2011) FSIM: A feature similarity index for image quality assessment. IEEE Trans Image Process 20(8):2378–2386

    Article  MathSciNet  MATH  Google Scholar 

  55. Zhang L, Shen Y, Li H (2014) VSI: A visual saliency-induced index for perceptual image quality assessment. IEEE Trans Image Process 23(10):4270–4280

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgments

This work was supported in part by the Fundamental Research Funds for the Central Universities under Grant No. 2017XKQY084.

Author information

Authors and Affiliations

  1. School of Information and Control Engineering, China University of Mining and Technology, Xuzhou, 221116, China

    Yu Zhou, Leida Li & Lijuan Tang

  2. Beijing Research Institute of Telemetry, Beijing, 100094, China

    Liu Yang

  3. BJUT Faculty of Information Technology, Beijing University of Technology, Beijing, 100124, China

    Ke Gu

  4. School of Information and Electronics Engineering, Jiangsu Vocational College of Business, Nantong, 226011, China

    Lijuan Tang

Authors
  1. Yu Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Liu Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Leida Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ke Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lijuan Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Leida Li.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, Y., Yang, L., Li, L. et al. Reduced-reference quality assessment of DIBR-synthesized images based on multi-scale edge intensity similarity. Multimed Tools Appl 77, 21033–21052 (2018). https://doi.org/10.1007/s11042-017-5543-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-5543-7

Keywords

Search

Navigation