Skip to main content
SpringerLink
Log in

Objective quality assessment of retargeted images based on RBF neural network with structural distortion and content change

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Objective quality assessment of retargeted images aims to find the best retargeting method for showing an image on different display terminals. This paper uses a Radial Basis Function (RBF) neural network to assess the quality of retargeted images. First, invariant feature points in the original image and their counterparts in the retargeted image are matched in a spatial order-preserving manner. Feature points centered local patches are extracted from original and retargeted images. Then, multi-scale local structural similarity and multi-scale local HSV color histograms difference of matched local patches are measured. A saliency map is employed as the weights of the local patches for evaluating the structural distortion and content change. Finally, the overall assessment of the retargeted image quality can be obtained by the RBF neural network. Experimental results on a benchmark test show the high consistency between the proposed objective assessment and subjective evaluations, and our method is closer to the practical application due to its simplicity compared with those complex ones.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Avidan S, Shamir A (2007) Seam carving for content-aware image resizing. In: ACM SIGGRAPH 2007 papers, pp 10. https://doi.org/10.1145/1275808.1276390

  2. Ballard DH (1981) Generalizing the Hough transform to detect arbitrary shapes. Pattern Recogn 13(2):111–122

    Article  MATH  Google Scholar 

  3. Bay H, Ess A, Tuytelaars T, Gool LV (2008) Speeded-up robust features (SURF). Comput Vis Image Underst 110(3):346–359

    Article  Google Scholar 

  4. Broomhead DS, Lowe D (1988) Radial basis functions, multi-variable functional interpolation and adaptive networks. Royal Signals and Radar Establishment Malvern (United Kingdom)

  5. Castillo S, Judd T, Gutierrez D (2011) Using eye-tracking to assess different image retargeting methods. In: Proceedings of the ACM SIGGRAPH Symposium on Applied Perception in Graphics and Visualization, pp 7–14. https://doi.org/10.1145/2077451.2077453

  6. Damera-Venkata N, Kite TD, Geisler WS, Evans BL, Bovik AC (2000) Image quality assessment based on a degradation model. IEEE Trans Image Process 9(4):636–650

    Article  Google Scholar 

  7. Dong W, Zhou N, Paul JC, Zhang X (2009) Optimized image resizing using seam carving and scaling. ACM Trans Graph 28(5):1–10. https://doi.org/10.1145/1618452.1618471

  8. Fang Y, Zeng K, Wang Z, Lin W, Fang Z, Lin CW (2014) Objective quality assessment for image retargeting based on structural similarity. IEEE J Emerg Sel Top Circ Syst 4(1):95–105

    Article  Google Scholar 

  9. Fischler, MA, Bolles, RC (1987) Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM 24(6):381–395. https://doi.org/10.1145/358669.358692

  10. Girod B (1993) What's wrong with mean-squared error? In: Digital Images and Human Vision. MIT Press, Cambridge, pp 207–220

  11. Han J, Ngan KN, Li M, Zhang HJ (2006) Unsupervised extraction of visual attention objects in color images. IEEE Trans Circ Syst Video Technol 16(1):141–145

    Article  Google Scholar 

  12. Harel J, Koch C, Perona P (2006) Graph-based visual saliency. In: Conference on Advances in Neural Information Processing Systems

  13. Harris CG, Stephens MJ (1988) A combined corner and edge detector. In: Alvey vision conference, pp 10–5244

  14. Hsu CC, Lin CW, Fang Y, Lin W (2014) Objective quality assessment for image retargeting based on perceptual geometric distortion and information loss. IEEE J Sel Top Signal Process 8(3):377–389

    Article  Google Scholar 

  15. Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell 20(11):1254–1259

    Article  Google Scholar 

  16. Karni Z, Freedman D, Gotsman C (2009) Energy-based image deformation. Comput Graph Forum 28(5):1257–1268

    Article  Google Scholar 

  17. Kasutani E (2001) The MPEG-7 color layout descriptor: a compact image feature description for high-speed image/video segment retrieval. In: Proc. of International Conference on Image Processing Oct. 2001

  18. Kendall MG (1938) A new measure of rank correlation. Biometrika 30(1–2):81–93

    Article  MATH  Google Scholar 

  19. Krähenbühl P (2009) A system for retargeting of streaming video. ACM Trans Graph 28(5):1–10. https://doi.org/10.1145/1618452.1618472

  20. Liang Y, Liu YJ, Gutierrez D (2016) Objective quality prediction of image retargeting algorithms. IEEE Trans Vis Comput Graph 23(2):1099–1110

    Article  Google Scholar 

  21. Liu C, Yuen J, Torralba A, Sivic J, Freeman WT (2008) SIFT flow: dense correspondence across different scenes. In: European conference on computer vision (ECCV) 2008, pp 28–42

  22. Liu YJ, Luo X, Xuan YM, Chen WF, Fu XL (2011) Image retargeting quality assessment. Computer Graphics Forum 30(2):583–592. https://doi.org/10.1111/j.1467-8659.2011.01881.x

  23. Liu A, Lin W, Chen H, Zhang P (2015) Image retargeting quality assessment based on support vector regression. Signal Process Image Commun 39:444–456

    Article  Google Scholar 

  24. Lowe DG (2004) Distinctive image features from scale-invariant Keypoints. Int J Comput Vis 60(2):91–110

    Article  Google Scholar 

  25. Ma L, Lin W, Deng C, Ngan KN (2012) Image retargeting quality assessment: A study of subjective scores and objective metrics. IEEE J Sel Top Signal Process 6(6):626–639

    Article  Google Scholar 

  26. Manjunath BS, Ohm JR, Vasudevan VV et al (2001) Color and texture descriptors. IEEE Trans Circ Syst Video Technol 11:703–715

    Article  Google Scholar 

  27. Novak CL, Shafer SA (1992) Anatomy of a color histogram. In: Proceedings1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 599–605. https://doi.org/10.1109/CVPR.1992.223129

  28. Oliva A, Torralba A, Castelhano MS, Henderson JM (2003) Top-down control of visual attention in object detection. In: Proceedings 2003 International Conference on Image Processing, pp I–253. https://doi.org/10.1109/ICIP.2003.1246946

  29. Pele O, Werman M (2009) Fast and robust earth mover's distances. In: Computer Vision, 2009 IEEE 12th International Conference on

  30. Pritch Y, Kav-Venaki E, Peleg S (2009) Shift-map image editing. In: IEEE International Conference on Computer Vision

  31. Rubinstein M, Shamir A, Avidan S (2008) Improved seam carving for video retargeting. ACM Trans Graph 27(3):1–9. https://doi.org/10.1145/1360612.1360615

    Article  Google Scholar 

  32. Rubinstein M, Shamir A, Avidan S (2009) Multi-operator media retargeting. ACM Trans Graph 28(3):1–11. https://doi.org/10.1145/1531326.1531329

  33. Rubinstein M, Gutierrez D, Sorkine O, Shamir A (2010) A comparative study of image retargeting. ACM Trans Graph 29:1–10

    Article  Google Scholar 

  34. Sethi OK, Jain AK (1991) Artificial neural networks and statistical pattern recognition: Old and new connections. Elsevier

  35. Simakov D, Caspi Y, Shechtman E, Irani M (2008) Summarizing visual data using bidirectional similarity. In: 2008 IEEE Conference on Computer Vision and Pattern Recognition, pp 1–8. https://doi.org/10.1109/CVPR.2008.4587842

  36. Wang YS (2008) Optimized scale-and-stretch for image resizing. ACM Trans Graph 27(5):1–8. https://doi.org/10.1145/1409060.1409071

  37. Wang Z, Simoncelli EP (2005) Translation insensitive image similarity in complex wavelet domain. In: IEEE International Conference on Acoustics

  38. Wang Z, Simoncelli EP, Bovik AC (2003) Multiscale structural similarity for image quality assessment. In: Proc IEEE Asilomar Conference on Signals

  39. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  Google Scholar 

  40. Wolf L, Guttmann M, Cohen-Or D (2007) Non-homogeneous Content-driven Video-retargeting. In: IEEE International Conference on Computer Vision

  41. Yan BYB, Bare BBB, Li KLK, Li JLJ, Bovik ACBAC (2017) Learning quality assessment of retargeted images. Signal Process Image Commun 56:12–19

    Article  Google Scholar 

Download references

Funding

This study was funded by National Natural Science Foundation of China (Grant No. 52075483).

Author information

Authors and Affiliations

  1. State Key Laboratory of Fluid Power and Mechatronic Systems, Zhejiang University, No. 38 Zheda Road, Hangzhou, 310027, Zhejiang, China

    Bin Zhou, Ze’an Liu, Jiayu Ji & Xuanyin Wang

Authors
  1. Bin Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ze’an Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jiayu Ji
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xuanyin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xuanyin Wang.

Ethics declarations

Conflict of interest

The authors have no conflicts of interests to declare that are relevant to the content of this article.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, B., Liu, Z., Ji, J. et al. Objective quality assessment of retargeted images based on RBF neural network with structural distortion and content change. Multimed Tools Appl 82, 7463–7477 (2023). https://doi.org/10.1007/s11042-022-13662-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-13662-w

Keywords

Search

Navigation