Skip to main content

Advertisement

Springer Nature Link
Log in

Multi-operator image retargeting with visual quality preservation of salient regions

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Most of the existing content-aware retargeting algorithms are prone to introduce geometric distortions or information loss undesirably in salient regions of resized images, especially for pictures with visually important content taking up a lot of space. The main reason is that these algorithms rescale photos only toward the horizontal or vertical direction when shrinking the width or height of the original image. To address this issue, we propose a width-height synchronization resizing strategy and integrate it into a multi-operator retargeting method that combines three techniques: indirect seam carving, uniform scaling, and direct seam carving based on gradient vector flow (GVF). Moreover, we also develop a novel algorithm to determine the switching point from one retargeting technique to another, with low computational overhead. Besides, we design a novel method that achieves the weighted parameter of scaling adaptively, improving the robustness of the retargeting algorithm. Experimental results demonstrate that the proposed method outperforms the existing approaches in the quality of retargeted images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

References

  1. Guo, Y., Liu, F., Shi, J., et al.: Image Retargeting Using Mesh Parametrization. IEEE Trans. Multimedia 11(5), 856–867 (2009)

    Article  Google Scholar 

  2. Rubinstein, M., Gutierrez, D., Sorkine, O. et al.: A comparative study of image retargeting. ACM Trans Graph 29(6), 160:1–160:9 (2010)

  3. Lin, S.S., Yeh, I.C., Lin, C.H., et al.: Image retargeting using mesh parametrization. IEEE Trans. Multimed 11(5), 856–867 (2009)

    Article  Google Scholar 

  4. Wang, Y.S., Tai, C.L., Sorking, O. et al.: Optimized scale-and-stretch for image resizing. ACM Trans. Graph. 27(5), 118:1–118:8 (2008)

  5. Avidan, S., Shamir, A.: Seam carving for content-aware image resizing. ACM transactions on graphics 26(3), 1–10 (2007)

    Article  Google Scholar 

  6. Shafieyan, F., Karimi, N., Mirmahboub, B., et al.: Image seam carving using depth assisted saliency map, pp. 1155–1159. Paris, France, Oct, Proc. Int. Conf. on Image Processing (2014)

    Google Scholar 

  7. Mishra, A., Scharfenberger, C., Siva, P., et al.: Desire: discontinuous energy seam carving for image retargeting via structural and textural energy functionals, pp. 3695–3699. QC, Canada, Sep, Proc. Int. Conf. on Image Processing (2015)

    Google Scholar 

  8. Battiato, S., Farinella, G.M., Puglisi, G., et al.: Saliency-based selection of Gradient Vector Flow paths for content aware image resizing. IEEE Trans. Image Process. 23(5), 2081–2095 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  9. Rubinstein, M., Shamir, A., Avidan, S.: ’Multi-operator media retargeting,’, ACM transactions on graphics, 2009, 28, (3), pp. 23:1–23:11

  10. Luo, S., Zhang, J., Zhang, Q., et al.: Multi-operator image retargeting with automatic integration of direct and indirect seam carving. Image Vis. Comput. 30(9), 655–667 (2012)

    Article  Google Scholar 

  11. Dong, W., Zhou, N., Paul, J.C., et al.: ’Optimized Image Resizing Using Seam Carving and Scaling,’, ACM transactions on graphics, 2009, 28, (5), pp. 125:1–125:10

  12. Zhu, L., Chen, Z., Chen, X., et al.: Saliency and Structure Preserving Multi-operator Image Retargeting, pp. 1706–1710. Speech and Signal Processing, Shanghai, China, Mar, IEEE Int. Conf. on Acoustics (2016)

    Google Scholar 

  13. Tang, Z., Yao, J., Zhang, Q.: Multi-operator Image Retargeting in Compressed Domain by Preserving Aspect Ratio of Important Contents. Multimedia Tools and Applications 81(1), 1501–1522 (2022)

    Article  Google Scholar 

  14. Zhang, Q., Tang, Z., Jiang, H., et al.: Multi-operator Image Retargeting with Preserving Aspect Ratio of Important Contents, pp. 306–315. Harbin, China, Sep, Proc. Pacific Rim Conf. on Multimedia (2017)

    Google Scholar 

  15. Rubinstein, M., Shamir, A., Avidan, S.: ’Improved seam carving for video retargeting,’, ACM transactions on graphics, 2008, 27, (3), pp. 16:1–16:9

  16. Tang, Z., Yao, J.: Image Retargeting Quality Assessment Based on Saliency-Driven Classification. Signal Processing: Image Communication 105(116688), 1–13 (2022)

    Google Scholar 

  17. Niu, Y., Feng, W.C., Liu, F.: Enabling warping on stereoscopic images. ACM transactions on graphics 31(6), 1–7 (2012)

    Article  Google Scholar 

  18. Li, B., Duan, L.Y., Lin, C.W., et al.: Depth-Preserving Warping for Stereo Image Retargeting. IEEE Trans. Image Process. 24(9), 2811–2826 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  19. Shao, F., Lin, W., Lin, W., et al.: QoE-guided warping for stereoscopic image retargeting. IEEE Trans. Image Process. 26(10), 4790–4805 (2017)

    Article  MathSciNet  Google Scholar 

  20. Cho, D., Park, J., Oh, T., et al.: ’Weakly- and Self-Supervised Learning for Content-Aware Deep Image Retargeting’. Proc. IEEE Int. Conference on Computer Vision, Venice, Italy, Oct 2017, pp. 4568–4577

  21. Tan, W., Yan, B., Lin, C., et al.: Cycle-IR: Deep Cyclic Image Retargeting. IEEE Trans. Multimedia 22(7), 1730–1743 (2020)

    Article  Google Scholar 

  22. Lau, C.P., Yung, C.P., Lui, L.M.: Image retargeting via beltrami representation’. IEEE Trans. Image Process. 27(5), 5787–5801 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  23. Chang, H.H., Shih, T.K., Chang, C.K., et al.: CMAIR: content and mask-aware image retargeting. Multimedia Tools and Applications 78(3), 21731–21758 (2019)

    Article  Google Scholar 

  24. Mukherjee, P., Lall, B.: Conditional Random Field based salient proposal set generation and its application in content aware seam carving,’ Signal Processing Image Communication, 2020, 87, pp.115890-1-115890-12

  25. Cui, J., Cai, Q. Q., Lu, H. J., et al.: Distortion-aware image retargeting based on continuous seam carving model’, Signal Processing, 2020, 166, pp.107242-1-107242-10

  26. Yan, B., Niu, X.J., Bare, B., et al.: Semantic Segmentation Guided Pixel Fusion for Image Retargeting. IEEE Trans. Multimedia 22(3), 676–687 (2020)

    Article  Google Scholar 

  27. Qi, S., Chi, Y.T.J., eter, A.M., et al.: ’CASAIR: Content and Shape-aware image retargeting and its applications,’, IEEE Transactions on Image Processing, 2016, 25, (5), pp. 2222–2232

  28. Zhou, Y., Zhang, L., Zhang, C., et al.: Perceptually Aware Image Retargeting for Mobile Devices. IEEE Trans. Image Process. 27(5), 2301–2313 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  29. GribbonK, T., Bailey, D.G.: A novel approach to real-time bilinear interpolation, pp. 126–131. Test and Applications, Perth, Australia, Jan, Proc. IEEE Int. Workshop on Electronic Design (2004)

    Google Scholar 

  30. Goferman, S., Manor, L.Z., Tal, A.: Context-aware saliency detection. IEEE Trans. Pattern Anal. Mach. Intell. 34(10), 1915–1926 (2012)

    Article  Google Scholar 

  31. Otsu, N.: A Threshold Selection Method from Gray-Level Histograms. IEEE Trans. Syst. Man Cybern. 9(1), 62–66 (1979)

    Article  MathSciNet  Google Scholar 

  32. Niblack W. An introduction to digital image processing. Advances in Computer Graphics Vi, Images: Synthesis, Analysis, & Interaction. Springer-Verlag, 1986

  33. Kittler, J., Illingworth, J.: Minimum error thresholding. Pattern Recogn. 19(1), 41–47 (1986)

    Article  Google Scholar 

  34. Achanta, R., Hemami, S., Estrada, F., et al.: Frequency-tuned salient region detection, pp. 1597–1604. Miami, USA, Jun, IEEE Int. Conf. on Computer Vision and Pattern Recognition (2009)

    Google Scholar 

  35. Fang, Y., Chen, Z., Lin, W., et al.: Saliency detection in the compressed domain for adaptive image retargeting. IEEE Trans. Image Process. 21(9), 3888–3901 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  36. ’Retargetme. A benchmark for image retargeting’, http://people .csail.mit.edu/mrub/, accessed 27 November 2020

  37. Wolf, L., Guttmann, M., Cohenor, D.: Nonhomogeneous content-driven video-retargeting, pp. 1–6. Rio De Janeiro, Brazil, Oct, IEEE Int. Conf. on Computer Vision (2007)

    Google Scholar 

  38. Pritch, Y., Kav-Venaki, E., Peleg, S.: Shift-map image editing, pp. 151–158. Kyoto, Japan, Sept, IEEE Int. Conf. on Computer Vision (2009)

    Google Scholar 

  39. Krahenbuhl, P., Lang, M., Hornung, A., et al.: ’A system for retargeting of streaming video,’, ACM transactions on graphics, 2009, 28, (5), pp. 126:1–10

  40. Hsu, C.C., Lin, C.W., Fang, Y., et al.: Objective quality assessment for image retargeting based on perceptual geometric distortion and information loss. IEEE Journal of Selected Topics in Signal Processing 8(3), 377–389 (2014)

    Article  Google Scholar 

  41. Zhang, Y., Fang, Y., Lin, W., et al.: Backward Registration-Based Aspect Ratio Similarity for Image Retargeting Quality Assessment. IEEE Trans. Image Process. 5(9), 4286–4297 (2016)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the Natural Science Foundation of Guangxi, China (no. 2021GXNSFAA220058).

Author information

Authors and Affiliations

  1. School of Computer and Electronics Information, Guangxi University, Nanning, Guangxi, China

    Zhenhua Tang, Jiemei Yao, Qian Zhang & Yuanting Luo

  2. Guangxi Key Laboratory of Multimedia Communication and Network Technology, Nanning, Guangxi, China

    Zhenhua Tang

Authors
  1. Zhenhua Tang
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Jiemei Yao
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Qian Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Yuanting Luo
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Zhenhua Tang.

Additional information

Communicated by P. Pala.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tang, Z., Yao, J., Zhang, Q. et al. Multi-operator image retargeting with visual quality preservation of salient regions. Multimedia Systems 29, 811–829 (2023). https://doi.org/10.1007/s00530-022-01023-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-022-01023-4

Keywords