Skip to main content
SpringerLink
Log in

Multi-operator image retargeting in compressed domain by preserving aspect ratio of important contents

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Content-aware image retargeting have received extensively research attentions. However, most of exiting retargeting approaches perform resizing on raw image data in the pixel domain. Since images in the actual world are mostly stored and transmitted in the compressed domain, e.g. discrete cosine transformation (DCT) domain, the complete decompression and recompression are almost inevitable by using the pixel domain based retargeting methods, causing extra overheads with high computation complexity. To address this issue, we propose a novel multi-operator image retargeting method in the DCT domain, in which three techniques including indirect seam carving, similarity transformation, and direct seam carving based on gradient vector flow (GVF), are utilized to perform resizing. To eliminate the zigzag effects in the retargeted images, we also present a novel similarity transformation algorithm in the DCT domain by which the DCT coefficients instead of a whole block are rescaled during resizing. In addition, we develop two decoding schemes to solve the issue that the traditional inverse DCT cannot be directly applied to the decoding the retargeted images. Extensive results demonstrate that the presented multi-operator image retargeting method in the DCT domain can preserve the aspect ratio of visual important contents well and obtain the resized images of better quality than the existing methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Avidan S, Shamir A (2007) Seam carving for content-aware image resizing. ACM Trans Graph 26(3):1–10

    Article  Google Scholar 

  2. Battiato S, Farinella G -M, Puglisi G, Ravì D (2014) Saliency-based selection of Gradient Vector Flow paths for content aware image resizing. IEEE Trans Image Process 23(5):2081–2095

    Article  MathSciNet  Google Scholar 

  3. Chang H, Shih T-K, Chang C-K (2019) CMAIR:Content and mask-aware image retargeting. Multimed Tools Appl 78:21731–21758

    Article  Google Scholar 

  4. Chang H -S, Kang K (2005) A compressed domain scheme for classifying block edge patterns. IEEE Trans Image Process 8(3):145–151

    Article  MathSciNet  Google Scholar 

  5. Cheng M, Mitra N-J, Huang X, Torr P-H-S, Hu S (2015) Global contrast based salient region detection. IEEE Trans Pattern Anal Machine Intell 37(3):569–582

    Article  Google Scholar 

  6. Cho D, Park J, Oh T, Tai Y, Kweon I-S (2017) Weakly-and self-supervised learning for content-aware deep image retargeting. In: Proceedings of IEEE international conference on computer vision, pp 4568–4577

  7. Choi K-S, Ko S-J (2009) Fast Content-Aware image resizing scheme in the compressed domain. IEEE Trans Consum Electron 55(3):1514–1521

    Article  Google Scholar 

  8. Chou Y-C, Su P-C (2016) Efficient content-based cropping using visual saliency and SLIC for image retargeting. In: Proceedings of IEEE international conference on consumer electronics, pp 1–2

  9. Cui J, Cai Q, Lu H, Jia Z, Tang M (2020) Distortion-aware image retargeting based on continuous seam carving model. Signal Process 166 (Jan):107242.1-107242.10

    Google Scholar 

  10. Dong W, Zhou N, Paul J-C, Zhang X (2009) Optimized image resizing using seam carving and scaling. ACM Trans Graph 28(5):1–10

    Article  Google Scholar 

  11. Fang Y, Chen Z, Lin W, Lin C (2012) Saliency detection in the compressed domain for adaptive image retargeting. IEEE Trans Image Process 21 (9):3888–3901

    Article  MathSciNet  Google Scholar 

  12. Guo Y, Liu F, Shi J, Zhou Z -H, Gleicher M (2009) Image retargeting using mesh parametrization. IEEE Trans Multimed 11(5):856–867

    Article  Google Scholar 

  13. Hsu C-C, Lin C-W, Fang Y, Lin W (2014) Objective quality assessment for image retargeting based on perceptual geometric distortion and information loss. IEEE J Select Topics Signal Process 8(3):377–389

    Article  Google Scholar 

  14. Itti L, Baldi P (2009) Bayesian surprise attracts human attention. Vis Res 49(10):1295–1306

    Article  Google Scholar 

  15. Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell 20(11):1254–1259

    Article  Google Scholar 

  16. Kim J-S, Jeong S-G, Joo Y, Kim C-S (2011) Content-aware image and video resizing based on frequency domain analysis. IEEE Trans Consum Electron 57(2):615–622

    Article  Google Scholar 

  17. Kim Y, Jung S, Jung C, Kim C (2018) A structure-aware axis-aligned grid deformation approach for robust image retargeting. Multimed Tools Appl 77(6):7717–7739

    Article  Google Scholar 

  18. Krahenbuhl P, Lang M, Hornung A, Gross M (2009) A system for retargeting of streaming video. ACM Trans Graph 126

  19. Li K, Yan B, Liu L, Sun K (2014) Efficient DCT-based image retargeting in compressed domain. Proc IEEE Int Conf Multimed Expo 3(1):1–6

    Google Scholar 

  20. Lin S-S, Yeh I-C, Lin C-H, Lee T-Y (2013) Patch-Based Image warping for Content-Aware retargeting. IEEE Trans Multimed 15(2):359–368

    Article  Google Scholar 

  21. Liu T, Sun J, Zheng N, Tang X, Shum H-Y (2007) Learning to detect a salient object. IEEE Computer Vision and Pattern Recognition 1–8

  22. Luo S, Zhang J, Zhang Q, Yuan X (2012) Multi-operator image retargeting with automatic integration of direct and indirect seam carving. Image Vis Comput 655–667

  23. Mishra A, Scharfenberger C, Siva P, Li F, Wong A, Clausi D (2015) Desire:discontinuous energy seam carving for image retargeting via structural and textural energy functionals. In: Proceedings of IEEE international conference on image processing, pp 3695–3699

  24. Mukherjee P, Lall B (2020) Conditional random field based salient proposal set generation and its application in content aware seam carving. Signal Process Image Commun 87(2020):115890

    Article  Google Scholar 

  25. Murthy O-V-R, Muthuswamy K, Rajan D (2010) Image retargeting in compressed domain. In: Proceedings of IEEE international conference on pattern recognition, pp 4424–4427

  26. Nguyen T-V, Gao G (2017) Novel evaluation metrics for seam carving based image retargeting. In: Proceedings of IEEE international conference on image processing, pp 450–454

  27. Pritch Y, Kavvenaki E, Peleg S (2009) Shift-map image editing. Proc IEEE Int Conf Comput Vision 30(2):151–158

    Google Scholar 

  28. Qi S, Chi Y-J, Peter A-M, Ho J (2016) CASAIR:Content and Shape-aware image retargeting and its applications. IEEE Trans Image Process 25 (5):2222–2232

    Article  MathSciNet  Google Scholar 

  29. Rubinstein M, Gutierrez D, Sorkine O, Shamir A (2010) A comparative study of image retargeting. ACM Trans Graph 29(6):1–9

    Article  Google Scholar 

  30. Rubinstein M, Shamir A, Avidan S (2008) Improved seam carving for video retargeting. ACM Trans Graph 27(3):23–31

    Article  Google Scholar 

  31. Rubinstein M, Shamir A, Avidan S (2009) Multi-operator media retargeting. ACM Trans Graph 28(3):23–31

    Article  Google Scholar 

  32. Shafieyan F, Karimi N, Mirmahboub B, Samavi S, Shirani S (2014) Image seam carving using depth assisted saliency map. In: Proceedings of IEEE international conference on image processing , pp 1155–1159

  33. Shao F, Lin W, Lin W, Jiang Q, Jiang G (2017) QoE-guided warping for stereoscopic image retargeting. IEEE Trans Image Process 26 (10):4790–4805

    Article  MathSciNet  Google Scholar 

  34. Tan W, Yan B, Lin C, Niu X (2020) Cycle-IR:deep Cyclic Image Retargeting. IEEE Trans Multimed 22(7):1730–1743

    Article  Google Scholar 

  35. Wang Y-S, Tai C-L, Sorking O, Lee T-Y (2008) Optimized scaleand-stretch for image resizing. ACM Trans Graph 27(5):1–8

    Article  Google Scholar 

  36. Wolf L, Guttmann M, Cohenor D (2007) Non-homogeneous content-driven video-retargeting. In: Proceedings of IEEE international conference on computer vision, pp 1–6

  37. Yan B, Niu X, Bare B, Tan W (2020) Semantic segmentation guided pixel fusion for image retargeting. IEEE Trans Multimed 22(3):676–687

    Article  Google Scholar 

  38. Yan J, Lin S, Kang S-B, Tang X (2013) Learning the change for automatic image cropping. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 971–978

  39. Zhang D, Yang G, Li F (2020) Detecting seam carved images using uniform local binary patterns. Multimed Tools Appl 79:8415–8430

    Article  Google Scholar 

  40. Zhang J, Li S (2014) Compressed-domain video retargeting. IEEE Trans Image Process 23(2):797–809

    Article  MathSciNet  Google Scholar 

  41. Zhang Y, Sun Z, Jiang P, Huang Y, Peng J-L (2017) Hybrid image retargeting using optimized seam carving and scaling. Multimed Tools Appl 76(6):1–19

    Google Scholar 

  42. Zhang Y-B, Fang Y-M, Lin W-S, Zhang X-F, Li L-D (2016) Backward Registration-Based aspect ratio similarity for image retargeting quality assessment. IEEE Trans Image Process 25(9):4286–429

    Article  MathSciNet  Google Scholar 

  43. Zhou Y, Zhang L, Zhang C, Li P, Li X (2018) Perceptually aware image retargeting for mobile devices. IEEE Trans Image Process 27(5):2301–2313

    Article  MathSciNet  Google Scholar 

  44. Zhu L, Chen Z, Chen X, Liao N (2016) Saliency and structure preserving multi-operator image retargeting. In: Proceedings of IEEE international conference on acoustics, speech and signal processing, pp 1706–1710

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (Grant No. 61461006, 61563004). Zhenhua Tang is the corresponding author.

Author information

Authors and Affiliations

  1. School of Computer and Electronics Information, Guangxi University, Guangxi, China

    Zhenhua Tang, Jiemei Yao & Qian Zhang

  2. Guangxi Key Laboratory of Multimedia Communication and Network Technology, Guangxi, China

    Zhenhua Tang, Jiemei Yao & Qian Zhang

Authors
  1. Zhenhua Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiemei Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qian Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhenhua Tang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tang, Z., Yao, J. & Zhang, Q. Multi-operator image retargeting in compressed domain by preserving aspect ratio of important contents. Multimed Tools Appl 81, 1501–1522 (2022). https://doi.org/10.1007/s11042-021-11376-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11376-z

Keywords

Search

Navigation