Skip to main content
SpringerLink
Log in

CMAIR: content and mask-aware image retargeting

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In recent years, more and more image retargeting techniques have been proposed to facilitate our daily life, in particular those based on the use of seam carving, warping or the combination of them. However, these techniques can only retarget the source picture into the same shape of a square, and these approaches cannot be reshape into a circular, a polygon or other shapes. This paper focuses on creating a graphics editing system, named CMAIR (Content and Mask-Aware Image Retargeting), for image retargeting, which can retarget the source images into different shapes of image to highlight the salient objects of primary region of interest. CMAIR effectively supports removal of unimportant pixels, and frames as many surrounding objects inside the provided mask as possible. Also, we propose a unique irregular interpolation method to produce four possible target images, and an evaluation mechanism to decide the best candidate image as the final output with the consideration of image saliency. The results show that not only the source image can be placed into different targeted shapes of mask, but also the salient objects are retained and highlighted as much as possible.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

References

  1. Avidan S, Shamir A (2007) Seam carving for content-aware image resizing. In: ACM Transactions on Graphics (TOG), vol 26. ACM, pp 10

  2. Barnes C, Shechtman E, Finkelstein A, Goldman DB (2009) Patchmatch: a randomized correspondence algorithm for structural image editing. ACM Trans Graph 28(3):24–1

    Article  Google Scholar 

  3. Bo Y, Sun K, Liu L (2013) Matching-area-based seam carving for video retargeting. IEEE Trans Circuits Syst Video Technol 23(2):302–310

    Article  Google Scholar 

  4. Bo Y, Yuan B, Bo Y (2014) Effective video retargeting with jittery assessment. IEEE Trans Multimedia 16(1):272–277

    Article  Google Scholar 

  5. Castillo S, Judd T, Gutierrez D (2011) Using eye-tracking to assess different image retargeting methods. In: Proceedings of the ACM SIGGRAPH Symposium on Applied Perception in Graphics and Visualization, ACM, pp 7–14

  6. Cho S, Choi H, Matsushita Y, Lee S (2009) Image retargeting using importance diffusion. In: 2009 16th IEEE International Conference on Image processing (ICIP), IEEE, pp 977–980

  7. Deng C, Lin W, Cai J (2012) Content-based image compression for arbitrary-resolution display devices. IEEE Trans Multimedia 14(4):1127–1139

    Article  Google Scholar 

  8. Dong W, Zhou Ning, Paul J-C, Zhang X (2009) Optimized image resizing using seam carving and scaling. In: ACM Transactions on Graphics (TOG), vol 28, ACM, pp 125

  9. Fan X, Xie X, Zhou H-Q, Ma W-Y (2003) Looking into video frames on small displays. In: Proceedings of the Eleventh ACM International Conference on Multimedia, ACM, pp 247–250

  10. Fang Y, Chen Z, Lin W, Lin C-W (2012) Saliency detection in the compressed domain for adaptive image retargeting. IEEE Trans Image Process 21(9):3888–3901

    Article  MathSciNet  MATH  Google Scholar 

  11. Fang Y, Lin W, Lee B-S, Lau C-T, Chen Z, Lin C-W (2012) Bottom-up saliency detection model based on human visual sensitivity and amplitude spectrum. IEEE Trans Multimedia 14(1):187–198

    Article  Google Scholar 

  12. Fang Y, Zeng K, Wang Z, Lin W, Fang Z, Lin C-W (2014) Objective quality assessment for image retargeting based on structural similarity. cIEEE J Emerging Sel Top Circuits Syst 4(1):95–105

    Article  Google Scholar 

  13. Grundmann Matthias, Kwatra Vivek, Han Mei, Essa Irfan (2010) Discontinuous seam-carving for video retargeting. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, pp 569–576

  14. Guo Y, Liu F, Shi J, Zhou Z-H, Gleicher M (2009) Image retargeting using mesh parametrization. IEEE Trans Multimedia 11(5):856–867

    Article  Google Scholar 

  15. Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell 20(11):1254–1259

    Article  Google Scholar 

  16. Kiess J, Kopf S, Guthier B, Effelsberg W (2010) Seam carving with improved edge preservation. In: ISAndamp;t/SPIE electronic imaging, International Society for Optics and Photonics, pp 75420g–75420g

  17. Kopf S, Guthier B, Lemelson H, Effelsberg W (2009) Adaptation of web pages and images for mobile applications. In: ISAndamp;t/SPIE Electronic Imaging, International Society for Optics and Photonics, pp 72560c–72560c

  18. Lee J-H, Kim K, Park J, Park JY, Jung S-W (2016) Image recomposition using seam carving and insertion considering the rule of thirds. IEIE Transaction on Smart Processing and Computing 5(1):1–4

    Article  Google Scholar 

  19. Li B, Duan L-Y, Wang J, Ji R, Lin C-W, Gao W (2014) Spatiotemporal grid flow for video retargeting. IEEE Trans Image Process 23(4):1615–1628

    Article  MathSciNet  MATH  Google Scholar 

  20. Liang Y, Liu Y-J, Gutierrez D (2017) Objective quality prediction of image retargeting algorithms. IEEE Trans Vis Comput Graph 23(2):1099–1110

    Article  Google Scholar 

  21. Lin S-S, Yeh I-C, Lin C-H, Lee T-Y (2013) Patch-based image warping for content-aware retargeting. IEEE Trans Multimedia 15(2):359–368

    Article  Google Scholar 

  22. Liu F, Gleicher M (2005) Automatic image retargeting with fisheye-view warping. In: Proceedings of the 18th Annual ACM Symposium on User Interface Software and Technology, ACM, pp 153– 162

  23. Liu F, Gleicher M (2006) Video retargeting: automating pan and scan. In: Proceedings of the 14th ACM International Conference on Multimedia, ACM, pp 241–250

  24. Liu Hao, Xie Xing, Ma Wei-Ying, Zhang Hong-Jiang (2003) Automatic browsing of large pictures on mobile devices. In: Proceedings of the Eleventh ACM International Conference on Multimedia, ACM, pp 148–155

  25. Liu Y-J, Xi L, Xuan Y-M, Chen W-F, Fu X-L (2011) Image retargeting quality assessment. In: Computer Graphics Forum, vol 30. Wiley online library, pp 583–592

  26. Ma L, Lin W, Deng C, Ngan KN (2012) Image retargeting quality assessment: a study of subjective scores and objective metrics. IEEE J Sel Top Sign Proces 6(6):626–639

    Article  Google Scholar 

  27. Ma L, Lin W, Deng C, Ngan KN (2012) Study of subjective and objective quality assessment of retargeted images. In: 2012 IEEE International Symposium on Circuits and Systems (ISCAS), IEEE, pp 2677–2680

  28. Ma L, Xu L, Zhang Y, Yan Y, Ngan KN (2016) No-reference retargeted image quality assessment based on pairwise rank learning. IEEE Trans Multimedia 18 (11):2228–2237

    Article  Google Scholar 

  29. Ni B, Xu M, Cheng B, Wang M, Yan S, Qi T. (2013) Learning to photograph a compositional perspective. IEEE Trans Multimedia 15(5):1138–1151

    Article  Google Scholar 

  30. Pritch Y, Kav-Venaki E, Peleg S (2009) Shift-map image editing. In: 2009 IEEE 12th International Conference on Computer Vision, IEEE, pp 151–158

  31. Qi S, Chi Y-TJ, Peter AM, Ho J (2016) Casair: Content and shape-aware image retargeting and its applications. IEEE Trans Image Process 25(5):2222–2232

    Article  MathSciNet  MATH  Google Scholar 

  32. Qu Z, Wang J, Xu M, Lu H (2013) Context-aware video retargeting via graph model. IEEE Trans Multimedia 15(7):1677–1687

    Article  Google Scholar 

  33. Rubinstein M, Gutierrez D, Sorkine O, Shamir A (2010) A comparative study of image retargeting. In: ACM Transactions on Graphics (TOG), vol 29. ACM, pp 160

  34. Rubinstein M, Shamir A, Avidan S (2008) Improved seam carving for video retargeting. In: ACM Transactions on Graphics (TOG), vol 27. ACM, pp 16

  35. Rubinstein M, Shamir A, Avidan S (2009) Multi-operator media retargeting. In: ACM Transactions on Graphics (TOG), vol 28, ACM, pp 23

  36. Santella A, Agrawala M, DeCarlo D, Salesin D, Cohen M (2006) Gaze-based interaction for semi-automatic photo cropping. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, pp 771–780

  37. Shamir A, Sorkine O (2009) Visual media retargeting. In: ACM SIGGRAPH ASIA 2009 Courses, ACM, pp 11

  38. Simakov Denis, Caspi Yaron, Shechtman Eli, Irani Michal (2008) Summarizing visual data using bidirectional similarity. In: 2008. CVPR 2008. IEEE Conference on Computer Vision and Pattern Recognition, IEEE, pp 1–8

  39. Suh B, Ling H, Bederson BB, Jacobs DW (2003) Automatic thumbnail cropping and its effectiveness. In: Proceedings of the 16th Annual ACM Symposium on User Interface Software and Technology, ACM, pp 95–104

  40. Tao C, Jia J, Sun H (2007) Active window oriented dynamic video retargeting. In: Proceedings of the Workshop on Dynamical Vision, ICCV

  41. Wang Y-S, Fu H, Sorkine O, Lee T-Y, Seidel H-P (2009) Motion-aware temporal coherence for video resizing. ACM Trans Graph (TOG) 28(5):127

    Google Scholar 

  42. Wang Y-S, Tai C-L, Sorkine O, Lee T-Y (2008) Optimized scale-and-stretch for image resizing. In: ACM Transactions on Graphics (TOG), vol 27. ACM, pp 118

  43. Wolf L, Guttmann M, Cohen-Or D (2007) Non-homogeneous content-driven video-retargeting. In: 2007. ICCV 2007. IEEE 11th International Conference on Computer Vision, IEEE, pp 1–6

  44. Wu L, Cao L, Min X, Wang J (2014) A hybrid image retargeting approach via combining seam carving and grid warping. J Multimed 9(4):483–492

    Article  Google Scholar 

  45. Wu L, Gong Y, Yuan X, Zhang X, Cao L (2014) Semantic aware sport image resizing jointly using seam carving and warping. Multimed Tools Appl 70 (2):721–739

    Article  Google Scholar 

  46. Yen T-C, Tsai C-M, Lin C-W (2011) Maintaining temporal coherence in video retargeting using mosaic-guided scaling. IEEE Trans Image Process 20(8):2339–2351

    Article  MathSciNet  MATH  Google Scholar 

  47. Zhang L, Wang M, Nie L, Hong L, Rui Y, Qi T (2015) Retargeting semantically-rich photos. IEEE Trans Multimedia 17(9):1538–1549

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to express thanks to National Central University, Taiwan and the Iowa State University, America for providing a comfortable experimental environment and adequate research facilities.

Author information

Authors and Affiliations

  1. No. 300, Zhongda Rd., Zhongli District, Taoyuan City, 32001, Taiwan (R.O.C.)

    Hon-Hang Chang & Timothy K. Shih

  2. Iowa State University, 226 Atanasoff Hall, Ames, IA, 50011-1090, USA

    Carl K. Chang

  3. Iowa State University, 232 Atanasoff Hall, Ames, IA, 50011-1040, USA

    Wallapak Tavanapong

Authors
  1. Hon-Hang Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Timothy K. Shih
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Carl K. Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wallapak Tavanapong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hon-Hang Chang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chang, HH., Shih, T.K., Chang, C.K. et al. CMAIR: content and mask-aware image retargeting. Multimed Tools Appl 78, 21731–21758 (2019). https://doi.org/10.1007/s11042-019-7462-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-7462-2

Keywords

Search

Navigation