Skip to main content

Advertisement

SpringerLink
Log in

Improved seam carving for structure preservation using efficient energy function

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

To reduce the consumption of power and making the handling of display devices easy the manufacturers are reducing the size of the screen. To fit the visual contents into the small screen of display devices without any image deformation development of an efficient image retargeting technique is required. The traditional seam carving technique is widely used to change the aspect ratio of the image. This technique produces the distortions on the prominent line segments when the successive optimal seam starts intersecting at the same location on their structure. In this paper, an improved seam carving technique is proposed which overcomes the limitation of the traditional seam carving technique. The novelty of the proposed technique is to restrict the occurrence of intersection by the optimal seams at the same location on the line structures and bypass them at the nearby locations. To meet this objective the proposed technique performs the energy enhancement operation at the intersection point. To justify the effectiveness of the proposed technique, the obtained results are compared and analyzed using subjective and objective image quality assessment (IQA) methods. In the subjective analysis, the observers provide their opinions about deformations on three quality parameters namely luminance (l), color (c), and structure (s). In the objective assessment, the structural similarity index measure (SSIM) generated various SSIM index values based on these parameters. After the IQA assessment, it is found that the average percentage similarly obtained from the traditional seam carving technique is 64.03% while in the case of the proposed technique it is 70.87%. The high percentage of similarity among all state of art justify the efficiency of the proposed technique.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25

Similar content being viewed by others

References

  1. Afshar F, Mansouri A, Zandi M (2017) Image retargeting quality assessment using structural similarity and information preservation rate. In: 2017 10th Iranian conference on machine vision and image processing (MVIP). IEEE, pp 58–63

  2. Andreadis I, Amanatiadis A (2005) Digital image scaling. In: 2005 IEEE instrumentation and measurement technology conference proceedings, vol 3. IEEE, pp 2028–2032

  3. Avidan S, Shamir A (2007) Seam carving for content-aware image resizing. In: ACM SIGGRAPH 2007 papers, pp 10–es

  4. Bansal M, Kumar M, Kumar M, Kumar K (2021) An efficient technique for object recognition using Shi-Tomasi corner detection algorithm. Soft Comput 25(6):4423–4432

    Article  Google Scholar 

  5. Chang CH, Chuang YY (2012, June) A line-structure-preserving approach to image resizing. In: 2012 IEEE conference on computer vision and pattern recognition. IEEE, pp 1075–1082

  6. Chen J, Bai G, Liang S, Li Z (2016) Automatic image cropping: a computational complexity study. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 507–515

  7. Chhabra P, Garg NK, Kumar M (2020) Content-based image retrieval system using ORB and SIFT features. Neural Comput & Applic 32(7):2725–2733

    Article  Google Scholar 

  8. Choi J, Kim C (2016) Sparse seam-carving for structure preserving image retargeting. Journal of Signal Processing Systems 85(2):275–283

    Article  Google Scholar 

  9. Cui J, Cai Q, Lu H, Jia Z, Tang M (2020) Distortion-aware image retargeting based on continuous seam carving model. Signal Process 166:107242

    Article  Google Scholar 

  10. Dong W, Zhou N, Paul JC, Zhang X (2009) Optimized image resizing using seam carving and scaling. ACM Transactions on Graphics (TOG) 28(5):1–10

    Article  Google Scholar 

  11. Dong WM, Bao GB, Zhang XP, Paul JC (2012) Fast multi-operator image resizing and evaluation. J Comput Sci Technol 27(1):121–134

    Article  Google Scholar 

  12. Fan P, Zhou RG, Hu WW, Jing N (2019) Quantum image edge extraction based on Laplacian operator and zero-cross method. Quantum Inf Process 18(1):1–23

    MATH  Google Scholar 

  13. Fang Y, Fang Z, Yuan F, Yang Y, Yang S, Xiong NN (2016) Optimized multioperator image retargeting based on perceptual similarity measure. IEEE Transactions on Systems, Man, and Cybernetics: Systems 47(11):2956–2966

    Article  Google Scholar 

  14. Fang Y, Yan J, Liu J, Wang S, Li Q, Guo Z (2017) Objective quality assessment of screen content images by uncertainty weighting. IEEE Trans Image Process 26(4):2016–2027

    Article  MathSciNet  MATH  Google Scholar 

  15. Gal, R., Sorkine, O., & Cohen-Or, D. (2006). Feature-aware texturing Rendering techniques, 2006 (17th), 2.

  16. Garg A, Negi A (2020) A survey on content aware image resizing methods. KSII Transactions on Internet and Information Systems (TIIS) 14(7):2997–3017

  17. Garg A, Negi A (2020) Structure preservation in content-aware image retargeting using multi-operator. IET Image Process 14(13):2965–2975

    Article  Google Scholar 

  18. Garg D, Garg NK, Kumar M (2018) Underwater image enhancement using blending of CLAHE and percentile methodologies. Multimed Tools Appl 77(20):26545–26561

    Article  Google Scholar 

  19. Garg A, Negi A, Jindal P (2021) Structure preservation of image using an efficient content-aware image retargeting technique. Signal, Image and Video Processing 15(1):185–193

  20. Guo Y, Hao Y, Yu M (2018) Image retargeting quality assessment based on content deformation measurement. Signal processing: image communication 67:171–181

    Google Scholar 

  21. Han D (2005) Real-time digital image warping for display distortion correction. In: International conference image analysis and recognition. Springer, Berlin, Heidelberg, pp 1258–1265

  22. Hashemzadeh M, Asheghi B, Farajzadeh N (2019) Content-aware image resizing: an improved and shadow-preserving seam carving method. Signal Process 155:233–246

    Article  Google Scholar 

  23. Hore A, Ziou D (2010) Image quality metrics: PSNR vs. SSIM. In: 2010 20th international conference on pattern recognition. IEEE, pp 2366–2369

  24. Hu W, Luo Z, Fan X (2014) Image retargeting via adaptive scaling with geometry preservation. IEEE Journal on Emerging and Selected Topics in Circuits and Systems 4(1):70–81

    Article  Google Scholar 

  25. Huo L, Jiao L, Wang S, Yang S (2016) Object-level saliency detection with color attributes. Pattern Recogn 49:162–173

    Article  Google Scholar 

  26. Israni S, Jain S (2016) Edge detection of license plate using Sobel operator. In: 2016 international conference on electrical, electronics, and optimization techniques (ICEEOT). IEEE, pp 3561–3563

  27. Jiang W, Xu H, Chen G, Zhao W, Xu W (2009) An improved edge-adaptive image scaling algorithm. In: 2009 IEEE 8th international conference on ASIC. IEEE, pp 895–897

  28. Kiess J, Kopf S, Guthier B, Effelsberg W (2010) Seam carving with improved edge preservation. In: Multimedia on Mobile devices 2010, vol 7542. International Society for Optics and Photonics, p 75420G

  29. Kiess J, Guthier B, Kopf S, Effelsberg W (2012) SeamCrop for image retargeting. In: Multimedia on Mobile devices 2012; and multimedia content access: algorithms and systems VI, vol 8304. International Society for Optics and Photonics, p 83040K

  30. Kiess J, Kopf S, Guthier B, Effelsberg W (2018) A survey on content-aware image and video retargeting. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 14(3):1–28

    Article  Google Scholar 

  31. Kim CH, Seong SM, Lee JA, Kim LS (2003) Winscale: an image-scaling algorithm using an area pixel model. IEEE Transactions on circuits and systems for video technology 13(6):549–553

    Article  Google Scholar 

  32. Kopf S, Guthier B, Lemelson H, Effelsberg W (2009) Adaptation of web pages and images for Mobile applications. In: Multimedia on mobile devices 2009, vol 7256. International Society for Optics and Photonics, p 72560C

  33. Krähenbühl P, Lang M, Hornung A, Gross M (2009) A system for retargeting of streaming video. In: ACM SIGGRAPH Asia 2009 papers, pp 1–10

  34. Kumar M, Chhabra P, Garg NK (2018) An efficient content based image retrieval system using BayesNet and K-NN. Multimed Tools Appl 77(16):21557–21570

    Article  Google Scholar 

  35. Kumar A, Kumar M, Kaur A (2021) Face detection in still images under occlusion and non-uniform illumination. Multimed Tools Appl 80(10):14565–14590

    Article  Google Scholar 

  36. Liang Y, Liu YJ, Luo XN, Xie L, Fu X (2012a) Optimal-scaling-factor assignment for patch-wise image retargeting. IEEE Comput Graph Appl 33(5):68–78

    Article  Google Scholar 

  37. Liang Y, Su Z, Luo X (2012b) Patchwise scaling method for content-aware image resizing. Signal Process 92(5):1243–1257

    Article  Google Scholar 

  38. Liang Y, Liu YJ, Gutierrez D (2016) Objective quality prediction of image retargeting algorithms. IEEE Trans Vis Comput Graph 23(2):1099–1110

    Article  Google Scholar 

  39. Liu F, Gleicher M (2005) Automatic image retargeting with fisheye-view warping. In: Proceedings of the 18th annual ACM symposium on user interface software and technology, pp 153–162

  40. Liu Z, Yan H, Shen L, Ngan KN, Zhang Z (2010) Adaptive image retargeting using saliency-based continuous seam carving. Opt Eng 49(1):017002

    Article  Google Scholar 

  41. Mohammadi, P., Ebrahimi-Moghadam, A., & Shirani, S. (2014). Subjective and objective quality assessment of image: a survey . arXiv preprint arXiv:1406.7799.

  42. Muñoz A, Blu T, Unser M (2001) Least-squares image resizing using finite differences. IEEE Trans Image Process 10(9):1365–1378

    Article  MathSciNet  MATH  Google Scholar 

  43. Othman Z, Abdullah A (2017) An adaptive threshold based on multiple resolution levels for canny edge detection. In: International conference of reliable information and communication technology. Springer, Cham, pp 316–323

  44. Palubinskas G (2017) Image similarity/distance measures: what is really behind MSE and SSIM? Int J Image Data Fusion 8(1):32–53

    Article  Google Scholar 

  45. Panozzo, D., Weber, O., & Sorkine, O. (2012). Robust image retargeting via axis-aligned deformation. In Computer Graphics Forum (Vol. 31, No. 2pt1, pp. 229–236). Oxford, UK: Blackwell Publishing Ltd.

  46. Patel D, Raman S (2018) Object proposals-based significance map for image retargeting. In: Proceedings of 2nd international conference on Computer Vision & Image Processing. Springer, Singapore, pp 89–101

  47. Patel D, Raman S (2019) Accelerated seam carving for image retargeting. IET Image Process 13(6):885–895

    Article  Google Scholar 

  48. Pritch Y, Kav-Venaki E, Peleg S (2009) Shift-map image editing. In: 2009 IEEE 12th international conference on computer vision. IEEE, pp 151–158

  49. Qi S, Chi YTJ, Peter AM, Ho J (2016) CASAIR: content and shape-aware image retargeting and its applications. IEEE Trans Image Process 25(5):2222–2232

    Article  MathSciNet  MATH  Google Scholar 

  50. Rubinstein M, Shamir A, Avidan S (2009) Multi-operator media retargeting. ACM Transactions on Graphics (TOG) 28(3):1–11

    Article  Google Scholar 

  51. Rubinstein M, Gutierrez D, Sorkine O, Shamir A (2010) A comparative study of image retargeting. In: ACM SIGGRAPH Asia 2010 papers, pp 1–10

    Google Scholar 

  52. Sara U, Akter M, Uddin MS (2019) Image quality assessment through FSIM, SSIM, MSE and PSNR—a comparative study. Journal of Computer and Communications 7(3):8–18

    Article  Google Scholar 

  53. Senturk ZK, Akgun D (2019) Seam carving based image retargeting: a survey. In: 2019 1st international informatics and software engineering conference (UBMYK). IEEE, pp 1–6

    Google Scholar 

  54. Setlur V, Takagi S, Raskar R, Gleicher M, Gooch B (2005) Automatic image retargeting. In: Proceedings of the 4th international conference on Mobile and ubiquitous multimedia, pp 59–68

    Google Scholar 

  55. Shafieyan F, Karimi N, Mirmahboub B, Samavi S, Shirani S (2017) Image retargeting using depth assisted saliency map. Signal Process Image Commun 50:34–43

    Article  Google Scholar 

  56. Shamir A, Avidan S (2009) Seam carving for media retargeting. Commun ACM 52(1):77–85

    Article  Google Scholar 

  57. Shi Z, Yao S, Zhao Y (2008) A novel video image scaling algorithm based on morphological edge interpolation. In: 2008 international conference on neural networks and signal processing. IEEE, pp 388–391

    Google Scholar 

  58. Suh B, Ling H, Bederson BB, Jacobs DW (2003) Automatic thumbnail cropping and its effectiveness. In: Proceedings of the 16th annual ACM symposium on user interface software and technology, pp 95–104

    Chapter  Google Scholar 

  59. Tomaszewska AL (2016) Scene reduction for subjective image quality assessment. Journal of Electronic Imaging 25(1):013015

    Article  Google Scholar 

  60. Tsai YJ, Lin HJ, Li YS (2013) A straight line preserving seam carving technique. In: Applied mechanics and materials (Vol. 385, pp. 1453-1456). Trans Tech Publications Ltd

    Google Scholar 

  61. Vaquero D, Turk M, Pulli K, Tico M, Gelfand N (2010) A survey of image retargeting techniques. In: Applications of digital image processing XXXIII (Vol. 7798, p. 779814). International Society for Optics and Photonics

    Google Scholar 

  62. Wang YS, Tai CL, Sorkine O, Lee TY (2008) Optimized scale-and-stretch for image resizing. In: ACM SIGGRAPH Asia 2008 papers, pp 1–8

    Google Scholar 

  63. Wu L, Cao L, Xu M, Wang J (2014) A hybrid image retargeting approach via combining seam carving and grid warping. J Multimed 9(4):483

    Article  Google Scholar 

  64. Xuan L, Hong Z (2017) An improved canny edge detection algorithm. In: 2017 8th IEEE international conference on software engineering and service science (ICSESS). IEEE, pp 275–278

    Chapter  Google Scholar 

  65. Yan B, Li K, Yang X, Hu T (2014) Seam searching-based pixel fusion for image retargeting. IEEE Transactions on Circuits and Systems for Video Technology 25(1):15–23

    Article  Google Scholar 

  66. Yang, L., Liu, S., & Salvi, M. (2020). A survey of temporal antialiasing techniques. In Computer graphics forum (Vol. 39, no. 2, pp. 607-621).

  67. Zeng H, Li L, Cao Z, Zhang L (2019) Reliable and efficient image cropping: a grid anchor based approach. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 5949–5957

    Google Scholar 

  68. Zhai G, Min X (2020) Perceptual image quality assessment: a survey. SCIENCE CHINA Inf Sci 63(11):211301

    Article  Google Scholar 

  69. Zhang L, Li K, Ou Z, Wang F (2017a) Seam warping: a new approach for image retargeting for small displays. Soft Comput 21(2):447–457

    Article  Google Scholar 

  70. Zhang Y, Sun Z, Jiang P, Huang Y, Peng J (2017b) Hybrid image retargeting using optimized seam carving and scaling. Multimed Tools Appl 76(6):8067–8085

    Article  Google Scholar 

  71. Zhou B, Wang X, Cao S, Xiang K, Zhao S (2016) Optimal bi-directional seam carving for compressibility-aware image retargeting. J Vis Commun Image Represent 41:21–30

    Article  Google Scholar 

  72. Zhou RG, Yu H, Cheng Y, Li FX (2019) Quantum image edge extraction based on improved Prewitt operator. Quantum Inf Process 18(9):1–24

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Amity University, Gurugram, Haryana, 122413, India

    Ankit Garg & Anuj Kumar Singh

  2. Graduate School, Duy Tan University, Da Nang, 550000, Vietnam

    Anand Nayyar

  3. Faculty of Information Technology, Duy Tan University, Da Nang, 550000, Vietnam

    Anand Nayyar

Authors
  1. Ankit Garg
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anand Nayyar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Anuj Kumar Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ankit Garg.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Garg, A., Nayyar, A. & Singh, A.K. Improved seam carving for structure preservation using efficient energy function. Multimed Tools Appl 81, 12883–12924 (2022). https://doi.org/10.1007/s11042-022-12003-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12003-1

Keywords

Search

Navigation