Skip to main content
SpringerLink
Log in

A survey of aesthetics-driven image recomposition

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The advancement of digital photography and the popularity of photo sharing social media such as Instagram and Facebook have undoubtedly stimulated growing interest in aesthetics quality improvement. One aspect of photography that contributes to high quality photos is image composition; the spatial arrangement of photo subjects in the image frame. Professional photographers often apply a wealth of photographic composition rules, e.g., rule of thirds, visual balance and simplicity to capture compelling photos. In the recent years, aesthetics-driven recomposition that attempts to computationally modify the composition of an image to mimic a professional photo has started to receive considerable research interest. Researchers have proposed numerous recomposition techniques that utilize a single or a combination of multiple image operators, i.e., cropping, warping and patch rearrangement operators, to modify the composition of an image. In this paper, we present a survey on the state-of-the-arts aesthetic-driven image recomposition. We define the image recomposition problem, outline its objectives, and provide a comprehensive review of the existing image recompositoin techniques, together with a detailed analysis of the effectiveness of each technique in achieving the recomposition objectives. This survey is intended as a good reference for researchers interested in image recomposition.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24

Similar content being viewed by others

Notes

  1. http://www.oxforddictionaries.com/definition/english/aesthetics

  2. http://www.maths.surrey.ac.uk/hosted-sites/R.Knott/Fibonacci/fib.html

  3. RT = Rule of thirds, DD = Diagonal Dominance, VB = Visual Balance, SD = Subject Dominance, RS = Region Size

References

  1. Avidan S, Shamir A (2007) Seam carving for content-aware image resizing. ACM Trans Graph (ToG) 26(3):1–10

    Article  Google Scholar 

  2. Barnes C, Shechtman E, Finkelstein A, Goldman D (2009) Patchmatch: a randomized correspondence algorithm for structural image editing. ACM Trans Graph (ToG) 28(3):24:1–24:11

    Article  Google Scholar 

  3. Bhattacharya S, Sukthankar R, Shah M (2011) A holistic approach to aesthetic enhancement of photographs. ACM Trans Multimed Comput Commun Appl (TOMM) 7(1):1–21

    Article  Google Scholar 

  4. Chang HT, Pan PC, Wang YCF, Chen MS (2015) R2p: recomposition and retargeting of photographic images. In: ACM international conference on multimedia. ACM, pp 927–930

  5. Chang HT, Wang YCF, Chen MS (2014) Transfer in photography composition. In: ACM international conference on multimedia. ACM, pp 957–960

  6. Cho TS, Avidan S, Freeman WT (2010) The patch transform. IEEE Trans Pattern Anal Mach Intell (TPAMI) 32(8):1489–1501

    Article  Google Scholar 

  7. Cho TS, Butman M, Avidan S, Freeman WT (2008) The patch transform and its applications to image editing. In: IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 1–8

  8. Cour T, Srinivasan P, Shi J (2007) Balanced graph matching. Adv Neural Inf Proces Syst 19:1–313

    Google Scholar 

  9. Criminisi A, Pérez P, Toyama K (2004) Region filling and object removal by exemplar-based image inpainting. IEEE Trans Image Process 13(9):1200–1212

    Article  Google Scholar 

  10. Datta R, Joshi D, Li J, Wang JZ (2006) Studying aesthetics in photographic images using a computational approach. In: European conference on computer vision (ECCV). Springer, pp 288–301

  11. Fang C, Lin Z, M?ech R, Shen X (2014) Automatic image cropping using visual composition, boundary simplicity and content preservation models. In: ACM international conference on multimedia. ACM, pp 1105–1108

  12. Goferman S, Zelnik-Manor L, Tal A (2012) Context-aware saliency detection. IEEE Trans Pattern Anal Mach Intell (TPAMI) 34(10):1915–1926

    Article  Google Scholar 

  13. Greco L, Cascia ML (2013) Saliency based aesthetic cut of digital images. In: International conference on image analysis and processing, pp 151–160

  14. Guo Y, Liu M, Gu T, Wang W (2012) Improving photo composition elegantly: considering image similarity during composition optimization. In: Computer graphics forum, vol 31. Wiley Online Library, pp 2193–2202

  15. Hsu CC, Lin CW, Fang Y, Lin W (2014) Objective quality assessment for image retargeting based on perceptual geometric distortion and information loss. IEEE J Sel Top Sign Proces 8(3):377–389

    Article  Google Scholar 

  16. Islam MB, Lai-Kuan W, Chee-Onn W, Low LK (2015) Stereoscopic image warping for enhancing composition aesthetics. In: Asian conference on pattern recognition (ACPR)

  17. Jin Y, Wu Q, Liu L (2012) Aesthetic photo composition by optimal crop-and-warp. Comput Graph 36(8):955–965

    Article  Google Scholar 

  18. Ke Y, Tang X, Jing F (2006) The design of high-level features for photo quality assessment. In: IEEE international conference on computer vision and pattern recognition (CVPR). IEEE, pp 419–426

  19. Li K, Yan B, Li J, Majumder A (2015) Seam carving based aesthetics enhancement for photos. Signal Process Image Commun 39:509–516

    Article  Google Scholar 

  20. Liu L, Chen R, Wolf L, Cohen-Or D (2010) Optimizing photo composition. Comput Graphics Forum 29(2):469–478

    Article  Google Scholar 

  21. Liu L, Jin Y, We Q (2010) Realtime aesthetic image retargeting. In: International conference on computational aesthetics in graphics, visualization and imaging, pp 1–8

  22. Lu X, Lin Z, Jin H, Yang J, Wang JZ (2014) Rapid: rating pictorial aesthetics using deep learning. In: ACM international conference on multimedia. ACM, pp 457–466

  23. Lu X, Lin Z, Shen X, Mech R, Wang JZ (2015) Deep multi-patch aggregation network for image style, aesthetics, and quality estimation. In: IEEE International conference on computer vision (ICCV), pp 990–998

  24. Luo Y, Tang X (2008) Photo and video quality evaluation: Focusing on the subject. In: European conference on computer vision (ECCV). Springer, pp 386–399

  25. Marchesotti L, Perronnin F, Larlus D, Csurka G (2011) Assessing the aesthetic quality of photographs using generic image descriptors. In: IEEE international conference on computer vision (ICCV). IEEE, pp 1784–1791

  26. Murray N, Marchesotti L, Perronnin F (2012) Ava: a large-scale database for aesthetic visual analysis. In: IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 2408–2415

  27. Park J, Lee JY, Tai YW, Kweon IS (2012) Modeling photo composition and its application to photo re-arrangement. In: IEEE international conference on image processing (ICIP). IEEE, pp 2741–2744

  28. Pritch Y, Kav-Venaki E, Peleg S (2009) Shift-map image editing. In: IEEE international conference on computer vision (ICCV). IEEE, pp 151–158

  29. Qi S, Ho J (2012) Shift-map based stereo image retargeting with disparity adjustment. In: Asian conference on computer vision (ACCV). Springer, pp 457–469

  30. Rother C, Kolmogorov V, Blake A (2004) Grabcut: interactive foreground extraction using iterated graph cuts. ACM Trans Graph (ToG) 23(3):309–314

    Article  Google Scholar 

  31. Samii A, Měch R, Lin Z (2015) Data-driven automatic cropping using semantic composition search. In: Computer graphics forum, vol 34, pp 141–151

  32. Santella A, Agrawala M, DeCarlo D, Salesin D, Cohen M (2006) Gaze-based interaction for semi-automatic photo cropping. In: ACM conference on human factors in computing systems. ACM, pp 771–780

  33. Stentiford F (2007) Attention based auto image cropping. In: Workshop on computational attention and applications (ICVS), vol 1. Citeseer

  34. Suh B, Ling H, Bederson BB, Jacobs DW (2003) Automatic thumbnail cropping and its effectiveness. In: ACM symposium on user interface software and technology. ACM, pp 95–104

  35. Tian X, Dong Z, Yang K, Mei T (2015) Query-dependent aesthetic model with deep learning for photo quality assessment. IEEE Trans Multimedia (TMM) 17 (11):2035–2048

    Article  Google Scholar 

  36. Wolf L, Guttmann M, Cohen-Or D (2007) Non-homogeneous content-driven video-retargeting. In: IEEE international conference on computer vision (ICCV). IEEE, pp 1–6

  37. Wong LK, Low KL (2009) Saliency-enhanced image aesthetics class prediction. In: IEEE international conference on image processing (ICIP). IEEE, pp 997–1000

  38. Wong LK, Low KL (2012) Enhancing visual dominance by semantics-preserving image recomposition. In: ACM international conference on multimedia. ACM, pp 845–848

  39. Wong LK, Low KL (2012) Tearable image warping for extreme image retargeting. In: Computer graphics international (CGI), pp 1–8

  40. Yan J, Lin S, Kang SB, Tang X (2013) Learning the change for automatic image cropping. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 971–978

  41. Yan T, He S, Lau RW, Xu Y (2013) Consistent stereo image editing. In: ACM international conference on multimedia. ACM, pp 677–680

  42. Zhang FL, Wang M, Hu SM (2013) Aesthetic image enhancement by dependence-aware object recomposition. IEEE Trans Multimedia (TMM) 15 (7):1480–1490

    Article  Google Scholar 

  43. Zhang M, Zhang L, Sun Y, Feng L, Ma W (2005) Auto cropping for digital photographs. In: IEEE international conference on multimedia and expo (ICME). IEEE, pp 4–7

Download references

Acknowledgments

The authors would like to thank all anonymous photographers who shared their photos in Flickr (license free). This work is supported by the Fundamental Research Grant Scheme (FRGS), Grant No. EP20130326018 and Multimedia University (MMU) Internal Grant, No IP20131108001.

Author information

Authors and Affiliations

  1. Multimedia University, Cyberjaya, 63100, Selangor, Malaysia

    Md Baharul Islam, Wong Lai-Kuan & Wong Chee-Onn

Authors
  1. Md Baharul Islam
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wong Lai-Kuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wong Chee-Onn
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Md Baharul Islam or Wong Lai-Kuan.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Islam, M.B., Lai-Kuan, W. & Chee-Onn, W. A survey of aesthetics-driven image recomposition. Multimed Tools Appl 76, 9517–9542 (2017). https://doi.org/10.1007/s11042-016-3561-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-016-3561-5

Keywords

Search

Navigation