Skip to main content

Advertisement

Springer Nature Link
Log in

Image aesthetics enhancement using composition-based saliency detection

  • Special Issue Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Visual saliency detection and segmentation are widely used in many applications in image processing and computer vision. However, existing saliency detection methods have not fully taken the spatial information of salient regions into account. Inspired by the basic photographic composition rules, we present a novel saliency detection method, which utilizes the knowledge of photographic composition as priors to improve the saliency detection results. Moreover, an online parameter selection method is proposed when utilizing GrabCut to achieve the saliency segmentation result. Besides, to test the applicability of our method, we present a novel post-processing framework for the photographs to be more artistic. The salient region and depth map are firstly computed. The salient region keeps its sharpness, while other parts in the photograph get blurred based on the depth map. To our best knowledge, this is a novel image-based attempt to enhance aesthetics by post-processing a photograph via realistic blurring. We test our method on the 1,000 benchmark test images and dataset MSRA. Extensive experimental results show the applicability and effectiveness of our method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Achanta, R., Estrada, F.J., Wils, P., Süsstrunk, S.: Salient region detection and segmentation. In: International conference on computer vision, system, pp. 66–75 (2008)

  2. Achanta, R., Hemami, S.S., Estrada, F.J., Süsstrunk, S.: Frequency-tuned salient region detection. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1597–1604 (2009)

  3. Bae, S., Durand, F.: Defocus magnification. Comput. Graph. Forum 26, 571–579 (2007)

    Article  Google Scholar 

  4. Chen, J., Zhao, H., Han, Y., Cao, X.: Visual saliency detection based on photographic composition. In: International conference on internet multimedia computing and service, pp. 13–16 (2013)

  5. Cheng, M.M., Mitra, N.J., Huang, X., Hu, S.M.: Salient shape: group saliency in image collections. Vis. Comput. 30(4), 443–453 (2014)

    Google Scholar 

  6. Cheng, M.M., Zhang, G.X., Mitra, N.J., Huang, X., Hu, S.M.: Global contrast based salient region detection. In: CVPR, pp. 409–416 (2011)

  7. Daly, S.: The visible differences predictor: an algorithm for the assessment of image fidelity. In: SPIE/IS&T 1992 Symposium on Electronic Imaging: Science and Technology, pp 2–15 (1992)

  8. Das, S., Ahuja, N.: Performance analysis of stereo, vergence, and focus as depth cues for active vision. IEEE. Trans. Pattern Anal. Mach. Intell.17(12), 1213–1219 (1995)

    Article  Google Scholar 

  9. Datta, R., Joshi, D., Li, J., Wang, J.Z.: Studying aesthetics in photographic images using a computational approach. In: ECCV, pp. 288–301 (2006)

  10. Datta, R., Li, J., Wang, J.Z.: Learning the consensus on visual quality for next generation image management. In: ACM multimedia, pp. 533–536 (2007)

  11. Datta, R., Li, J., Wang, J.Z.: Algorithmic inferencing of aesthetics and emotion in natural images: An exposition. In: ICIP, special session on image aesthetics: mood and emotion, pp. 105–108 (2008)

  12. Davies, E.R.: Machine vision: theory, algorithms and practicalities. In: pp. 42–44. Academic Press, London (1990)

    Google Scholar 

  13. Eltoukhy, H.A., Kavusi, S.: Computationally efficient algorithm for multifocus image reconstruction. In: Sensors and camera systems for scientific, industrial, and digital photography applications, pp. 332–341 (2003)

  14. Forsyth, D.A., Ponce, J.: Computer vision: a modern approach. Prentice Hall Professional Technical Reference (2002)

  15. Goferman, S., Manor, L.Z., Tal, A.: Context-aware saliency detection. In: CVPR, pp. 2376–2383 (2010)

  16. Harel, J., Koch, C., Perona, P.: Graph-based visual saliency. In: Advances in Neural Information Processing Systems (NIPS), pp. 545–552 (2006)

  17. Hong, R., Wang, M., Xu, M., Yan, S., Chua, T.S.: Dynamic captioning: Video accessibility enhancement for hearing impairment. In: ACM multimedia, pp. 421–430 (2010)

  18. Hong, R., Wang, M., Yuan, X.T., Xu, M., Jiang, J., Yan, S., Chua, T.S.: Video accessibility enhancement for hearing impaired users. ACM. Trans. Multimed. Comput.7S, 24–42 (2011)

    Google Scholar 

  19. Hou, X., Zhang, L.: Saliency detection: a spectral residual approach. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–8 (2007)

  20. Huhle, B., Schairer, T., Jenke, P., Straßer, W.: Realistic depth blur for images with range data. Dynamic 3D, imaging pp. 84–95 (2009)

  21. Krages, B.: Photography: the art of composition, 1st edn. Allworth Press, New York (2005)

    Google Scholar 

  22. Liu, L., Chen, R., Wolf, L., Cohen-Or, D.: Optimizing photo composition. Comput. Graph. Forum 29, 469–478 (2010)

    Article  Google Scholar 

  23. Liu, T., Sun, J., Zheng, N.N., Tang, X., Shum, H.Y.: Learning to detect a salient object. In: CVPR, pp. 1–8 (2007)

  24. Ma, Y.F., Zhang, H.: Contrast-based image attention analysis by using fuzzy growing. In: ACM multimedia, pp. 374–381 (2003)

  25. Mahmoud, T.A., Marshall, S.: Threshold decomposition driven adaptive morphological filter for image sharpening. In: VISAPP, pp. 40–45 (2007)

  26. Maki, A., Watanabe, M., Geotensity, C.W.: Combining motion and lighting for 3D surface reconstruction. Int. J. Comput. Vis.48(2), 75–90 (2002)

    Article  MATH  Google Scholar 

  27. Malik, J., Rosenholtz, R.: Computing local surface orientation and shape from texture for curved surfaces. Int. J. Comput. Vis. 23(2), 149–168 (1997)

    Article  Google Scholar 

  28. McGuire, M., Matusik, W., Pfister, H., Hughes, J.F., Durand, F.: Defocus video matting. ACM Trans. Graph. 24(3), 567–576 (2005)

    Google Scholar 

  29. Moutoussis, K., Zeki, S.: A direct demonstration of perceptual asynchrony in vision. In: Proceedings of the Royal Society of London. Series B: Biological Sciences, pp. 393–399 (1997)

  30. Nagai, T., Ikehara, M., Kurematsu, A.: Hmm-based surface reconstruction from single images. Syst. Comput. Jpn. 38(11), 80–89 (2007)

    Article  Google Scholar 

  31. Peng, B., Veksler, O.: Parameter selection for graph cut based image segmentation. In: BMVC, pp. 332–341 (2008)

  32. Peters, G.: Aesthetic primitives of images for visualization. In: IEEE international conference on information visualization, pp. 316–325 (2007)

  33. Rother, C., Kolmogorov, V., Blake, A.: Grabcut: interactive foreground extraction using iterated graph cuts. ACM Trans. Graph 23, 309–314 (2004)

    Article  Google Scholar 

  34. Saxena, A., Chung, S.H., Ng, A.Y.: Learning depth from single monocular images. In: Advances in Neural Information Processing Systems (NIPS) (2005)

  35. Saxena, A., Chung, S.H., Ng, A.Y.: 3-d depth reconstruction from a single still image. Int. J. Comput. Vis. 76, 53–69 (2008)

    Article  Google Scholar 

  36. Saxena, A., Sun, M., Ng, A.Y.: Make3d: learning 3d scene structure from a single still image. IEEE Trans. Pattern Anal. Mach. Intell. 31(5), 824–840 (2009)

    Article  Google Scholar 

  37. Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 47, 1–35 (2002)

    Article  Google Scholar 

  38. Schavemaker, J.G.M., Reinders, M.J.T., Gerbrands, J.J., Backer, E.: Image sharpening by morphological filtering. Pattern Recogn. 33(6), 997–1012 (2000)

    Google Scholar 

  39. Subbarao, M., Wei, T.C., Surya, G.: Focused image recovery from two defocused images recorded with different camera settings. IEEE Trans. Image Process. 4(12), 1613–1628 (1995)

    Google Scholar 

  40. Tatler, B.W.: The central fixation bias in scene viewing: selecting an optimal viewing position independently of motor biases and image feature distributions. J. Vis. pp. 1–17 (2007)

  41. Valenti, R., Jaimes, A., Sebe, N.: Sonify your face: Facial expressions for sound generation. In: ACM multimedia, pp. 1363–1372 (2010)

  42. Valenti, R., Sebe, N., Gevers, T.: Facial expression recognition: a fully integrated approach. In: International conference on image analysis and processing workshops, pp. 125–130 (2007)

  43. Wang, M., Hong, R., Yuan, X.T., Yan, S., Chua, T.S.: Movie2comics: towards a lively video content presentation. Trans. Multimed.14, 858–870 (2012)

    Article  Google Scholar 

  44. Watson, A.B.: Toward a perceptual video quality metric. In: SPIE, pp. 139–147 (1998)

  45. Zhai, Y., Shah, M.: Visual attention detection in video sequences using spatiotemporal cues. In: ACM multimedia, pp. 815–824 (2006)

  46. Zhang, M., Zhang, L., Sun, Y., Feng, L., Ma, W.Y.: Auto cropping for digital photographs. In: ICME, pp. 438–441 (2005)

Download references

Acknowledgment

This work is partly supported by National Program on Key Basic Research Project (973 Program) under Grant 2013CB329301, National High-tech R&D Program of China (2013AA01A601), 100 Talents Program of The Chinese Academy of Sciences, the NSFC (under Grant 61202166), and Doctoral Fund of Ministry of Education of China (under Grant 20120032120042).

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Tianjin University, Tianjin, 300072, China

    Handong Zhao, Jingjing Chen & Yahong Han

  2. State Key Laboratory of Information Security, Institute of Information Engineering, Chinese Academy of Sciences, Beijing, 100093, China

    Xiaochun Cao

  3. Tianjin Key Laboratory of Cognitive Computing and Application, Tianjin University, Tianjin, China

    Yahong Han

Authors
  1. Handong Zhao
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Jingjing Chen
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Yahong Han
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Xiaochun Cao
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Yahong Han.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Zhao, H., Chen, J., Han, Y. et al. Image aesthetics enhancement using composition-based saliency detection. Multimedia Systems 21, 159–168 (2015). https://doi.org/10.1007/s00530-014-0373-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-014-0373-1

Keywords