Skip to main content
SpringerLink
Log in

Saliency for image manipulation

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Every picture tells a story. In photography, the story is portrayed by a composition of objects, commonly referred to as the subjects of the piece. Were we to remove these objects, the story would be lost. When manipulating images, either for artistic rendering or cropping, it is crucial that the story of the piece remains intact. As a result, the knowledge of the location of these prominent objects is essential. We propose an approach for saliency detection that combines previously suggested patch distinctness with an object probability map. The object probability map infers the most probable locations of the subjects of the photograph according to highly distinct salient cues. The benefits of the proposed approach are demonstrated through state-of-the-art results on common data sets. We further show the benefit of our method in various manipulations of real-world photographs while preserving their meaning.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23

Similar content being viewed by others

References

  1. http://cgm.technion.ac.il/Computer-Graphics-Multimedia/Software/ImMnplSal

  2. Achanta, R., Hemami, S., Estrada, F., Susstrunk, S.: Frequency-tuned salient region detection. In: CVPR, pp. 1597–1604 (2009)

    Google Scholar 

  3. Achanta, R., Shaji, A., Fua, P., Süsstrunk, S.: Image summaries using database saliency. In: SIGGRAPH ASIA Posters (2009)

    Google Scholar 

  4. Boiman, O., Irani, M.: Detecting irregularities in images and in video. Int. J. Comput. Vis. 74(1), 17–31 (2007)

    Article  Google Scholar 

  5. Bruce, N., Tsotsos, J.: Saliency based on information maximization. In: NIPS, vol. 18, p. 155 (2006)

    Google Scholar 

  6. Cheng, M.M., Zhang, G.X., Mitra, N.J., Huang, X., Hu, S.M.: Global contrast based salient region detection. In: CVPR, pp. 409–416 (2011)

    Chapter  Google Scholar 

  7. Collomosse, J.P., Hall, P.M.: Painterly rendering using image salience. In: Eurographics, 2002, pp. 122–128 (2002)

    Google Scholar 

  8. Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The Pascal visual object classes (VOC) challenge. Int. J. Comput. Vis. 88(2), 303–338 (2010)

    Article  Google Scholar 

  9. Goferman, S., Zelnik-Manor, L., Tal, A.: Context-aware saliency detection. In: CVPR, pp. 2376–2383 (2010)

    Google Scholar 

  10. Guo, C., Ma, Q., Zhang, L.: Spatio-temporal saliency detection using phase spectrum of quaternion Fourier transform. In: CVPR, pp. 1–8 (2008)

    Google Scholar 

  11. Harel, J., Koch, C., Perona, P.: Graph-based visual saliency. In: NIPS, vol. 19, p. 545 (2007)

    Google Scholar 

  12. Hays, J., Essa, I.: Image and video based painterly animation. In: NPAR, pp. 113–120 (2004)

    Chapter  Google Scholar 

  13. Hou, X., Zhang, L.: Saliency detection: a spectral residual approach. In: CVPR, pp. 1–8 (2007)

    Google Scholar 

  14. Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. In: PAMI, pp. 1254–1259 (1998)

    Google Scholar 

  15. Judd, T., Ehinger, K., Durand, F., Torralba, A.: Learning to predict where humans look. In: ICCV, pp. 2106–2113 (2009)

    Google Scholar 

  16. Liu, T., Sun, J., Zheng, N.N., Tang, X., Shum, H.Y.: Learning to detect a salient object. In: CVPR (2007)

    Google Scholar 

  17. Prinzmetal, W.: Visual feature integration in a world of objects. Curr. Dir. Psychol. Sci. 4(3), 90–94 (1995)

    Article  Google Scholar 

  18. Rubinstein, M., Shamir, A., Avidan, S.: Multi-operator media retargeting. TOG 28(3) (2009)

  19. Walther, D., Koch, C.: Modeling attention to salient proto-objects. Neural Netw. 19(9), 1395–1407 (2006)

    Article  MATH  Google Scholar 

  20. Yeshurun, Y., Kimchi, R., Sha’shoua, G., Carmel, T.: Perceptual objects capture attention. Vis. Res. 49(10), 1329–1335 (2009)

    Article  Google Scholar 

  21. Zeng, K., Zhao, M., Xiong, C., Zhu, S.C.: From image parsing to painterly rendering. TOG 29(1) (2009)

  22. Zhang, G., Cheng, M.M., Hu, S.M., Martin, R.R.: A shape-preserving approach to image resizing. Comput. Graph. Forum 28(7), 1897–1906 (2009)

    Article  Google Scholar 

Download references

Acknowledgements

This research was supported in part by Intel, the Ollendorf foundation, the Israel Ministry of Science, and by the Israel Science Foundation under Grant 1179/11.

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, Technion—Israel Institute of Technology, Haifa, Israel

    Ran Margolin, Lihi Zelnik-Manor & Ayellet Tal

Authors
  1. Ran Margolin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lihi Zelnik-Manor
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ayellet Tal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ran Margolin.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Margolin, R., Zelnik-Manor, L. & Tal, A. Saliency for image manipulation. Vis Comput 29, 381–392 (2013). https://doi.org/10.1007/s00371-012-0740-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-012-0740-x

Keywords

Search

Navigation