Skip to main content

Advertisement

SpringerLink
Log in

A New Framework for Multiscale Saliency Detection Based on Image Patches

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

In this paper, we propose a new multiscale saliency detection algorithm based on image patches. To measure saliency of pixels in a given image, we segment the image into patches by a fixed scale and then use principal component analysis to reduce the dimensions which are noises with respect to the saliency calculation. The dissimilarities between a patch and other patches, which indicate the patch’s saliency, are computed based on the dissimilarity of colors and the spatial distance. Finally, we implement our algorithm through multiple scales that further decrease the saliency of background. Our method is compared with other saliency detection approaches on two public image datasets. Experimental results show that our method outperforms the state-of-the-art methods on predicting human fixations and salient object segmentation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Santella A, Agrawala M, Decarlo D, Salesin D, Cohen M (2006) Gaze-based interaction for semi-automatic photo cropping. In: ACM human factors in computing systems (CHI). ACM, Montreal, pp 771–780

  2. Chen L, Xie X, Fan X, Ma W, Shang H, Zhou H (2002) H A visual attention mode for adapting images on small displays. Technical report, Microsoft Research, Redmond, WA

  3. Itti L (2000) Models of bottom-up and top-down visual attention. PhD thesis, California Institute of Technology, Pasadena

  4. Navalpakkam V, Itti L (2006) An integrated model of top-down and bottom-up attention for optimizing detection speed. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 2049–2056

  5. Rutishauser U, Walther D, Koch C, Perona P (2004) Is bottom-up attention useful for object recognition? In: IEEE conference on computer vision and pattern recognition (CVPR), pp 37–44

  6. Jong Seo H, Milanfar P (2009) Nonparametric bottom-up saliency detection by self-resemblance. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 45–52

  7. Baldi P, Itti L (2010) Of bits and wows: a Bayesian theory of surprise with applications to attention. Neural Netw 23(5):649–666

    Article  Google Scholar 

  8. Torralba A, Oliva A, Castelhano M, Henderson J (2006) Contextual guidance of eye movements and attention in real-world scenes: the role of global features on object search. Psychol Rev 113(4):766–786

    Article  Google Scholar 

  9. Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell 20(11):1254–1259

    Article  Google Scholar 

  10. Hou X, Zhang L (2008) Saliency detection: a spectral residual approach. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 1–8

  11. Guo C, Ma Q, Zhang L (2008) Spatio-temporal saliency detection using phase spectrum of quaternion Fourier transform. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 1–8

  12. Ma Y-F, Zhang H (2003) Contrast-based image attention analysis by using fuzzy growing. In: ACM multimedia, pp 374–381

  13. Itti L, Baldi P (2005) Bayesian surprise attracts human attention. In: NIPS, Cambridge

  14. Liu T, Sun J, Zheng N, Tang X, Shum H-Y (2007) Learning to detect a salient object. In: CVPR

  15. Goferman S, Zelnik-Manor L, Tal A (2010) Context-aware saliency detection. In: CVPR, pp 2376–2383

  16. Jiang H et al (2011) Automatic salient object segmentation based on context and shape prior. In: BMVC

  17. Achanta R, Hemami S, Estrada F, Süsstrunk S (2009) Frequency-tuned salient region detection. IEEE conference on computer vision and pattern recognition (CVPR), pp 1597–1604

  18. Cheng M et al (2011) Global contrast based salient region detection. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 409–416

  19. Duan L, Wu C, Miao J, Qing L, Fu Y (2011) Visual saliency detection by spatially weighted dissimilarity. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 21–23

  20. Tatler BW (2007) The central fixation bias in scene viewing: selecting an optimal viewing position independently of motor biases and image feature distributions. J Vis 7(14):4.1–4.17

    Article  Google Scholar 

  21. Bruce N, Tsotsos J (2006) Saliency based on information maximization. In: Advances in neural information processing systems, pp 155–162

  22. Hou X, Harel J, Koch C (2012) Image signature: highlighting sparse salient regions. IEEE Trans Pattern Anal Mach Intell 34(1):194–201

    Article  Google Scholar 

  23. Yang J, Zhang D, Frangi AF, Yang JY (2004) Two-dimensional PCA: a new approach to appearance-based face representation and recognition. IEEE Trans Pattern Anal Mach Intell 26(1):131–137

    Article  Google Scholar 

  24. Kadir T, Brady M (2001) Saliency, scale and image description. Int J Comput Vis 45(2):83–105

    Article  MATH  Google Scholar 

  25. Hyvärinen A, Hurri J, Hoyer PO (2009) Natural image statistics: a probabilistic approach to early computational vision. Springer, London

    Book  Google Scholar 

  26. Zhang L, Tong M, Marks T, Shan H, Cottrell G (2008) SUN: a Bayesian framework for saliency using natural statistics. J Vis 8(7):1–20

    Article  Google Scholar 

  27. Gao D, Mahadevan V, Vasconcelos N (2008) On the plausibility of the discriminant center-surround hypothesis for visual saliency. J Vis 8(7):1–18

    Article  Google Scholar 

  28. Gao D, Vasconcelos N (2004) Discriminant saliency for visual recognition from cluttered scenes. In: Advances in neural information processing systems, pp 481–488

  29. Harel J, Koch C, Perona P (2007) Graph-based visual saliency. In: Advances in neural information processing systems, pp 545–555

  30. Achanta R, Estrada FJ, Wils P, Süsstrunk S (2008) Salient region detection and segmentation. In: ICVS, pp 66–75

Download references

Acknowledgments

This research is partially sponsored by Natural Science Foundation of China (No. 90820306 and No. KT06015), Natural Science Research Project of Jiang Su Provincial Colleges and Universities (No. 11KJD520003). The authors would like to thank the technical assistance from the PhD student Dongyan Guo.

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Nanjing University of Science and Technology, Nanjing, 210094, China

    Jingbo Zhou & Zhong Jin

Authors
  1. Jingbo Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhong Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jingbo Zhou.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Zhou, J., Jin, Z. A New Framework for Multiscale Saliency Detection Based on Image Patches. Neural Process Lett 38, 361–374 (2013). https://doi.org/10.1007/s11063-012-9276-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-012-9276-3

Keywords

Search

Navigation