Skip to main content
SpringerLink
Log in

Saliency bagging: a novel framework for robust salient object detection

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Salient object detection is a challenging task, and several methods have been proposed for the same in the literature. The problem lies in that most of the methods perform good on a particular set of images but fail when exposed to a variety of different set of images. Here, we address this problem by proposing a novel framework called saliency bagging for detecting salient object(s) in digital images across a variety of images in a robust manner. The proposed framework generates the saliency map of an image in three phases: (i) Selection of existing saliency detection models and generation of initial saliency maps (ii) Generation of integrated binary map from the initial saliency maps by applying adaptive thresholding and majority voting (iii) Computation of final saliency map using integrated binary map and initial saliency maps by applying proposed integration logic. Extensive experiments on six publicly available datasets viz. MSRA10K, DUT-OMRON, ECSSD, PASCAL-S, SED2, and THUR15K have been performed to determine the effectiveness of the proposed method. The performance of the proposed method is measured in terms of Precision, Recall, F-Measure, Mean Absolute Error (MAE) and Receiver Operating Characteristic (ROC) curve and compared with 25 state-of-the-art methods including 17 classic best-performing methods of the last decade, five existing selected, and three aggregation saliency methods. The proposed method outperforms all the compared classic and existing selected methods in terms of Precision, F-Measure, and MAE, while it is comparable to the best-performing methods in terms of Recall and ROC curve across all the six datasets. The proposed framework is computationally very fast than all compared aggregation methods, while performance is almost same on all datasets that support its superiority.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Achanta, R., Hemami, S., Estrada, F., Susstrunk, S.: Frequency-tuned salient region detection. In: IEEE Conference on Computer Vision and Pattern Recognition, 2009. CVPR 2009. pp. 1597–1604. IEEE (2009)

  2. Alpert, S., Galun, M., Basri, R., Brandt, A.: Image segmentation by probabilistic bottom-up aggregation and cue integration. In: IEEE Conference on Computer Vision and Pattern Recognition, 2007. CVPR\(\backslash \)’07. pp. 1–8. IEEE (2007)

  3. Borji, A., Cheng, M.M., Jiang, H., Li, J.: Salient object detection: a benchmark. IEEE Trans. Image. Process. 12(24), 5706–5722 (2015)

    Article  MathSciNet  Google Scholar 

  4. Borji, A., Sihite, D.N., Itti, L.: Salient object detection: a benchmark. In: Proceedings of the 12th European Conference on Computer Vision-Volume Part II, pp. 414–429. Springer (2012)

  5. Breiman, L.: Bagging predictors. Mach. Learn. 24(2), 123–140 (1996)

    MATH  Google Scholar 

  6. Chen, T., Cheng, M.M., Tan, P., Shamir, A., Hu, S.M.: Sketch2photo: internet image montage. ACM Trans. Graph. (TOG) 28(5), 124 (2009)

    Google Scholar 

  7. Cheng, M.M., Mitra, N.J., Huang, X., Hu, S.M.: Salientshape: group saliency in image collections. Vis. Comput. 30(4), 443–453 (2014)

    Article  Google Scholar 

  8. Cheng, M.M., Mitra, N.J., Huang, X., Torr, P.H., Hu, S.M.: Global contrast based salient region detection. IEEE Trans. Pattern Anal. Mach. Intell. 37(3), 569–582 (2015)

    Article  Google Scholar 

  9. Chia, A.Y.S., Zhuo, S., Gupta, R.K., Tai, Y.W., Cho, S.Y., Tan, P., Lin, S.: Semantic colorization with internet images. ACM Trans. Gr. (TOG) 30(6), 156 (2011)

    Google Scholar 

  10. Donoser, M., Urschler, M., Hirzer, M., Bischof, H.: Saliency driven total variation segmentation. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 817–824. IEEE (2009)

  11. Duan, L., Wu, C., Miao, J., Qing, L., Fu, Y.: Visual saliency detection by spatially weighted dissimilarity. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 473–480. IEEE (2011)

  12. Erdem, E., Erdem, A.: Visual saliency estimation by nonlinearly integrating features using region covariances. J. Vis. 13(4), 11–11 (2013)

    Article  Google Scholar 

  13. GOFERMAN, S.: Context-aware saliency detection. In: Proceedings of IEEE Conference Computer Vision and Pattern Recognition, 2010, pp. 2376–2383 (2010)

  14. Harel, J., Koch, C., Perona, P.: Graph-based visual saliency. In: Advances in Neural Information Processing Systems, pp. 545–552 (2007)

  15. He, S., Lau, R.W., Liu, W., Huang, Z., Yang, Q.: Supercnn: a superpixelwise convolutional neural network for salient object detection. Int. J. Comput. Vis. 115(3), 330–344 (2015)

    Article  MathSciNet  Google Scholar 

  16. Hou, X., Zhang, L.: Saliency detection: a spectral residual approach. In: IEEE Conference on Computer Vision and Pattern Recognition, 2007. CVPR’07. pp. 1–8. IEEE (2007)

  17. Huang, F., Qi, J., Lu, H., Zhang, L., Ruan, X.: Salient object detection via multiple instance learning. IEEE Trans. Image Process. 26(4), 1911–1922 (2017)

    Article  MathSciNet  Google Scholar 

  18. Itti, L., Koch, C.: Computational modelling of visual attention. Nat. Rev. Neurosci. 2(3), 194 (2001)

    Article  Google Scholar 

  19. Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20(11), 1254–1259 (1998)

    Article  Google Scholar 

  20. Jiang, B., Zhang, L., Lu, H., Yang, C., Yang, M.H.: Saliency detection via absorbing markov chain. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1665–1672 (2013)

  21. Jiang, H., Wang, J., Yuan, Z., Wu, Y., Zheng, N., Li, S.: Salient object detection: a discriminative regional feature integration approach. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2083–2090 (2013)

  22. Jiang, P., Ling, H., Yu, J., Peng, J.: Salient region detection by ufo: uniqueness, focusness and objectness. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1976–1983 (2013)

  23. Kanan, C., Tong, M.H., Zhang, L., Cottrell, G.W.: Sun: top-down saliency using natural statistics. Vis. Cognit. 17(6–7), 979–1003 (2009)

    Article  Google Scholar 

  24. Klein, D.A., Frintrop, S.: Center-surround divergence of feature statistics for salient object detection. In: 2011 International Conference on Computer Vision, pp. 2214–2219. IEEE (2011)

  25. Le Meur, O., Liu, Z.: Saliency aggregation: Does unity make strength? In: Asian Conference on Computer Vision, pp. 18–32. Springer (2014)

  26. Li, G., Xie, Y., Lin, L., Yu, Y.: Instance-level salient object segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2386–2395 (2017)

  27. Li, G., Yu, Y.: Visual saliency based on multiscale deep features. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5455–5463 (2015)

  28. Li, H., Lu, H., Lin, Z., Shen, X., Price, B.: Inner and inter label propagation: salient object detection in the wild. IEEE Trans. Image Process. 24(10), 3176–3186 (2015)

    Article  MathSciNet  Google Scholar 

  29. Li, R., Cai, J., Zhang, H., Wang, T.: Aggregating complementary boundary contrast with smoothing for salient region detection. Vis. Comput. 33(9), 1155–1167 (2017)

    Article  Google Scholar 

  30. Li, X., Lu, H., Zhang, L., Ruan, X., Yang, M.H.: Saliency detection via dense and sparse reconstruction. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2976–2983 (2013)

  31. Li, X., Zhao, L., Wei, L., Yang, M.H., Wu, F., Zhuang, Y., Ling, H., Wang, J.: Deepsaliency: multi-task deep neural network model for salient object detection. IEEE Trans. Image Process. 25(8), 3919–3930 (2016)

    Article  MathSciNet  Google Scholar 

  32. Li, Y., Hou, X., Koch, C., Rehg, J.M., Yuille, A.L.: The secrets of salient object segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 280–287 (2014)

  33. Liu, G.H., Yang, J.Y.: Exploiting color volume and color difference for salient region detection. IEEE Trans. Image Process. 28(1), 6–16 (2019)

    Article  MathSciNet  Google Scholar 

  34. Liu, H., Zhang, L., Huang, H.: Web-image driven best views of 3d shapes. Vis. Comput. 28(3), 279–287 (2012)

    Article  Google Scholar 

  35. Liu, T., Sun, J., Zheng, N.N., Tang, X., Shum, H.Y.: Learning to detect a salient object. In: IEEE Conference on Computer Vision and Pattern Recognition, 2007. CVPR’07. pp. 1–8. IEEE (2007)

  36. Liu, T., Yuan, Z., Sun, J., Wang, J., Zheng, N., Tang, X., Shum, H.Y.: Learning to detect a salient object. IEEE Trans. Pattern Anal. Mach. Intell. 33(2), 353–367 (2011)

    Article  Google Scholar 

  37. Lu, Y., Zhou, K., Wu, X., Gong, P.: A novel multi-graph framework for salient object detection. Vis. Comput. (2019). https://doi.org/10.1007/s00371-019-01637-2

    Article  Google Scholar 

  38. Mai, L., Niu, Y., Liu, F.: Saliency aggregation: A data-driven approach. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1131–1138 (2013)

  39. Margolin, R., Tal, A., Zelnik-Manor, L.: What makes a patch distinct? In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1139–1146 (2013)

  40. Murray, N., Vanrell, M., Otazu, X., Parraga, C.A.: Saliency estimation using a non-parametric low-level vision model. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 433–440. IEEE (2011)

  41. Ninassi, A., Le Meur, O., Le Callet, P., Barba, D.: Does where you gaze on an image affect your perception of quality? Applying visual attention to image quality metric. In: IEEE International Conference on Image Processing, 2007. ICIP 2007. vol. 2, pp. II–169. IEEE (2007)

  42. Perazzi, F., Krähenbühl, P., Pritch, Y., Hornung, A.: Saliency filters: contrast based filtering for salient region detection. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 733–740. IEEE (2012)

  43. Qin, Y., Feng, M., Lu, H., Cottrell, G.W.: Hierarchical cellular automata for visual saliency. Int. J. Comput. Vis. 126(7), 751–770 (2018)

    Article  MathSciNet  Google Scholar 

  44. Rahtu, E., Kannala, J., Salo, M., Heikkilä, J.: Segmenting salient objects from images and videos. In: Computer Vision–European Conference on Computer Vision 2010, pp. 366–379. Springer (2010)

  45. Rutishauser, U., Walther, D., Koch, C., Perona, P.: Is bottom-up attention useful for object recognition? In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004. vol. 2, pp. II–II. IEEE (2004)

  46. Seo, H.J., Milanfar, P.: Static and space-time visual saliency detection by self-resemblance. J. Vis. 9(12), 15–15 (2009)

    Article  Google Scholar 

  47. Shi, J., Yan, Q., Xu, L., Jia, J.: Hierarchical image saliency detection on extended cssd. IEEE Trans. Pattern Anal. Mach. Intell. 38(4), 717–729 (2016)

    Article  Google Scholar 

  48. Stalder, S., Grabner, H., Van Gool, L.: Dynamic objectness for adaptive tracking. In: Asian Conference on Computer Vision, pp. 43–56. Springer (2012)

  49. Tavakoli, H.R., Rahtu, E., Heikkilä, J.: Fast and efficient saliency detection using sparse sampling and kernel density estimation. In: Scandinavian Conference on Image Analysis, pp. 666–675. Springer (2011)

  50. Tu, W.C., He, S., Yang, Q., Chien, S.Y.: Real-time salient object detection with a minimum spanning tree. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2334–2342 (2016)

  51. Wang, L., Wang, L., Lu, H., Zhang, P., Ruan, X.: Salient object detection with recurrent fully convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 41(7), 1734–1746 (2018)

    Article  Google Scholar 

  52. Wei, Y., Wen, F., Zhu, W., Sun, J.: Geodesic saliency using background priors. In: European Conference on Computer Vision, pp. 29–42. Springer (2012)

  53. Xu, Y., Hong, X., Porikli, F., Liu, X., Chen, J., Zhao, G.: Saliency integration: an arbitrator model. IEEE Trans. Multimed. 21(1), 98–113 (2018)

    Article  Google Scholar 

  54. Yan, Q., Xu, L., Shi, J., Jia, J.: Hierarchical saliency detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1155–1162 (2013)

  55. Yang, C., Zhang, L., Lu, H., Ruan, X., Yang, M.H.: Saliency detection via graph-based manifold ranking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3166–3173 (2013)

  56. Yang, J., Yang, M.H.: Top-down visual saliency via joint crf and dictionary learning. IEEE Trans. Pattern Anal. Mach. Intell. 39(3), 576–588 (2016)

    Article  Google Scholar 

  57. Yuan, Y., Li, C., Kim, J., Cai, W., Feng, D.D.: Reversion correction and regularized random walk ranking for saliency detection. IEEE Trans. Image Process. 27(3), 1311–1322 (2018)

    Article  MathSciNet  Google Scholar 

  58. Zhang, J., Sclaroff, S.: Exploiting surroundedness for saliency detection: a boolean map approach. IEEE Trans. Pattern Anal. Mach. Intell. 38(5), 889–902 (2016)

    Article  Google Scholar 

  59. Zhang, J., Sclaroff, S., Lin, Z., Shen, X., Price, B., Mech, R.: Minimum barrier salient object detection at 80 fps. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1404–1412 (2015)

  60. Zhang, L., Tong, M.H., Marks, T.K., Shan, H., Cottrell, G.W.: Sun: a bayesian framework for saliency using natural statistics. J. Vis. 8(7), 32–32 (2008)

    Article  Google Scholar 

  61. Zhao, R., Ouyang, W., Li, H., Wang, X.: Saliency detection by multi-context deep learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1265–1274 (2015)

  62. Zhu, W., Liang, S., Wei, Y., Sun, J.: Saliency optimization from robust background detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2814–2821 (2014)

Download references

Author information

Authors and Affiliations

  1. National Institute of Technology Uttarakhand, Uttarakhand, India

    Vivek Kumar Singh & Nitin Kumar

Authors
  1. Vivek Kumar Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nitin Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vivek Kumar Singh.

Ethics declarations

Conflict of interest

All the authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Singh, V.K., Kumar, N. Saliency bagging: a novel framework for robust salient object detection. Vis Comput 36, 1423–1441 (2020). https://doi.org/10.1007/s00371-019-01750-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-019-01750-2

Keywords

Search

Navigation