Skip to main content
SpringerLink
Log in

Robust salient object detection for RGB images

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Recent advances in supervised salient object detection modeling have resulted in significant performance improvements on benchmark dataset. However, most of the existing salient object detection models assume that an image contains at least one salient object. Such an assumption that often leads to their effectiveness may be impaired once they are applied to real-world scenes. To solve the problem, salient object existence prediction designed to judge whether the image contains salient objects is introduced into deep network to learn a better salient object detection model. For dense salient object detection task, high-level semantic feature is progressively hybrid upsampled from deep to shallow to remedy the spatial information loss guided by higher layer feature and saliency existence information. Our model is aware of non-salient image that contains no salient objects at all and thus reduces the false-positive rate. Experimental results show that our model wins similar multi-task model and outperforms state-of-the-art models in robustness and accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al.: Tensorflow: a system for large-scale machine learning. OSDI 16, 265–283 (2016)

    Google Scholar 

  2. Achanta, R., Hemami, S., Estrada, F., Susstrunk, S.: Frequency-tuned salient region detection. In: IEEE Conference on Computer Vision and Pattern Recognition, 2009, CVPR 2009. IEEE, pp. 1597–1604 (2009)

  3. Alpert, S., Galun, M., Basri, R., Brandt, A.: Image segmentation by probabilistic bottom-up aggregation and cue integration. In: IEEE Conference on Computer Vision and Pattern Recognition, 2007. CVPR’07. IEEE, pp. 1–8 (2007)

  4. Borji, A., Cheng, M.M., Jiang, H., Li, J.: Salient object detection: a benchmark. IEEE Trans. Image Process. 24(12), 5706–5722 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  5. Chen, H., Li, Y.: Three-stream attention-aware network for rgb-d salient object detection. IEEE Trans. Image Process. 28(6), 2825–2835 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  6. Chen, H., Li, Y., Su, D.: Multi-modal fusion network with multi-scale multi-path and cross-modal interactions for rgb-d salient object detection. Pattern Recognit. 86, 376–385 (2019)

    Article  Google Scholar 

  7. Cheng, M.M., Mitra, N.J., Huang, X., Torr, P.H., Hu, S.M.: Global contrast based salient region detection. IEEE Trans. Pattern Anal. Mach. Intell. 37(3), 569–582 (2015)

    Article  Google Scholar 

  8. Cholakkal, H., Johnson, J., Rajan, D.: A classifier-guided approach for top-down salient object detection. Signal Process. Image Commun. 45(C), 24–40 (2016)

    Article  Google Scholar 

  9. Deng, Z., Hu, X., Zhu, L., Xu, X., Qin, J., Han, G., Heng, P.A.: R3net: Recurrent residual refinement network for saliency detection. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence, AAAI Press, pp. 684–690 (2018)

  10. Fan, D.P., Cheng, M.M., Liu, Y., Li, T., Borji, A.: Structure-measure: a new way to evaluate foreground maps. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4558–4567 (2017)

  11. Fan, D.P., Cheng, M.M., Liu, J.J., Gao, S.H., Hou, Q., Borji, A.: Salient objects in clutter: bringing salient object detection to the foreground. arXiv preprint arXiv:1803.06091 (2018)

  12. Fan, D.P., Lin, Z., Zhao, J.X., Liu, Y., Zhang, Z., Hou, Q., Zhu, M., Cheng, M.M.: Rethinking rgb-d salient object detection: models, datasets, and large-scale benchmarks. arXiv preprint arXiv:1907.06781 (2019)

  13. Fan, D.P., Wang, W., Cheng, M.M., Shen, J.: Shifting more attention to video salient object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8554–8564 (2019)

  14. He, S., Jiao, J., Zhang, X., Han, G., Lau, R.W.: Delving into salient object subitizing and detection. In: 2017 IEEE International Conference on Computer Vision (ICCV), IEEE, pp. 1059–1067 (2017)

  15. Hou, Q., Cheng, M.M., Hu, X., Borji, A., Tu, Z., Torr, P.: Deeply supervised salient object detection with short connections. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, pp. 5300–5309 (2017)

  16. Hou, Q., Liu, J., Cheng, M.M., Borji, A., Torr, P.H.: Three birds one stone: a unified framework for salient object segmentation, edge detection and skeleton extraction. arXiv preprint arXiv:1803.09860 (2018)

  17. Hou, Q., Cheng, M.M., Hu, X., Borji, A., Tu, Z., Torr, P.: Deeply supervised salient object detection with short connections. In: IEEE TPAMI (2019). https://doi.org/10.1109/TPAMI.2018.2815688

  18. Huang, K., Gao, S.: Image saliency detection via multi-scale iterative CNN. In: The Visual Computer, pp. 1–13 (2019)

  19. Jiang, H., Cheng, M.M., Li, S.J., Borji, A., Wang, J.: Joint salient object detection and existence prediction. Front. Comput. Sci. 13(4), 778–788 (2017)

    Article  Google Scholar 

  20. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  21. Li, G., Yu, Y.: Visual saliency based on multiscale deep features. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5455–5463 (2015)

  22. Li, G., Xie, Y., Lin, L., Yu, Y.: Instance-level salient object segmentation. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, pp. 247–256 (2017)

  23. Li, G., Xie, Y., Lin, L.: Weakly supervised salient object detection using image labels. arXiv preprint arXiv:1803.06503 (2018)

  24. Li, M., Dong, S., Zhang, K., Gao, Z., Wu, X., Zhang, H., Yang, G., Li, S.: Deep learning intra-image and inter-images features for co-saliency detection. In: BMVC, p. 291 (2018)

  25. Li, X., Yang, F., Chen, L., Cai, H.: Saliency transfer: An example-based method for salient object detection. In: IJCAI, pp. 3411–3417 (2016)

  26. Li, X., Zhao, L., Wei, L., Yang, M.H., Wu, F., Zhuang, Y., Ling, H., Wang, J.: Deepsaliency: multi-task deep neural network model for salient object detection. IEEE Trans. Image Process. 25(8), 3919–3930 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  27. Li, X., Yang, F., Cheng, H., Chen, J., Guo, Y., Chen, L.: Multi-scale cascade network for salient object detection. In: Proceedings of the 25th ACM International Conference on Multimedia, ACM, pp. 439–447 (2017)

  28. Li, X., Yang, F., Cheng, H., Liu, W., Shen, D.: Contour knowledge transfer for salient object detection. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 355–370 (2018)

  29. Li, Y., Hou, X., Koch, C., Rehg, J.M., Yuille, A.L.: The secrets of salient object segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 280–287 (2014)

  30. Liu, T., Yuan, Z., Sun, J., Wang, J., Zheng, N., Tang, X., Shum, H.Y.: Learning to detect a salient object. IEEE Trans. Pattern Anal. Mach. Intell. 33(2), 353–367 (2011)

    Article  Google Scholar 

  31. Liu, Z., Shi, S., Duan, Q., Zhang, W., Zhao, P.: Salient object detection for rgb-d image by single stream recurrent convolution neural network. Neurocomputing 363, 46–57 (2019)

    Article  Google Scholar 

  32. Liu, Z., Tang, J., Zhao, P.: Salient object detection via hybrid upsampling and hybrid loss computing. In: The Visual Computer, pp. 1–11 (2019)

  33. Lu, Y., Zhou, K., Wu, X., Gong, P.: A novel multi-graph framework for salient object detection. In: The Visual Computer, pp. 1–17 (2019)

  34. Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Eighth IEEE International Conference on Computer Vision, 2001, ICCV 2001, Proceedings, IEEE, vol. 2, pp. 416–423 (2001)

  35. Mochizuki, I., Toyoura, M., Mao, X.: Visual attention prediction for images with leading line structure. Vis. Comput. 34(6–8), 1031–1041 (2018)

    Article  Google Scholar 

  36. Qin, X., Zhang, Z., Huang, C., Gao, C., Dehghan, M., Jagersand, M.: Basnet: Boundary-aware salient object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7479–7489 (2019)

  37. Shen, J., Peng, J., Shao, L.: Submodular trajectories for better motion segmentation in videos. IEEE Trans. Image Process. 27(6), 2688–2700 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  38. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

  39. Song, H., Wang, W., Zhao, S., Shen, J., Lam, K.M.: Pyramid dilated deeper convlstm for video salient object detection. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 715–731 (2018)

  40. Tang, Y., Tong, R., Min, T., Yun, Z.: Depth incorporating with color improves salient object detection. Vis. Comput. 32(1), 111–121 (2015)

    Article  Google Scholar 

  41. Wang, B., Zhang, T., Wang, X.: Salient object detection based on Laplacian similarity metrics. Vis. Comput. 34(5), 645–658 (2018)

    Article  Google Scholar 

  42. Wang, C., Zha, Z.J., Liu, D., Xie, H.: Robust deep co-saliency detection with group semantic. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 8917–8924 (2019)

  43. Wang, L., Lu, H., Wang, Y., Feng, M., Wang, D., Yin, B., Ruan, X.: Learning to detect salient objects with image-level supervision. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pp. 136–145 (2017)

  44. Wang, N., Gong, X.: Adaptive fusion for rgb-d salient object detection. arXiv preprint arXiv:1901.01369 (2019)

  45. Wang, P., Wang, J., Zeng, G., Feng, J., Zha, H., Li, S.: Salient object detection for searched web images via global saliency. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, IEEE, pp. 3194–3201 (2012)

  46. Wang, W., Shen, J.: Deep visual attention prediction. IEEE Trans. Image Process. 27(5), 2368–2378 (2017)

    Article  MathSciNet  Google Scholar 

  47. Wang, W., Shen, J., Shao, L.: Video salient object detection via fully convolutional networks. IEEE Trans. Image Process. 27(1), 38–49 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  48. Wang, W., Shen, J., Yang, R., Porikli, F.: Saliency-aware video object segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 40(1), 20–33 (2017)

    Article  Google Scholar 

  49. Wang, W., Shen, J., Guo, F., Cheng, M.M., Borji, A.: Revisiting video saliency: a large-scale benchmark and a new model. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4894–4903 (2018)

  50. Wang, W., Shen, J., Ling, H.: A deep network solution for attention and aesthetics aware photo cropping. IEEE Trans. Pattern Anal. Mach. Intell. 41(7), 1531–1544 (2018)

    Article  Google Scholar 

  51. Wei, L., Zhao, S., Bourahla, O.E.F., Li, X., Wu, F., Zhuang, Y.: Deep group-wise fully convolutional network for co-saliency detection with graph propagation. IEEE Trans. Image Process. 28(10), 2052–5063 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  52. Wu, Z., Su. L., Huang, Q.: Cascaded partial decoder for fast and accurate salient object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3907–3916 (2019)

  53. Xie, S., Tu, Z: Holistically-nested edge detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1395–1403 (2015)

  54. Yan, Q., Xu, L., Shi, J., Jia, J.: Hierarchical saliency detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1155–1162 (2013)

  55. Yang, C., Zhang, L., Lu, H., Ruan, X., Yang, M.H.: Saliency detection via graph-based manifold ranking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3166–3173 (2013)

  56. Zhang, J., Ma, S., Sameki, M., Sclaroff, S., Betke, M., Lin, Z., Shen, X., Price, B., Mech, R.: Salient object subitizing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4045–4054 (2015)

  57. Zhang, K., Li, T., Liu, B., Liu, Q.: Co-saliency detection via mask-guided fully convolutional networks with multi-scale label smoothing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3095–3104 (2019)

  58. Zhang, L., Dai, J., Lu, H., He, Y., Wang, G.: A bi-directional message passing model for salient object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1741–1750 (2018)

  59. Zhang, Q., Lin, J., Li, W., Shi, Y., Cao, G.: Salient object detection via compactness and objectness cues. Vis. Comput. 34(4), 473–489 (2018)

    Article  Google Scholar 

  60. Zhao, J., Cao, Y., Fan, D.P., Li, X.Y., Zhang, L., Cheng, M.M.: Contrast prior and fluid pyramid integration for rgbd salient object detection. In: IEEE CVPR, pp. 1–10 (2019)

  61. Zhuge, Y., Yang, G., Zhang, P., Lu, H.: Boundary-guided feature aggregation network for salient object detection. IEEE Signal Process. Lett. 25(12), 1800–1804 (2018)

    Article  Google Scholar 

Download references

Acknowledgements

We thank Ming-Ming Cheng from Nankai University for providing JSOD dataset. We also thank all anonymous reviewers for their valuable comments. This research is supported by National Natural Science Foundation of China (61602004), Natural Science Foundation of Anhui Province (1908085MF182) and University Natural Science Research Project of Anhui Province (KJ2019A0034).

Author information

Authors and Affiliations

  1. Key Laboratory of Intelligent Computing and Signal Processing of Ministry of Education, School of Computer Science and Technology, Anhui University, Hefei, China

    Zhengyi Liu, Qian Xiang, Jiting Tang, Yuan Wang & Peng Zhao

Authors
  1. Zhengyi Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qian Xiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jiting Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yuan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Peng Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhengyi Liu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, Z., Xiang, Q., Tang, J. et al. Robust salient object detection for RGB images. Vis Comput 36, 1823–1835 (2020). https://doi.org/10.1007/s00371-019-01778-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-019-01778-4

Keywords

Search

Navigation