Skip to main content
SpringerLink
Log in

Approximately decoupled component supervision for salient object detection

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Salient object detection (SOD) aims to find the most attractive object(s) in a scene. In recent years, SOD methods based on deep learning have become the mainstream. Existing methods mostly aggregate the multi-level features extracted by convolutional neural network (CNN) to model the object, or refine the boundary details of the object through multi-scale feature fusion. In this paper, the paradigm of approximately decoupled component supervision is proposed for SOD. Our insight is that the attractive performance of SOD requires explicit modeling of the body and edge of the object with different supervisions. Specifically, we first capture image features through foreground attention (FA) mechanism, cross-block semantic correlation aggregation (CBSC), and resolution-based feature integration (RFI) to make object parts more consistent and complete. Then the detailed edge is obtained by subtracting the body part from the complete mask. By explicitly sampling the body and edge pixels of the salient object, we further optimize the resulting body features and the residual edge features under the supervision of approximate decoupling. Benefiting from the abundant edge information and accurate location information, the framework with various backbone proposed by us can achieve better internal consistency and accurate boundaries of the object. Experimental results on six widely used benchmark datasets demonstrate the superiority and competitiveness of our approach in terms of four popular evaluation metrics. Moreover, the proposed method is an end-to-end saliency detection network without any pre-processing or post-processing.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. He J, Feng F, Liu X, Cheng T, Lin T, Chung H, Chang S (2012) Mobile product search with bag of hash bits and boundary reranking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3005–3012

  2. Liu G, Fan D (2013) A model of visual attention for natural image retrieval. In: Proceedings of the IEEE Conference on information science and cloud computing companion, pp 728–733

  3. Hong S, You T, Kwak S, Han B (2015) Online tracking by learning discriminative saliency map with convolutional neural network. In: International conference on machine learning

  4. Zhang D, Meng D, Zhao L, Han J (2016) Bridging saliency detection to weakly supervised object detection based on selfpaced curriculum learning. In: International joint conferences on artificial intelligence

  5. Zhang S, He F (2020) DRCDN: Learning deep residual convolutional dehazing network. The Visual Computer 36:1797–1808

    Article  Google Scholar 

  6. Chen Y, He F, Li H, Zhang D, Wu Y (2020) A full migration BBO algorithm with enhanced population quality bounds for multimodal biomedical image registration. Applied Soft Computing 93:1–9

    Article  Google Scholar 

  7. Jia F, Guan J, Qi S (2020) A mix-supervised unified framework for salient object detection. Applied Intelligence 50:2945–2958

    Article  Google Scholar 

  8. Jiao J, Xue H, Ding J (2021) Non-local duplicate pooling network for salient object detection. Applied Intelligence

  9. Liu J, Hou Q, Cheng M, Feng J, Jiang J (2019) A simple pooling-based design for real-time salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3917–3926

  10. Zhao X, Pang Y, Zhang L, Lu H, Zhang L (2020) Suppress and balance: A simple gated network for salient object detection. In: Proceedings of the european conference on computer vision

  11. Feng M, Lu H, Ding E (2019) Attentive feedback network for boundary-aware salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1623–1632

  12. Zhang P, Wang D, Lu H, Wang H, Ruan X (2017) Amulet: aggregating multi-level convolutional features for salient object detection. In: Proceedings of the IEEE international conference on computer vision, pp 202–211

  13. Wang B, Chen Q, Zhou M, Zhang Z, Jin X, Gai K (2020) Progressive feature polishing network for salient object detection. In: Processings of the international joint conference on artificial intelligence

  14. Qin X, Zhang Z, Huang C, Gao C, Dehghan M, Jagersand M (2019) BASNet: boundary-aware salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7479–7489

  15. Zhang L, Dai J, Lu H, He Y, Wang G (2018) A bi-directional message passing model for salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1741–1750

  16. Zhao T, Wu X (2019) Pyramid feature attention network for saliency detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  17. Li T, Song H, Zhang K, Liu Q (2020) Learning residual refinement network with semantic context representation for real-time saliency object detection. Pattern Recognition

  18. Wang T, Zhang L, Wang S, Lu H, Yang G, Ruan X, Borji A (2018) Detect globally, re-fine locally: a novel approach to saliency detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3127–3135

  19. Liu N, Han J, Yang M (2018) Picanet: Learning pixel-wise contextual attention for saliency detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3089–3098

  20. Zhang L, Zhang J, Lin Z, Lu H, He Y (2019) CapSal: Leveraging captioning to boost semantics for salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6024–6033

  21. Chen Z, Xu Q, Cong R, Huang Q (2020) Global context-aware progressive aggregation network for salient object detection. In: Proceedings of the international joint conference on artificial intelligence

  22. Li X, Li X, Zhang L, Cheng G, Shi J (2020) Improving semantic segmentation via decoupled body and edge supervision. In: Proceedings of the european conference on computer vision

  23. Wei J, Wang S, Wu Z, Su C, Huang Q, Tian Q (2020) Label decoupling framework for salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  24. Wang W, Lai Q, Fu H, Shen J, Ling H, Yang R (2021) Salient object detection in the deep learning era: an in-depth survey. IEEE Trans Pattern Anal Mach Intell

  25. Fan D, Zhang J, Xu G, Cheng M-M, Shao L (2021) Salient objects in clutter. IEEE Trans Pattern Anal Mach Intell

  26. Xie S, Tu Z (2015) Holistically-nested edge detection. In: Proceedings of the IEEE international conference on computer vision, pp 1395–1403

  27. Hou Q, Cheng M, Hu X, Borji A, Tu Z, Torr P (2017) Deeply supervised salient object detection with short connections. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3203–3212

  28. Wu Z, Su L, Huang Q (2019) Cascaded partial decoder for fast and accurate salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3907–3916

  29. Wei J, Wang S, Huang Q (2020) \({\text F}^{3}Net\): Fusion, feedback and focus for salient object detection. In: Proceedings of the 34th international joint conference on artificial intelligence

  30. Wang T, Borji A, Zhang L, Zhang P, Lu H (2017) A stage-wise refinement model for detecting salient objects in images. In: Proceedings of the IEEE international conference on computer vision, pp 4039–4048

  31. Zhang X, Wang T, Qi J, Lu H, Wang G (2018) Progressive attention guided recurrent network for salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 714–722

  32. Wu Z, Su L, Huang Q (2019) Stacked cross refinement network for edge-aware salient object detection. In: Proceedings of the IEEE international conference on computer vision, pp 7263–7273

  33. Wang X, Ma H, Chen X (2016) Salient object detection via fast r-cnn and low-level cues. In: International conference on image processing, pp 1042–1046

  34. Wang X, Ma H, Chen X, You S (2018) Edge preserving and multi-scale contextual neural network for salient object detection. IEEE Transactions on image processing 27:121–134

    Article  MathSciNet  MATH  Google Scholar 

  35. Girshick R (2015) Fast R-CNN. In: Proceedings of the IEEE international conference on computer vision

  36. Luo Z, Mishra A, Achkar A, Eichel J, Li S, Jodoin P (2017) Non-local deep features for salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  37. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556

  38. Zhao J, Liu J, Fan D, Cao Y, Yang J, Cheng M (2019) EGNet: Edge guidance network for salient object detection. In: Proceedings of the IEEE international conference on computer vision

  39. Jake B (2006) Notes on convolutional neural network. In: neural nets

  40. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7132–7141

  41. Woo S, Park J, Lee J, Kweon I (2018) CBAM: Convolutional block attention module. In: Proceedings of the european conference on computer vision

  42. Su J, Li J, Zhang Y, Xia C, Tian Y (2019) Selectivity or Invariance: Boundary-aware salient object detection. In: Proceedings of the IEEE international conference on computer vision

  43. He K, Zhang X, Ren S, Sun J (2015) Deep residual learning for image recognition. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 770–778

  44. Krizhevsky A, Sutskever I, Hinton G E (2012) Imagenet classification with deep convolutional neural networks. In: NIPS, pp 1097–1105

  45. Movahedi V, Elder J (2010) Design and perceptual validation of performance measures for salient object segmentation. In: 2010 IEEE Computer society conference on computer vision and pattern recognition-workshops (CVPR-Workshops), pp 49–56

  46. Yan Q, Xu L, Shi J, Jia J (2013) Hierarchical saliency detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1155–1162

  47. Yang C, Zhang L, Lu H, Ruan X, Yang M (2013) Saliency detection via graph-based manifold ranking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3166-3173

  48. Li G, Yu Y (2015) Visual saliency based on multiscale deep features. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5455-5463

  49. Li Y, Hou X, Koch C, Rehg J, Yuille A (2014) The secrets of salient object segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 280–287

  50. Wang L, Lu H, Wang Y, Feng M, Wang D, Yin B, Ruan X (2017) Learning to detect salient objects with image-level supervision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 136–145

  51. Achanta R, Hemami S, Estrada F, Susstrunk S (2009) Frequency-tuned salient region detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1597–1604

  52. Perazzi F, Krahenbuhl P, Pritch Y, Hornung A (2012) Saliency filters: Contrast based filtering for salient region detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 733–740

  53. Fan D, Cheng M, Liu Y, Li T, Borji A (2017) Structure-measure: a new way to evaluate foreground maps. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4548–4557

  54. Deng J, Dong W, Socher R, Li L.-J, Li K, and Li F.-F (2009) Imagenet: A large-scale hierarchical image database. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 248–255

  55. Sun Y, Chen G, Zhou T, Zhang Y, Liu N (2021) Context-aware cross-level fusion network for camouflaged object detection. In: Processings of the international joint conference on artificial intelligence

  56. Zhang P, Wang D, Lu H, Wang H, Yin B (2017) Learning uncertain convolutional features for accurate saliency detection. In: Proceedings of the IEEE international conference on computer vision, pp 212–221

  57. Li X, Yang F, Cheng H, Liu W, Shen D (2018) Contour knowledge transfer for salient object detection. In: Proceedings of the european conference on computer vision, pp 370–385

  58. Wu R, Feng M, Guan W, Wang D, Lu H, Ding E (2019) A mutual learning method for salient object detection with intertwined multi-supervision. In: Proceedings of the IEEE international conference on computer vision

  59. Pang Y, Zhao X, Zhang L, Lu H (2020) Multi-scale interactive network for salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  60. Gao S, Tan Y, Cheng M, Lu C, Chen Y, Yan S (2020) Highly efficient salient object detection with 100k parameters. In: Proceedings european conference on computer vision

  61. Zhou H, Xie X, Lai J, Chen Z, Yang L (2020) Interactive two-stream decoder for accurate and fast saliency detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 9141–9150

  62. Feng M, Lu H, Yu Y (2020) Residual learning for salient object detection. IEEE Transactions on Image Processing 29:4696–4708

    Article  MATH  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China under Grant (No. 61872164).

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Jilin University, 130012, Changchun, China

    Yanhua Liang, Guihe Qin, Minghui Sun, Jie Yan & Zhonghan Zhang

  2. Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University, 130012, Changchun, China

    Yanhua Liang, Guihe Qin, Minghui Sun, Jie Yan & Zhonghan Zhang

Authors
  1. Yanhua Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guihe Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Minghui Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jie Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhonghan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guihe Qin.

Ethics declarations

Conflicts of interest

The authors declared that they have no conflicts of interest to this work.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liang, Y., Qin, G., Sun, M. et al. Approximately decoupled component supervision for salient object detection. Appl Intell 52, 16117–16137 (2022). https://doi.org/10.1007/s10489-021-03046-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-03046-2

Keywords

Search

Navigation