Skip to main content

Advertisement

Springer Nature Link
Log in

A multi-source feature extraction network for salient object detection

  • S.i.: Applications and Techniques in Cyber Intelligence (atci2022)
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Salient object detection (SOD) must capture the multi-level features from both global and local view. Furthermore, interiors and boundaries of salient objects must be processed simultaneously in order to generate a clear salient map with sharp boundaries. In addition, the object-part relationship should be taken into consideration to segment the salient object as a whole. To address above issues, we propose a novel multi-source feature extraction network (MFEN), which is capable of integrating salient features, boundary features and global feature, simultaneously. First of all, the multi-source global and local module (MGLM) is introduced to integrate multi-source features, composing of a series of hybrid dilation convolution modules with different dilated rate. Furthermore, the boundary detection module is introduced to predict the boundary map and boundary features, helping for locating the salient object and sharpen edge. In addition, the adjacent features from MGLM are fused progressively to generate the final salient map by feature fusion modules. Experimental results on five datasets demonstrate that our proposed MFEN outperforms recent 18 SOD methods. More importantly, the ablation study shows that the MGLM is an effective feature fusion module for multi-level and multi-source feature.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Explore related subjects

Discover the latest articles and news from researchers in related subjects, suggested using machine learning.

Data availability

The data of this paper can be obtained through the email to the authors.

References

  1. Wang W, Shen J, Yang R, Porikli F (2017) Saliency-aware video object segmentation. IEEE Trans Pattern Anal Mach Intell 40(1):20–33

    Article  Google Scholar 

  2. Zheng L, Wang SJ, Liu ZQ (2015) fast image retrieval: query pruning and early termination. IEEE Transaction Multimed 17(5):648–659

    Article  Google Scholar 

  3. Cheng MM, Hou Q, Zhang S, Rosin PL (2017) Intelligent visual media processing: when graphics meets vision. J Comput Sci Technol 32(1):110–121

    Article  Google Scholar 

  4. Feng MY, Lu HC, Ding E (2019) Attentive feedback network for boundary-aware salient object detection. In: IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), pp 1623–1632

  5. Mahadevan V, Vasconcelos N (2009) Saliency-based discriminant tracking. In: IEEE International Conference on Computer Vision and Pattern Recognition (CVPR)

  6. Tiantian W, Ali B, Lihe Z, Pingping Z, Huchuan L (2017) A stagewise refinement model for detecting salient objects in images. In: IEEE/CVF International Conference on Computer Vision (ICCV)

  7. Changqun X, Jia L, Xiaowu C, Anlin Z, Yu Z (2017) What is and what is not a salient object? learning salient object detector by ensembling linear exemplar regressors. In: IEEE International Conference on Computer Vision and Pattern Recognition (CVPR)

  8. Achanta R, Hemami S, Estrada F, Susstrunk S (2009) Frequency-tuned salient region detection. In: IEEE International Conference on Computer Vision and Pattern Recognition (CVPR)

  9. Zhang J, Yu X, Li AX, Song PP, Liu BW, Dai YC (2020) Weakly-supervised salient object detection via scribble annotations. In: IEEE International Conference on Computer Vision and Pattern Recognition (CVPR)

  10. Gao SH, Cheng MM, Zhao K, Zhang XY, Yang MH, Torr P (2019) Res2Net: a new multi-scale backbone architecture. In: IEEE International Conference on Computer Vision and Pattern Recognition (CVPR)

  11. Wu Z, Su L, Huang Q (2019) Cascaded partial decoder for fast and accurate salient object detection. In: IEEE International Conference on Computer Vision and Pattern Recognition (CVPR)

  12. Liu N, Han J, Yang MH (2020) PiCANet: pixel-wise contextual attention learning for accurate saliency detection. In: IEEE Transactions on Image Processing, p 1

  13. Hou Q, Cheng MM, Hu X, et al (2017) Deeply supervised salient object detection with short connections. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

  14. Jia-Xing Z, Jiang-Jiang L, Deng-Ping F, Yang C, Jufeng Y, Ming-Ming C Egnet: Edge guidance network for salient object detection. In: IEEE/CVF International Conference on Computer Vision (ICCV)

  15. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. In: Comput Science

  16. Fisher Y, Vladlen K (2016) Multi-scale context aggregation by dilated convolutions. In: ICLR

  17. ]Wang X, Girshick R, Gupta A, He K (2017) Non-local neural networks. In: IEEE International Conference on Computer Vision and Pattern Recognition (CVPR)

  18. Wei-Chih T, Shengfeng H, Qingxiong Y, Shao-Yi C (2016) Real-time salient object detection with a minimum spanning tree. In: IEEE International Conference on Computer Vision and Pattern Recognition (CVPR)

  19. Ming-Ming C, Niloy JM, Xiaolei H, Philip HST, Shi-Min H (2015) Global contrast based salient region detection. IEEE TPAMI 37(3):569–582

    Article  Google Scholar 

  20. Borji A, Cheng M-M, Jiang H, Li J (2015) Salient object detection: a benchmark. IEEE TIP 24(12):5706–5722

    MathSciNet  MATH  Google Scholar 

  21. Pingping Z, Dong W, Huchuan L, Hongyu W, Xiang R (2017) Amulet: aggregating multi-level convolutional features for salient object detection. In: IEEE/CVF International Conference on Computer Vision (ICCV), pp 202–211

  22. Wang B, Chen Q, Zhou M (2020) Progressive feature polishing network for salient object detection. In: Accepted by AAAI

  23. Wu R, Feng M, Guan W (2020) A mutual learning method for salient object detection with intertwined multi-supervision [C]. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

  24. Liu JJ, Hou Q, Cheng MM (2019) A simple pooling-based design for real-time salient object detection. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

  25. Su J, Li J, Zhang Y, Xia CT (2019) Selectivity or invariance: boundary-aware salient object detection. In: IEEE/CVF International Conference on Computer Vision (ICCV)

  26. Liu Y, Zhang Q, Zhang D, et al (2019) Employing deep part-object relationships for salient object detection. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV)

  27. Guan W, Wang T, Qi J, Zhang L, Huchuan Lu (2018) Edge-aware convolution neural network based salient object detection. IEEE SPL 26(1):114–118

    Google Scholar 

  28. Xie S, Tu Z (2015) Holistically-nested edge detection. In: IEEE/CVF International Conference on Computer Vision, pp 1395–1403

  29. Zhuge Y, Yang G, Zhang P, Huchuan Lu (2018) Boundary-guided feature aggregation network for salient object detection. IEEE SPL 25(12):1800–1804

    Google Scholar 

  30. Zhiming L, Akshaya KM, Andrew A, Justin AE, Shaozi L, Pierre-Marc J (2017) Non local deep features for salient object detection. In: IEEE International Conference on Computer Vision and Pattern Recognition (CVPR)

  31. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), pp 3431–3440

  32. Liu S, Huang D, Wang Y (2018) Receptive field block net for accurate and fast object detection. In: Proceedings of the European conference on computer vision (ECCV), pp 404–419

  33. Fan DP, Zhai Y, Ali B (2020) BBS-Net: RGB-D salient object detection with a bifurcated backbone strategy network. Accepted in ECCV

  34. Chen Z, Xu Q, Cong R (2020) Global context-aware progressive aggregation network for salient object detection. Proc AAAI Conf Artif Intell 34(7):10599–10606

    Google Scholar 

  35. Long JS et al (2017) Fully convolutional networks for semantic segmentation. In: IEEE Transactions on Pattern Analysis & Machine Intelligence

  36. Qin X, Zhang Z, Huang C (2019) BASNet: Boundary-aware salient object detection. In; 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

  37. Yupei WX (2018) Deep crisp boundaries: from boundaries to higher-level tasks. In: IEEE Transactions on Image Processing A Publication of the IEEE Signal Processing Society

  38. Hu X, Yang K, Fei L, Wang K (2019) Acnet: attention based network to exploit complementary features for RGBD semantic segmentation. In: ICIP, pp 1440–1444

  39. Zijun D, Xiaowei H, Lei Z, Xuemiao X, Jing Q, Guoqiang H, Pheng-Ann H (2018) R3net: Recurrent residual refinement network for saliency detection. In: IJCAI, pp 684–690

  40. Chen Q, Koltun V (2017) Photographic image synthesis with cascaded refinement networks. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 1511–1520

  41. Wang L, Lu H, Zhang P, Ruan X (2018) Salient object detection with recurrent fully convolutional networks. IEEE TPAMI 41(7):1734–1746

    Article  Google Scholar 

  42. Islam, M.A., Kalash, M., Rochan, M., Bruce, N.D., & Wang, Y. (2017). Salient Object Detection using a Context-Aware Refinement Network. In: BMVC.

  43. Md Atiqur R, Yang W (2016) Optimizing intersection-over-union in deep neural networks for image segmentation. In: International Symposium on Visual Computing, Springer, pp 234–244

  44. Gellert M, Wenjie L, Raquel U (2017) Deep-road mapper: Extracting road topology from aerial images. In: IEEE International Conference on Computer Vision (ICCV)

  45. Qiong Y, Li X, Jianping S, Jiaya J (2013) Hierarchical saliency detection. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

  46. Chuan Y, Lihe Z, Huchuan L, Xiang R, Ming-Hsuan Y (2013) Saliency detection via graph-based manifold ranking. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

  47. Lin L, Xiaodi H, Christof K, James MR, Alan LY (2014) The secrets of salient object segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

  48. Guanbin L, Yizhou Y (2015) Visual saliency based on multiscale deep features. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

  49. Lijun W, Huchuan L, Yifan W, Mengyang F, Dong W, Baocai Y, Xiang R (2017) Learning to detect salient objects with image-level supervision. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

  50. Achanta R, Hemami S, Estrada F, Sustrunk S (2009) Frequencytuned salient region detection. In: The IEEE Conference on Computer Vision and Pattern Recognition, pp 1597–1604

  51. Fan D-P, Cheng M-M, Liu Y, Li T, Borji A (2017) Structure measure: a new way to evaluate foreground maps. In: The IEEE International Conference on Computer Vision, pp 4548–4557

  52. Fan D-P, Gong C, Cao Y, Ren B, Cheng M-M, Borji A (2018) Enhanced-alignment measure for binary foreground map evaluation. In: International Joint Conference on Artificial Intelligence, pp 698–704

  53. Nian L, Junwei H, Ming-Hsuan Y (2018) Picanet: Learning pixel-wise contextual attention for saliency detection. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 3089–3098

  54. Tiantian W, Lihe Z, Shuo W, Huchuan L, Gang Y, Xiang R, Ali B (2018) Detect globally, refine locally: a novel approach to saliency detection. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 3127–3135

  55. Xuebin Q, Zichen Z, Chenyang H, Chao G, Masood D, Martin J (2019) Basnet: Boundaryaware salient object detection. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 7479–7489

  56. Wenguan W, Jianbing S, Ming-Ming C, Ling S (2019) An iterative and cooperative top-down and bottom-up inference network for salient object detection. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 5968–5977

  57. Xiaoning Z, Tiantian W, Jinqing Q, Huchuan L, Gang W (2018) Progressive attention guided recurrent network for salient object detection. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 714–722

  58. Xin L, Fan Y, Hong C, Wei L, Dinggang S (2018) Contour knowledge transfer for salient object detection. In: Proceedings of the European conference on computer vision (ECCV), pp 355–370

  59. Yi Z, Pingping Z, Jianming Z, Zhe L, Huchuan L (2019) Towards high-resolution salient object detection. arXiv preprint arXiv:1908.07274

  60. Wenguan W, Shuyang Z, Jianbing S, Steven CHH, Ali B (2019) Salient object detection with pyramid attention and salient edges. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 1448–1457

  61. Pang Y, Zhao X, Zhang L, et al (2020) Multi-scale interactive network for salient object detection. In: IEEE International Conference on Computer Vision and Pattern Recognition (CVPR)

  62. Zhe W, Li S, Qingming H (2019) Stacked cross refinement network for edge-aware salient object detection. In: IEEE/CVF International Conference on Computer Vision (ICCV), pp 7264–7273

  63. Xie Y, Rambach J, Shu F (2021) PlaneSegNet: Fast and robust plane estimation using a single-stage instance segmentation CNN. In: Proceedings of the IEEE International Conference on Robotics and Automation

  64. Jie H, Li S, Gang S (2018) Squeeze-and-excitation networks. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 7132–7141

  65. Sanghyun W, Jongchan P, Joon-Young L, In So K (2018) Cbam: Convolutional block attention module. In Proceedings of the European conference on computer vision (ECCV), pp 3–19

  66. Hou Q, Zhou D, Feng J (2021) Coordinate attention for efficient mobile network design. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

  67. Xiaoqi Z, Youwei P, Lihe Z, Huchuan L, Xiang R (2022) Self-supervised pretraining for RGB-D salient object detection. In: AAAI

Download references

Funding

This work was supported by National Natural Science Foundation of China. (No.62171315).

Author information

Authors and Affiliations

  1. School of Electrical and Information Engineering, Tianjin University, Tianjin, 300000, China

    Kun Xu & Jichang Guo

  2. China Mobile Communications Group, Department of Government and Enterprise, Shandong Co. LTD, Jinan, 250001, Shandong, China

    Kun Xu

Authors
  1. Kun Xu
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Jichang Guo
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Jichang Guo.

Ethics declarations

Conflict of interest

The authors declare that there is no conflict of interest regarding the publication of this work.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, K., Guo, J. A multi-source feature extraction network for salient object detection. Neural Comput & Applic 35, 24727–24742 (2023). https://doi.org/10.1007/s00521-022-08172-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-022-08172-7

Keywords