Skip to main content
SpringerLink
Log in

Feature extraction and fusion network for salient object detection

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In the salient object detection (SOD) models based on convolutional neural network (CNN), the high-level semantic features and low-level features of the image are effectively fused and complementary, which can effectively improve the performance of SOD. However, there are usually great differences between high-level semantic features and low-level features, and low-level features are often rich in noise. How to make full use of different features and avoid noise interference is a hot issue for researchers. Different from the traditional methods, this paper proposes a novel feature extraction and fusion network (EFNet). By setting a middle-level feature extraction module as the medium for the fusion of high-level semantic features and low-level image features, this special module integrates the two by reducing the difference between low-level image features and deep semantic features; In addition, a feature enhancement module is applied to enhance the image features, and the proposed SOD method can obtain good performance. Experimental results on five benchmark datasets show that the proposed method outperforms 15 state-of-the-art methods on five important evaluation metrics. Code will be available at: https://github.com/dc3234/EFNet.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Atrish A, Singh N, Kumar K , Kumar V (2017) An automated hierarchical framework for player recognition in sports image. In: Proceedings of the international conference on video and image processing, pp 103–108

  2. Chen Z, Xu Q, Cong R, Huang Q (2020) Global context-aware progressive aggregation network for salient object detection. In: Proceedings of the AAAI conference on artificial intelligence, vol 34, pp 10599–10606

  3. Cheng MM, Zhang FL, Mitra NJ, Huang X, Hu SM (2010) Repfinder: finding approximately repeated scene elements for image editing. ACM transactions on graphics (TOG) 29(4):1–8

    Article  Google Scholar 

  4. Fang C, Tian H, Zhang D, Zhang Q, Han J, Han J (2021) Densely Nested Top-Down Flows for Salient Object Detection. arXiv:2102.09133

  5. Feng M, Lu H, Ding E (2019) Attentive feedback network for boundary-aware salient object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 1623–1632

  6. Gu Y, Wang L, Wang Z, Liu Y, Cheng MM, Lu SP (2020) Pyramid constrained self-attention network for fast video salient object detection. In proceedings of the AAAI conference on artificial intelligence 34(07):10869–10876

  7. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  8. Hou Q, Cheng MM, Hu X, Borji A, Tu Z, Torr PH (2017) Deeply supervised salient object detection with short connections. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3203–3212

  9. Kang D, Park S, Paik J (2020) Sd BAN: Salient object detection using bilateral attention network with dice coefficient loss. IEEE Access 8:104357–104370

    Article  Google Scholar 

  10. Krishna R, Kumar K (2020) P-MEC: polynomial congruence based multimedia encryption technique over cloud. IEEE consumer electronics magazine

  11. Kumar K (2021) Text query based summarized event searching interface system using deep learning over cloud. Multimed Tools Appl 80:11079–11094

    Article  Google Scholar 

  12. Kumar K, Kumar A, Bahuguna A (2017) D-CAD: Deep and crowded anomaly detection. In: Inproceedings of the 7th International Conference on Computer and Communication Technology, pp 100–105

  13. Kumar K, Shrimankar DD (2017) F-DES: Fast and deep event summarization. IEEE Transactions on Multimedia 20(2):323–334

    Article  Google Scholar 

  14. Kumar K, Shrimankar DD, Singh N (2018) Eratosthenes sieve based key-frame extraction technique for event summarization in videos. Multimed Tools Appl 77(6):7383–7404

    Article  Google Scholar 

  15. Kumar K, Shrimankar DD, Singh N (2019) Key-lectures: keyframes extraction in video lectures. In: Machine intelligence and signal analysis. Springer, Singapore, pp 453–459

  16. Kumar A, Singh N, Kumar P, Vijayvergia A, Kumar K (2017) A novel superpixel based color spatial feature for salient object detection. In: 2017 conference on information and communication technology (CICT). IEEE, pp 1–5

  17. Lee H, Kim D (2018, March) Salient region-based online object tracking. In: 2018 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, pp 1170–1177

  18. Li Y, Hou X, Koch C, Rehg JM, Yuille AL (2014) The secrets of salient object segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 280–287

  19. Li G, Yu Y (2015) Visual saliency based on multiscale deep features. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5455–5463

  20. Liang Y, Qin G, Sun M, Yan J, Jiang H (2021) MAFNEt: Multi-style attention fusion network for salient object detection. Neurocomputing 422:22–33

    Article  Google Scholar 

  21. Liu Y, Cheng MM, Zhang XY, Nie GY (2021) DNA: deeply supervised nonlinear aggregation for salient object detection. IEEE Transactions on Cybernetics

  22. Liu N, Han J, Yang MH (2018) Picanet: Learning pixel-wise contextual attention for saliency detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3089–3098

  23. Liu JJ, Hou Q, Cheng MM (2020) Dynamic feature integration for simultaneous detection of salient object, edge, and skeleton. IEEE Trans Image Process 29:8652–8667

    Article  Google Scholar 

  24. Liu JJ, Hou Q, Cheng MM, Feng J, Jiang J (2019) A simple pooling-based design for real-time salient object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3917–3926

  25. Liu Y, Zhang XY, Bian JW, Zhang L, Cheng MM (2021) Samnet: Stereoscopically attentive multi-scale network for lightweight salient object detection. IEEE Trans Image Process 30:3804–3814

    Article  Google Scholar 

  26. Luo W, Li Y, Urtasun R, Zemel R (2016) Understanding the effective receptive field in deep convolutional neural networks. In: Proceedings of the 30th International conference on neural information processing systems, pp 4905–4913

  27. Ma YF, Lu L, Zhang HJ, Li M (2002, December) A user attention model for video summarization. In: Proceedings of the tenth ACM International conference on multimedia, pp 533–542

  28. Mohammadi S, Noori M, Bahri A, Majelan SG, Havaei M (2020) CAGNEt: content-aware guidance for salient object detection. Pattern Recogn 103:107303

    Article  Google Scholar 

  29. Noori M, Mohammadi S, Majelan SG, Bahri A, Havaei M (2020) DFNEt: Discriminative feature extraction and integration network for salient object detection. Eng Appl Artif Intell 89:103419

    Article  Google Scholar 

  30. Pang Y, Zhao X, Zhang L, Lu H (2020) Multi-scale interactive network for salient object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 9413–9422

  31. Qin X, Zhang Z, Huang C, Gao C, Dehghan M, Jagersand M (2019) Basnet: Boundary-aware salient object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 7479–7489

  32. Ravi V, Narasimhan H, Chakraborty C, Pham TD (2021) Deep learning-based meta-classifier approach for COVID-19 classification using CT scan and chest X-ray images

  33. Ravi V, Narasimhan H, Pham TD (2021) EfficientNet-Based Convolutional Neural Networks for Tuberculosis Classification. In: Advances in Artificial Intelligence, Computation, and Data Science. Springer, Cham, pp 227–244

  34. Sharma S, Kumar K (2021) ASL-3DCNN: American sign language recognition technique using 3-D convolutional neural networks. Multimed Tools Appl, 1–13

  35. Sharma S, Kumar K, Singh N (2017) D-FES: Deep facial expression recognition system. In: 2017 conference on information and communication technology (CICT). IEEE, pp 1–6

  36. Sharma S, Kumar K, Singh N (2020) Deep eigen space based asl recognition system. IETE J Res, 1–11

  37. Simonyan K (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556

  38. Wang L, Lu H, Wang Y, Feng M, Wang D, Yin B, Ruan X (2017) Learning to detect salient objects with image-level supervision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 136–145

  39. Wei J, Wang S, Huang Q (2020) F3Net: fusion, Feedback and Focus for Salient Object Detection. In: Proceedings of the AAAI conference on artificial intelligence, vol 34, pp 12321–12328

  40. Wu R, Feng M, Guan W, Wang D, Lu H, Ding E (2019) A mutual learning method for salient object detection with intertwined multi-supervision. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 8150–8159

  41. Wu Z, Su L, Huang Q (2019) Cascaded partial decoder for fast and accurate salient object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3907–3916

  42. Xi T, Zhao W, Wang H, Lin W (2016) Salient object detection with spatiotemporal background priors for video. IEEE Trans Image Process 26(7):3425–3436

    Article  MathSciNet  Google Scholar 

  43. Yan Q, Xu L, Shi J, Jia J (2013) Hierarchical saliency detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1155–1162

  44. Yang C, Zhang L, Lu H, Ruan X, Yang MH (2013) Saliency detection via graph-based manifold ranking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3166–3173

  45. Zhang L, Dai J, Lu H, He Y, Wang G (2018) A bi-directional message passing model for salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1741–1750

  46. Zhang P, Wang D, Lu H, Wang H, Ruan X (2017) Amulet: Aggregating multi-level convolutional features for salient object detection. In: Proceedings of the IEEE International conference on computer vision, pp 202–211

  47. Zhao JX, Liu JJ, Fan DP, Cao Y, Yang J, Cheng MM (2019) EGNEt: edge guidance network for salient object detection. In: Proceedings of the IEEE/CVF International conference on computer vision, pp 8779–8788

  48. Zhao X, Pang Y, Zhang L, Lu H, Zhang L (2020) Suppress and balance: a simple gated network for salient object detection. In: European conference on computer vision. Springer, Cham, pp 35–51

  49. Zinkevich M, Weimer M, Smola AJ, Li L (2010) Parallelized stochastic gradient descent. In: NIPS, vol 4, p 4

Download references

Acknowledgements

This research project was supported by the Natural Science Foundation of Zhejiang Province of China (NO.LY19F030013).

Author information

Authors and Affiliations

  1. China JiLiang University, Hangzhou, 310018, Zhejiang, China

    Chao Dai, Chen Pan & Wei He

Authors
  1. Chao Dai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chen Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wei He
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chen Pan.

Ethics declarations

Conflict of Interests

The authors declare that they have no conflict of interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dai, C., Pan, C. & He, W. Feature extraction and fusion network for salient object detection. Multimed Tools Appl 81, 33955–33969 (2022). https://doi.org/10.1007/s11042-022-12394-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12394-1

Keywords

Search

Navigation