Skip to main content

Advertisement

Springer Nature Link
Log in

Depth scale balance saliency detection with connective feature pyramid and edge guidance

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Convolutional Neural Networks (CNNs) have played an important role in saliency detection. How to detect a salient object as a whole is a key issue. However, most existing learning-based methods are not accurate enough to detect salient objects in complex scenes, such as easily overlooked small salient areas in a whole salient object, which is called scale imbalance problem in this paper. To address this issue, Scale Balance Network (SBN) based on fully convolutional network is proposed to accurately recognize and comprehensively detect salient objects. Firstly, to detect more small salient areas, a specially designed backbone instead of common backbone is adopted in this paper, which can capture larger resolution with more spatial features in deeper layers. Secondly, we present a novel progressive pyramid mechanism named Connective Feature Pyramid Module (CFPM), aiming to make the network focus on the balance between the large salient areas and the small ones. Finally, we present an Edge Enhancement Architecture with Various Kernels (EEAVK) to locate the saliency maps and refine the boundary features. Experimental results on five benchmark datasets show that the proposed SBN method achieves consistently superior performance in comparison with other state-of-the-art ones under different evaluation metrics.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Explore related subjects

Discover the latest articles and news from researchers in related subjects, suggested using machine learning.

References

  1. Jia F, Guan J, Qi S (2020) A mix-supervised unified framework for salient object detection. Appl Intell 50:2945–2958. https://doi.org/10.1007/s10489-020-01700-9

    Article  Google Scholar 

  2. Hou Q, Cheng MM, Hu X, Borji A, Tu Z, Torr PHS (2019) Deeply supervised salient object detection with short connections. IEEE Trans Pattern Anal Mach Intell 41(4):815–828. https://doi.org/10.1109/TPAMI.2018.2815688

    Article  Google Scholar 

  3. Chen S, Tan X, Wang B, Hu X (2018) Reverse attention for salient object detection. In: Proceedings of the European conference on computer vision, pp 234–250. https://doi.org/10.1007/978-3-030-01240-3_15

  4. Liu N, Han J, Yang MH (2020) PiCANet: pixel-wise contextual attention learning for accurate saliency detection. IEEE Trans Image Process 29(99):6438–6451. https://doi.org/10.1109/TIP.2020.2988568

    Article  Google Scholar 

  5. Zhang L, Dai J, Lu H, He Y, Wang G (2018) A bi-directional message passing model for salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1741–1750. https://doi.org/10.1109/CVPR.2018.00187

  6. Li Z, Peng C, Yu G, Zhang X, Deng Y, Sun J (2018) Detnet: a backbone network for object detection. European Conf Comput Vis. arXiv:1804.06215

  7. Yu F, Koltun V (2016) Multi-scale context aggregation by dilated convolutions. Multi-scale context aggregation with dilated convolutions. In: Proceedings of the international conference on learning representations. arXiv:1511.07122

  8. Yang C, Zhang L, Lu H, Ruan X, Yang MH (2013) Saliency detection via graph-based manifold ranking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3166–3173. https://doi.org/10.1109/CVPR.2013.407

  9. Cheng MM, Mitra NJ, Huang X, Torr PH, Hu SM (2014) Global contrast based salient region detection. IEEE Trans Pattern Anal Mach Intell 37(3):569–582. https://doi.org/10.1109/tpami.2014.2345401

    Article  Google Scholar 

  10. Zhang N, Ding S (2016) Unsupervised and semi-supervised extreme learning machine with wavelet kernel for high dimensional data. Memetic Comput 9(2):1–11. https://doi.org/10.1007/s12293-016-0198-x

    MathSciNet  Google Scholar 

  11. Zhang N, Ding S, Zhang J (2018) An overview on restricted Boltzmann machines. Neurocomputing 275:1186–1199. https://doi.org/10.1016/j.neucom.2017.09.065

    Article  Google Scholar 

  12. Zhang N, Ding S, Sun T (2020) Multi-view RBM with posterior consistency and domain adaptation. Inf Sci 516:142–157. https://doi.org/10.1016/j.ins.2019.12.062

    Article  Google Scholar 

  13. Zhang J, Ding S, Zhang N (2019) Adversarial training methods for Boltzmann machines. IEEE Access 8:4594–4604. https://doi.org/10.1109/ACCESS.2019.2962758

    Article  Google Scholar 

  14. Ding S, Zhang N, Zhang X (2017) Twin support vector machine: theory, algorithm and applications. Neural Comput Appl 28(11):3119–3130. https://doi.org/10.1007/s00521-016-2245-4

    Article  Google Scholar 

  15. Aquino G, Rubio J, Pacheco J (2020) Novel nonlinear hypothesis for the delta parallel robot modeling. IEEE Access 8:46324–46334. https://doi.org/10.1109/ACCESS.2020.2979141

    Article  Google Scholar 

  16. de Jesús Rubio J (2009) SOFMLS: online self-organizing fuzzy modified least-squares network. IEEE Trans Fuzzy Sys 17(6):1296–1309. https://doi.org/10.1109/TFUZZ.2009.2029569

    Article  Google Scholar 

  17. Chiang H, Chen M, Huang J (2019) Wavelet-based EEG processing for epilepsy detection using fuzzy entropy and associative petri net. IEEE Access 7:103255–103262. https://doi.org/10.1109/ACCESS.2019.2929266

    Article  Google Scholar 

  18. Elias I, Rubio J, Cruz D (2020) Hessian with mini-batches for electrical demand prediction. Appl Sci 10(6):2036. https://doi.org/10.3390/app10062036

    Article  Google Scholar 

  19. Meda-Campaña JA (2018) On the estimation and control of nonlinear systems with parametric uncertainties and noisy outputs. IEEE Access 6:31968–31973. https://doi.org/10.1109/ACCESS.2018.2846483

    Article  Google Scholar 

  20. Ashfahani A, Pratama M, Lughofer E (2020) DEVDAN: deep evolving denoising autoencoder. Neurocomputing 390:297–314. https://doi.org/10.1016/j.neucom.2019.07.106

    Article  Google Scholar 

  21. Zhang X, Wang T, Qi J, Lu H, Wang G (2018) Progressive attention guided recurrent network for salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 714–722. https://doi.org/10.1109/CVPR.2018.00081

  22. Li G, Yu Y (2015) Visual saliency based on multiscale deep features. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5455–5463. https://doi.org/10.1109/CVPR.2015.7299184

  23. Zhang P, Wang D, Lu H, Wang H, Yin B (2017) Learning uncertain convolutional features for accurate saliency detection. In: Proceedings of the IEEE international conference on computer vision, pp 212–221. https://doi.org/10.1109/ICCV.2017.32

  24. Zhao JX, Liu JJ, Fan DP, Cao Y, Yang J, Cheng MM (2019) EGNet: edge guidance network for salient object detection. In: Proceedings of the IEEE international conference on computer vision, pp 8779–8788. arXiv:1908.08297

  25. Wang T, Zhang L, Wang S, Lu H, Yang G, Ruan X, Borji A (2018) Detect globally, refine locally: a novel approach to saliency detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3127–3135. https://doi.org/10.1007/978-3-642-40246-3_41

  26. Zhang P, Wang D, Lu H, Wang H, Ruan X (2017) Amulet: aggregating multi-level convolutional features for salient object detection. In: Proceedings of the IEEE international conference on computer vision. https://doi.org/10.1109/ICCV.2017.31, pp 202–211

  27. Fidon L, Li W, Garcia L, Ekanayake J, Kitchen N, Ourselin B, Vercauteren T (2017) Generalised wasserstein dice score for imbalanced multi-class segmentation using holistic convolutional networks. In: International MICCAI brainlesion workshop, pp 64–76. https://doi.org/10.1007/978-3-319-75238-9_6

  28. Luo Z, Mishra A, Achkar A, Eichel J, Li S, Jodoin PM (2017) Non-local deep features for salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6609–6617. https://doi.org/10.1109/CVPR.2017.698

  29. Yan Q, Xu L, Shi J, Jia J (2013) Hierarchical saliency detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1155–1162. https://doi.org/10.1109/CVPR.2013.153

  30. Li Y, Hou X, Koch C, Rehg JM, Yuille AL (2014) The secrets of salient object segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 280–287. https://doi.org/10.1109/CVPR.2014.43

  31. Wang L, Lu H, Wang Y, Feng M, Wang D, Yin B, Ruan X (2017) Learning to detect salient objects with image-level supervision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 136–145. https://doi.org/10.1109/CVPR.2017.404

  32. Liu T, Yuan Z, Sun J, Wang J, Zheng N, Tang X, Shum HY (2010) Learning to detect a salient object. IEEE Trans Pattern Anal Mach Intell 33(2):353–367. https://doi.org/10.1109/TPAMI.2010.70

    Google Scholar 

  33. Wang T, Borji A, Zhang L, Zhang P, Lu H (2017) A stagewise refinement model for detecting salient objects in images. In: Proceedings of the IEEE international conference on computer vision, pp 4019–4028. https://doi.org/10.1109/ICCV.2017.433

  34. Li G, Yu Y (2018) Contrast-oriented deep neural networks for salient object detection. IEEE Trans Neural Netw Learn Sys 29(12):6038–6051. https://doi.org/10.1109/TNNLS.2018.2817540

    Article  Google Scholar 

  35. Li X, Yang F, Cheng H, Liu W, Shen D (2018) Contour knowledge transfer for salient object detection. In: Proceedings of the European conference on computer vision, pp 355–370. https://doi.org/10.1007/978-3-030-01267-0_22

  36. Zeng Y, Zhang P, Zhang J, Lin Z, Lu H (2019) Towards high-resolution salient object detection. In: Proceedings of the IEEE international conference on computer vision. https://doi.org/10.1109/ICCV.2019.00733, pp 7234–7243

  37. Hu P, Shuai B, Liu J, Wang G (2017) Deep level sets for salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition. https://doi.org/10.1109/CVPR.2017.65, pp 2300–2309

  38. Feng M, Lu H, Ding E (2019) Attentive feedback network for boundary-aware salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1623–1632. https://doi.org/10.1109/CVPR.2019.00172

  39. Huang M, Liu Z, Ye L, Zhou X, Wang Y (2019) Saliency detection via multi-level integration and multi-scale fusion neural networks. Neurocomputing 364:310–321. https://doi.org/10.1016/j.neucom.2019.07.054

    Article  Google Scholar 

  40. Qiu W, Gao X, Han B (2020) Saliency detection using a deep conditional random field network. Pattern Recogn 103:107–266. https://doi.org/10.1016/j.patcog.2020.107266

    Article  Google Scholar 

  41. Feng M, Lu H, Yu Y (2020) Residual learning for salient object detection. IEEE Trans Image Process 29:4696–4708. https://doi.org/10.1109/TIP.2020.2975919

    Article  Google Scholar 

  42. Fan D, Gong C, Cao Y, Ren B, Cheng M, Borji A (2018) Enhanced-alignment measure for binary fore- ground map evaluation. In: Proceedings of the 27th international joint conference on artificial intelligence, pp 698–704. https://doi.org/10.24963/ijcai.2018/97

  43. Fan D, Cheng M, Liu Y, Li T, Borji A (2017) Structure-measure: a new way to evaluate foreground maps. In: Proceedings of the IEEE international conference on computer vision, pp 548–4557. https://doi.org/10.1109/ICCV.2017.487

Download references

Acknowledgements

This work was supported in part by National Natural Science Foundation of China under grant 61771145 and 61371148.

Author information

Authors and Affiliations

  1. Department of Electronic Engineering, Fudan University, Shanghai, 200433, China

    Zhenshan Tan & Xiaodong Gu

Authors
  1. Zhenshan Tan
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Xiaodong Gu
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Xiaodong Gu.

Ethics declarations

Conflict of interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tan, Z., Gu, X. Depth scale balance saliency detection with connective feature pyramid and edge guidance. Appl Intell 51, 5775–5792 (2021). https://doi.org/10.1007/s10489-020-02150-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-020-02150-z

Keywords