Skip to main content
SpringerLink
Log in

A mix-supervised unified framework for salient object detection

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Recently, although deep learning network has shown its advantages in supervised salient object detection, supervised models often require massive pixel-wise annotations and learnable parameters, which seriously manacle training and testing of models. In this paper, we present a mix-supervised unified framework for salient object detection to avoid the insufficient training labels and speed training and testing up, which is composed of a region-wise stream and a pixel-wise stream. In the region-wise stream, to avoid the requirement of expensive pixel-wise annotations, an improved energy equation based manifold learning algorithm is employed, by which accurate object location and prior knowledge are introduced by the unsupervised learning. In the pixel-wise stream, to alleviate the problem of time-consuming, a simplified bi-directional reuse network is introduced, which can obtain clear object contour and competitive performance with fewer parameters. To relieve the bottleneck pressure of parallel training and testing, each steam is directly connected to its pre-processed color feature and post-processing refinement. Extensive experiments demonstrate that each component contributes to the final results and complement each other perfectly.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Wang Z, Wu X (2016) Salient object detection using biogeography-based optimization to combine features. Appl Intell 45(1):1–17

    Article  Google Scholar 

  2. Zhang T, Zou J, Jia W (2018) Fast and robust road sign detection in driver assistance systems. Appl Intell 48(11):4113–4127

    Article  Google Scholar 

  3. Yu L, Jin M, Zhou K (2020) Multi-channel biomimetic visual transformation for object feature extraction and recognition of complex scenes. Appl Intell 50(3):792–811

    Article  Google Scholar 

  4. Madani K, Kachurka V, Sabourin C, Amarger V, Golovko V, Rossi L (2018) A human-like visual-attention-based artificial vision system for wildland firefighting assistance. Appl Intell 48(8):2157–2179

    Article  Google Scholar 

  5. Borji A, Cheng M, Hou Q, Jiang H, Li J (2019) Salient object detection: A survey. Computational Visual Media 5(2):117–150

    Article  Google Scholar 

  6. Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell 20(11):1254–1259

    Article  Google Scholar 

  7. Liu T, Yuan Z, Sun J, Wang J, Zheng N, Tang X, Shum H-Y (2011) Learning to detect a salient object. IEEE Trans Pattern Anal Mach Intell 33(2):353–367

    Article  Google Scholar 

  8. Qin Y, Lu H, Xu Y, Wang H (2015) Saliency detection via cellular automata. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 110–119

  9. Zhang J, Sclaroff S, Lin Z, Shen X, Price B, Mech R (2015) Minimum barrier salient object detection at 80 fps. In: Proceedings of the IEEE International Conference on Computer Vision, pp 1404–1412

  10. Tu W-C, He S, Yang Q, Chien S-Y (2016) Real-time salient object detection with a minimum spanning tree. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 2334–2342

  11. Yang C, Zhang L, Lu H, Ruan X, Yang M-H (2013) Saliency detection via graph-based manifold ranking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3166–3173

  12. Yan Q, Xu L, Shi J, Jia J (2013) Hierarchical saliency detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1155–1162

  13. Jiang H, Wang J, Yuan Z, Wu Y, Zheng N, Li S (2013) Salient object detection: A discriminative regional feature integration approach. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 2083–2090

  14. Wang L, Lu H, Ruan X, Yang M-H (2015) Deep networks for saliency detection via local estimation and global search. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3183–3192

  15. Zhao R, Ouyang W, Li H, Wang X (2015) Saliency detection by multi-context deep learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1265–1274

  16. Li G, Yu Y (2016) Visual saliency detection based on multiscale deep cnn features. IEEE Trans Image Process 25(11):5012– 5024

    Article  MathSciNet  MATH  Google Scholar 

  17. Li X, Zhao L, Wei L, Yang M-H, Wu F, Zhuang Y, Ling H, Wang J (2016) Deepsaliency: Multi-task deep neural network model for salient object detection. IEEE Trans Image Process 25(8):3919–3930

    Article  MathSciNet  MATH  Google Scholar 

  18. Liu Y, Wang X, Qi S, Guan J, Jia F, Yao L (2018) Pixel meets region: A pratical framework for salient object detection. In: 2018 IEEE International Conference on Multimedia and Expo, pp 1–6

  19. Wang X, Guan J, Qi S, Liao Q, Li H (2019) Bi-directional features reuse network for salient object detection. In: Pacific Rim International Conference on Artificial Intelligence. Springer, pp 29–41

  20. Ganz M, Yang X, Slabaugh G (2012) Automatic segmentation of polyps in colonoscopic narrow-band imaging data. IEEE Trans Biomed Eng 59(8):2144–2151

    Article  Google Scholar 

  21. Tajbakhsh N, Gurudu SR, Liang J (2015) Automated polyp detection in colonoscopy videos using shape and context information. IEEE Trans Med Imaging 35(2):630–644

    Article  Google Scholar 

  22. Li X, Lu H, Zhang L, Ruan X, Yang M-H (2013) Saliency detection via dense and sparse reconstruction. In: Proceedings of the IEEE International Conference on Computer Vision, pp 2976–2983

  23. Jiang B, Zhang L, Lu H, Yang C, Yang M-H (2013) Saliency detection via absorbing markov chain. In: Proceedings of the IEEE International Conference on Computer Vision, pp 1665– 1672

  24. Wang W, Lai Q, Fu H, Shen J, Ling H Salient object detection in the deep learning era: An in-depth survey, arXivx:1904.09146

  25. Wang L, Wang L, Lu H, Zhang P, Ruan X (2016) Saliency detection with recurrent fully convolutional networks. In: European Conference on Computer Vision, pp 825–841

  26. Cao C, Huang Y, Wang Z, Wang L, Xu N, Tan T (2018) Lateral inhibition-inspired convolutional neural network for visual attention and saliency detection. In: Thirty-second AAAI conference on artificial intelligence

  27. Wang T, Borji A, Zhang L, Zhang P, Lu H (2017) A stagewise refinement model for detecting salient objects in images. In: Proceedings of the IEEE International Conference on Computer Vision, pp 4019–4028

  28. Chen X, Zheng A, Li J, Lu F (2017) Look, perceive and segment: Finding the salient objects in images via two-stream fixation-semantic cnns. In: Proceedings of the IEEE International Conference on Computer Vision, pp 1050–1058

  29. Hu X, Zhu L, Qin J, Fu C-W., Heng P-A. (2018) Recurrently aggregating deep features for salient object detection. In: Thirty-second AAAI conference on artificial intelligence

  30. Amirul Islam M, Kalash M, Bruce ND (2018) Revisiting salient object detection: Simultaneous detection, ranking, and subitizing of multiple salient objects. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 7142–7150

  31. Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp 234–241

  32. Liu N, Han J, Yang M-H (2018) Picanet: Learning pixel-wise contextual attention for saliency detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3089–3098

  33. Chen S, Tan X, Wang B, Hu X (2018) Reverse attention for salient object detection. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 234–250

  34. Li G, Xie Y, Lin L (2018) Weakly supervised salient object detection using image labels. In: Thirty-second AAAI conference on artificial intelligence

  35. Yang J, Price B, Cohen S, Lee H, Yang M-H (2016) Object contour detection with a fully convolutional encoder-decoder network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 193–202

  36. Li X, Yang F, Cheng H, Liu W, Shen D (2018) Contour knowledge transfer for salient object detection. In: Proceedings of the European Conference on Computer Vision, pp 355– 370

  37. Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Süsstrunk S (2012) Slic superpixels compared to state-of-the-art superpixel methods. IEEE Trans Pattern Anal Mach Intell 34(11):2274– 2282

    Article  Google Scholar 

  38. Fengwei J, Xuan W, Jian G, Shuhan Q (2019) Bi-directional features reuse network for salient object detection. In: Pacific rim international conference on artificial intelligence

  39. Krähenbühl P, Koltun V (2011) Efficient inference in fully connected crfs with gaussian edge potentials. In: Advances in Neural Information Processing Systems, pp 109–117

  40. Li G, Yu Y (2016) Deep contrast learning for salient object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 478–487

  41. Li Y, Hou X, Koch C, Rehg JM, Yuille AL (2014) The secrets of salient object segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 280–287

  42. Martin D, Fowlkes C, Tal D, Malik J (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings of the IEEE Conference on International Conference on Computer Vision, Vol 2, pp 416–423

  43. Einhäuser W, König P (2003) Does luminance-contrast contribute to a saliency map for overt visual attention?. Eur J NeuroSci 17(5):1089–1097

    Article  Google Scholar 

  44. Everingham M, Van Gool L, Williams CK, Winn J, Zisserman A (2010) The pascal visual object classes (voc) challenge. Int J Comput Vis 88(2):303–338

    Article  Google Scholar 

  45. Achanta R, Hemami S, Estrada F, Susstrunk S (2009) Frequency-tuned salient region detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1597–1604

  46. Kingma DP, Ba J Adam: A method for stochastic optimization, arXiv:1412.6980

  47. Srivastava G, Srivastava R (2019) Modification of gradient vector flow using directional contrast for salient object detection. IEEE MultiMedia 26(4):7–16

    Article  Google Scholar 

  48. Fatemi N, Sajedi H, Ahmadabadi MES (2019) Fully unsupervised salient object detection. In: 2019 4th International Conference on Pattern Recognition and Image Analysis, pp 32–38

  49. Wang L, Lu H, Wang Y, Feng M, Wang D, Yin B, Ruan X (2017) Learning to detect salient objects with image-level supervision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 136–145

  50. Hou Q, Cheng M, Hu X, Borji A, Tu Z, Torr P (2019) Deeply supervised salient object detection with short connections. IEEE Trans Pattern Anal Mach Intell 41(4):815

    Article  Google Scholar 

  51. Guan W, Wang T, Qi J, Zhang L, Lu H (2019) Edge-aware convolution neural network based salient object detection. IEEE Signal Process Lett 26(1):114–118

    Article  Google Scholar 

  52. Zeng Y, Zhuge Y, Lu H, Zhang L, Qian M, Yu Y (2019) Multi-source weak supervision for saliency detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 6074–6083

Download references

Acknowledgments

This work is supported by National Natural Science Foundation of China (No. 61902093) and Key Technology Program of Shenzhen, China (No. JSGG20170823152809704), Basic Research Project of Shenzhen, China, (No. JCYJ20180507183624136), Guangdong Key R&D Program (No. 2019B010136001), Natural Science Foundation of Guangdong (No. 2020A1515010652)

Author information

Authors and Affiliations

  1. Computer Application Research Center, Harbin Institute of Technology, Shenzhen, 518055, China

    Fengwei Jia, Shuhan Qi, Huale Li & Xuan Wang

  2. College of Computer Science and Technology, Harbin Engineering University, Harbin, 150001, China

    Jian Guan

Authors
  1. Fengwei Jia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jian Guan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shuhan Qi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Huale Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xuan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xuan Wang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jia, F., Guan, J., Qi, S. et al. A mix-supervised unified framework for salient object detection. Appl Intell 50, 2945–2958 (2020). https://doi.org/10.1007/s10489-020-01700-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-020-01700-9

Keywords

Search

Navigation