Skip to main content

Advertisement

Multiview diffusion-based affinity graph learning with good neighbourhoods for salient object detection

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Salient object detection is a challenging task in computer vision and has been used to extract valuable information from many real scenarios. The graph-based detection approach has attracted extensive attention because of its high efficiency and stability. Nevertheless, most existing approaches utilize multiview features to construct graph models, resulting in poor performance in extreme scenes. In graph-based models, the graph structure and neighbourhoods play essential roles in salient object detection performance. In this paper, we propose a novel saliency detection approach via multiview diffusion-based affinity learning with good neighbourhoods. The proposed model includes three components: 1) multiview diffusion-based affinity learning to produce a local/global affinity matrix, 2) subspace clustering to choose good neighbourhoods, and 3) an unsupervised graph-based diffusion model to guide saliency detection. The uniqueness of our affinity graph model lies in exploring multiview handcrafted features to identify different underlying salient objects in extreme scenes. Extensive experiments on several standard databases validate the superior performance of the proposed model over other state-of-the-art methods. The experimental results demonstrate that our graph model with multiview handcrafted features is competitive with the outstanding graph models with multiview deep features.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data Availibility and Acces

All data generated or analysed during this study are included in this published article.

References

  1. Kong Y, Wang Y, Li A (2021) Spatiotemporal saliency representation learning for video action recognitionl. IEEE Trans Multimed 1. https://doi.org/10.1109/TMM.2021.3066775

  2. Chen Y, Zhang Z, Dong L, Xiong S, Lu X (2024) A joint saliency temporal-spatial-spectral information network for hyperspectral image change detection. IEEE Trans Geosci Remote Sens 62:1–15. https://doi.org/10.1109/TGRS.2023.3344583

    Article  MATH  Google Scholar 

  3. Xiao M, Yang B, Wang S, Zhang Z, He Y (2023) Fine coordinate attention for surface defect detection. Eng Appl Artif Intell 123:106368. https://doi.org/10.1016/j.engappai.2023.106368

  4. Lateef F, Kas M, Ruichek Y (2022) Saliency heat-map as visual attention for autonomous driving using generative adversarial network (gan). IEEE Trans Intell Transp Syst 23(6):5360–5373. https://doi.org/10.1109/TITS.2021.3053178

    Article  MATH  Google Scholar 

  5. Sun C, Wu X, Sun J, Sun C, Xu M, Ge Q (2023) Saliency-induced moving object detection for robust rgb-d vision navigation under complex dynamic environments. IEEE Trans Intell Transp Syst 1–19. https://doi.org/10.1109/TITS.2023.3275279

  6. Liu J, Dian R, Li S, Liu H (2023) Sgfusion: A saliency guided deep-learning framework for pixel-level image fusion. Information Fusion 91:205–214. https://doi.org/10.1016/j.inffus.2022.09.030

    Article  MATH  Google Scholar 

  7. Zhu W, Liang S, Wei Y, Sun J (2014) Saliency optimization from robust background detection. Proceedings of the IEEE computer society conference on computer vision and pattern recognition. 2814–2821. https://doi.org/10.1109/CVPR.2014.360

  8. Cheng M-M, Mitra NJ, Huang X, Torr PHS, Hu S-M (2015) Global contrast based salient region detection. IEEE Trans Pattern Anal Mach Intell 37(3):569–582. https://doi.org/10.1109/TPAMI.2014.2345401

    Article  MATH  Google Scholar 

  9. Jian M, Wang J, Yu H, Wang G, Meng X, Yang L, Dong J, Yin Y (2021) Visual saliency detection by integrating spatial position prior of object with background cues. Expert Syst Appl 168:114219. https://doi.org/10.1016/j.eswa.2020.114219

    Article  MATH  Google Scholar 

  10. Song S, Jia Z, Yang J, Kasabov N (2022) Salient detection via the fusion of background-based and multiscale frequency-domain features. Inf Sci 618:53–71. https://doi.org/10.1016/j.ins.2022.10.103

    Article  MATH  Google Scholar 

  11. Sun X, Zhang X, Xu C, Xiao M, Tang Y (2023) Tensorial multiview representation for saliency detection via nonconvex approach. IEEE Transactions on cybernetics 53(3):1816–1829. https://doi.org/10.1109/TCYB.2021.3139037

    Article  MATH  Google Scholar 

  12. Li X, Song D, Dong Y (2020) Hierarchical feature fusion network for salient object detection. IEEE Trans Image Process 29:9165–9175. https://doi.org/10.1109/TIP.2020.3023774

    Article  MATH  Google Scholar 

  13. Zong G, Wei L, Guo S, Wang Y (2022) A cascaded refined rgb-d salient object detection network based on the attention mechanism. Appl Intell 53(11):1–22. https://doi.org/10.1007/s10489-022-04186-9

    Article  MATH  Google Scholar 

  14. Chen C, Song M, Song W, Guo L, Jian M (2023) A comprehensive survey on video saliency detection with auditory information: The audio-visual consistency perceptual is the key! IEEE Trans Circuits Syst Video Technol 33(2):457–477. https://doi.org/10.1109/TCSVT.2022.3203421

    Article  MATH  Google Scholar 

  15. Qian X, Zeng Y, Wang W, Zhang Q (2023) Co-saliency detection guided by group weakly supervised learning. IEEE Trans Multimedia 25:1810–1818. https://doi.org/10.1109/TMM.2022.3167805

    Article  MATH  Google Scholar 

  16. Wang K, Lin D, Li C, Tu Z, Luo B (2024) Alignment-free rgbt salient object detection: Semantics-guided asymmetric correlation network and a unified benchmark. IEEE Trans Multimedia 1–16. https://doi.org/10.1109/TMM.2024.3410542

  17. Goferman S, Zelnik-Manor L, Tal A, (2012) Context-aware saliency detection. IEEE Trans Pattern Anal Mach Intell 34(10):1915–1926. https://doi.org/10.1109/TPAMI.2011.272

  18. Yang C, Zhang L, Lu H, Ruan X, Yang MH (2013) Saliency detection via graph-based manifold ranking. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition 3166–3173. https://doi.org/10.1109/CVPR.2013.407

  19. Li C, Yuan Y, Cai W, Xia Y, Feng DD (2015) Robust saliency detection via regularized random walks ranking. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition 07-12-June, pp 2710–2717. https://doi.org/10.1109/CVPR.2015.7298887

  20. Moradi M, Bayat F (2021) A salient object segmentation framework using diffusion-based affinity learning. Expert Syst Appl 168:114428. https://doi.org/10.1016/j.eswa.2020.114428

    Article  MATH  Google Scholar 

  21. Zhang YY, Wang H, Lv X, Zhang P (2021) Capturing the grouping and compactness of high-level semantic feature for saliency detection. Neural Netw 142:351–362. https://doi.org/10.1016/j.neunet.2021.04.028

    Article  MATH  Google Scholar 

  22. Gong A, Huang L, Shi J, Liu C (2022) Unsupervised rgb-t saliency detection by node classification distance and sparse constrained graph learning. Appl Intell 52(1):1030–1043. https://doi.org/10.1007/s10489-021-02434-y

    Article  MATH  Google Scholar 

  23. Ganguly B, Dey D, Munshi S (2022) An unsupervised learning approach for road anomaly segmentation using rgb-d sensor for advanced driver assistance system. IEEE Trans Intell Transp Syst 23(10):19042–19053. https://doi.org/10.1109/TITS.2022.3164847

    Article  MATH  Google Scholar 

  24. Zhou L, Yang Z, Yuan Q, Zhou Z, Hu D (2015) Salient Region Detection via Integrating Diffusion-Based Compactness and Local Contrast. IEEE Trans Image Process 24(11):3308–3320. https://doi.org/10.1109/TIP.2015.2438546

    Article  MathSciNet  MATH  Google Scholar 

  25. Qin Y, Feng M, Lu H, Cottrell G (2018) Hierarchical cellular automata for visual saliency. Int J Comput Vision 126:751–770. https://doi.org/10.1007/s11263-017-1062-2

    Article  MathSciNet  MATH  Google Scholar 

  26. Zhang L, Ai J, Jiang B, Lu H, Li X (2018) Saliency Detection via Absorbing Markov Chain with Learnt Transition Probability. IEEE Trans Image Process 27(2):987–998. https://doi.org/10.1109/TIP.2017.2766787

    Article  MathSciNet  MATH  Google Scholar 

  27. Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell 20(11):1254–1259. https://doi.org/10.1109/34.730558

    Article  MATH  Google Scholar 

  28. Yuan B, Han L, Yan H (2021) Explore double-opponency and skin color for saliency detection. Neurocomputing 425:219–230. https://doi.org/10.1016/j.neucom.2020.04.089

    Article  MATH  Google Scholar 

  29. Huang Z, Chen H-X, Zhou T, Yang Y-Z, Wang C-Y, Liu B-Y (2021) Contrast-weighted dictionary learning based saliency detection for vhr optical remote sensing images. Pattern Recogn 113:107757. https://doi.org/10.1016/j.patcog.2020.107757

    Article  Google Scholar 

  30. Deng C, Yang X, Nie F, Tao D (2020) Saliency Detection via a Multiple Self-Weighted Graph-Based Manifold Ranking. IEEE Trans Multimedia 22(4):885–896. https://doi.org/10.1109/TMM.2019.2934833

    Article  MATH  Google Scholar 

  31. Xu M, Liu B, Fu P, Li J, Hu YH, Feng S (2020) Video salient object detection via robust seeds extraction and multi-graphs manifold propagation. IEEE Trans Circuits Syst Video Technol 30(7):2191–2206. https://doi.org/10.1109/TCSVT.2019.2920652

    Article  MATH  Google Scholar 

  32. Xia C, Gao X, Li KC, Zhao Q, Zhang S (2021) Salient object detection based on distribution-edge guidance and iterative bayesian optimization. Applied Intelligence: The International Journal of Artificial Intelligence, Neural Networks, and Complex Problem-Solving Technologies 10:50. https://doi.org/10.1007/s10489-020-01691-7

    Article  MATH  Google Scholar 

  33. Pang Y, Wu C, Wu H, Yu X (2023) Unsupervised multi-subclass saliency classification for salient object detection. IEEE Trans Multimedia 25:2189–2202. https://doi.org/10.1109/TMM.2022.3144070

    Article  MATH  Google Scholar 

  34. Pan W, Sun X, Qian Y() Rgb-d saliency detection via complementary and selective learning. Applied Intelligence: The International Journal of Artificial Intelligence, Neural Networks, and Complex Problem-Solving Technologies 53(7):7957–7969. https://doi.org/10.1007/s10489-022-03612-2

  35. Wang L, Wang L, Lu H, Zhang P, Ruan X (2019) Salient object detection with recurrent fully convolutional networks. IEEE Trans Pattern Anal Mach Intell 41(07):1734–1746. https://doi.org/10.1109/TPAMI.2018.2846598

    Article  MATH  Google Scholar 

  36. Xia C, Zhang H, Gao X, Li K (2020) Exploiting background divergence and foreground compactness for salient object detection. Neurocomputing 383(28):194–211. https://doi.org/10.1016/j.neucom.2019.09.096

    Article  MATH  Google Scholar 

  37. Huang L, Song K, Gong A, Liu C, Yan Y (2020) Rgb-t saliency detection via low-rank tensor learning and unified collaborative ranking. IEEE Signal Process Lett 27:1585–1589. https://doi.org/10.1109/LSP.2020.3020735

    Article  MATH  Google Scholar 

  38. Huang L, Song K, Wang J, Niu M, Yan Y (2022) Multi-graph fusion and learning for rgbt image saliency detection. IEEE Trans Circuits Syst Video Technol 32(3):1366–1377. https://doi.org/10.1109/TCSVT.2021.3069812

    Article  MATH  Google Scholar 

  39. Tu Z, Xia T, Li C, Wang X, Ma Y, Tang J (2020) RGB-T Image Saliency Detection via Collaborative Graph Learning. IEEE Transactions on Multimedia 22(1):160–173. https://doi.org/10.1109/TMM.2019.2924578. arXiv:1905.06741

  40. Xu M, Fu P, Liu B, Li J (2021) Multi-stream attention-aware graph convolution network for video salient object detection. IEEE Trans Image Process 30:4183–4197. https://doi.org/10.1109/TIP.2021.3070200

    Article  MATH  Google Scholar 

  41. Xu M, Fu P, Liu B, Yin H, Li J (2021) A novel dynamic graph evolution network for salient object detection. Appl Intell 52(3):2854–2871. https://doi.org/10.1007/s10489-021-02479-z

    Article  MATH  Google Scholar 

  42. Wu Y, Jia T, Chang X, Wang H, Chen D (2024) Rgb-t saliency detection based on multiscale modal reasoning interaction. IEEE Trans Instrum Meas 73:1–15. https://doi.org/10.1109/TIM.2024.3419115

    Article  MATH  Google Scholar 

  43. Li C, Liu F, Tian Z, Du S, Wu Y (2024) Dagcn: Dynamic and adaptive graph convolutional network for salient object detection. IEEE Transactions on Neural Networks and Learning Systems 35(6):7612–7626. https://doi.org/10.1109/TNNLS.2022.3219245

    Article  Google Scholar 

  44. Jiang B, Zhang L, Lu H, Yang C, Yang M-H (2013) Saliency Detection via Absorbing Markov Chain. In: The IEEE International Conference on Computer Vision (ICCV), pp 1665–1672

  45. Yuan Y, Li C, Kim J, Cai W, Feng DD (2018) Reversion correction and regularized random walk ranking for saliency detection. IEEE Trans Image Process 27(3):1311–1322. https://doi.org/10.1109/TIP.2017.2762422

    Article  MathSciNet  MATH  Google Scholar 

  46. Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Süsstrunk S (2012) SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans Pattern Anal Mach Intell 34(11):2274–2281. https://doi.org/10.1109/TPAMI.2012.120

    Article  Google Scholar 

  47. Heikkilä M, Pietikäinen M, Schmid C (2009) Description of interest regions with local binary patterns. Pattern Recogn 42(3):425–436. https://doi.org/10.1016/j.patcog.2008.08.014

    Article  MATH  Google Scholar 

  48. Simoncelli EP, Freeman WT (1995) The steerable pyramid: a flexible architecture for multi-scale derivative computation. In: Proceedings., International Conference on Image Processing, vol 3, pp 444–447. https://doi.org/10.1109/ICIP.1995.537667

  49. Arbeláez P, Maire M, Fowlkes C, Malik J (2011) Contour detection and hierarchical image segmentation. IEEE Trans Pattern Anal Mach Intell 33(5):898–916. https://doi.org/10.1109/TPAMI.2010.161

    Article  MATH  Google Scholar 

  50. Bai S, Bai X, Tian Q, Latecki LJ (2019) Regularized diffusion process on bidirectional context for object retrieval. IEEE Trans Pattern Anal Mach Intell 41(5):1213–1226. https://doi.org/10.1109/TPAMI.2018.2828815

    Article  MATH  Google Scholar 

  51. Tang C, Liu X, Zhu X, Zhu E, Gao W (2020) Cgd: Multi-view clustering via cross-view graph diffusion. Proceedings of the AAAI Conference on Artificial Intelligence 34(4):5924–5931

    Article  MATH  Google Scholar 

  52. You C, Robinson D.P, Vidal R (2016) Scalable sparse subspace clustering by orthogonal matching pursuit. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 3918–3927. https://doi.org/10.1109/CVPR.2016.425

  53. Yan Q, Xu L, Shi J, Jia J (2013) Hierarchical saliency detection. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 1155–1162. https://doi.org/10.1109/CVPR.2013.153

  54. Movahedi V, Elder J.H (2010) Design and perceptual validation of performance measures for salient object segmentation. 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops, CVPRW 2010, pp 49–56. https://doi.org/10.1109/CVPRW.2010.5543739

  55. Li Y, Hou X, Koch C, Rehg JM, Yuille AL (2014) The secrets of salient object segmentation. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp,280–287. https://doi.org/10.1109/CVPR.2014.43

  56. Li G, Yu Y (2016) Visual saliency detection based on multiscale deep CNN features. IEEE Transactions on Image Processing 25(11):5012–5024. https://doi.org/10.1109/TIP.2016.2602079. arXiv:1609.02077

  57. Achantay R, Hemamiz S, Estraday F, Süsstrunky S (2009) Frequency-tuned salient region detection. 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2009 2009 IEEE, pp 1597–1604. https://doi.org/10.1109/CVPRW.2009.5206596

  58. Fan D.-P, Gong C, Cao Y, Ren B, Cheng M.-M, Borji A (2018) Enhanced-alignment measure for binary foreground map evaluation. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence. IJCAI’18. pp 698–704

  59. Fan D.P, Cheng M.M, Liu Y, Li T, Borji A (2017) Structure-Measure: A New Way to Evaluate Foreground Maps. In: Proceedings of the IEEE International Conference on Computer Vision, vol 2017-Octob, pp 4558–4567. https://doi.org/10.1109/ICCV.2017.487

  60. Margolin R, Zelnik-Manor L, Tal A (2014) How to evaluate foreground maps. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition 248–255. https://doi.org/10.1109/CVPR.2014.39

  61. Peng H, Li B, Ling H, Hu W, Xiong W, Maybank SJ (2017) Salient Object Detection via Structured Matrix Decomposition. IEEE Trans Pattern Anal Mach Intell 39(4):818–832. https://doi.org/10.1109/TPAMI.2016.2562626

    Article  Google Scholar 

  62. Perazzi F, Krahenbuhl P, Pritch Y, Hornung A (2012) Saliency filters: Contrast based filtering for salient region detection. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition 733–740. https://doi.org/10.1109/CVPR.2012.6247743

  63. Li X, Lu H, Zhang L, Ruan X, Yang M-H (2013) Saliency detection via dense and sparse reconstruction. In: 2013 IEEE International Conference on Computer Vision, pp 2976–2983. https://doi.org/10.1109/ICCV.2013.370

  64. Qin Y, Lu H, Xu Y, Wang H (2015) Saliency detection via Cellular Automata. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol 07-12-June, pp 110–119. https://doi.org/10.1109/CVPR.2015.7298606

  65. Sun J, Lu H, Liu X (2015) Saliency Region Detection Based on Markov Absorption Probabilities. IEEE Trans Image Process 24(5):1639–1649. https://doi.org/10.1109/TIP.2015.2403241

    Article  MathSciNet  MATH  Google Scholar 

  66. Fu K, Gu I.Y.H, Gong C, Yang J (2015) Robust manifold-preserving diffusion-based saliency detection by adaptive weight construction. Neurocomputing 175(PartA):336–347. https://doi.org/10.1016/j.neucom.2015.10.066

  67. Zhou L, Yang Z, Zhou Z, Hu D (2017) Salient Region Detection Using Diffusion Process on a Two-Layer Sparse Graph. IEEE Trans Image Process 26(12):5882–5894. https://doi.org/10.1109/TIP.2017.2738839

  68. Wang J, Jiang H, Yuan Z, Cheng MM, Hu X, Zheng N (2013) Salient Object Detection: A Discriminative Regional Feature Integration Approach. Int J Comput Vision 123:251–268. https://doi.org/10.1007/s11263-016-0977-3

    Article  MATH  Google Scholar 

  69. Cong R, Lei J, Fu H, Hou J, Huang Q, Kwong S (2020) Going From RGB to RGBD Saliency: A Depth-Guided Transformation Model. IEEE Transactions on Cybernetics 50(8):3627–3639. https://doi.org/10.1109/TCYB.2019.2932005

    Article  MATH  Google Scholar 

  70. Wang F, Peng G (2021) Graph-based saliency detection using a learning joint affinity matrix. Neurocomputing 458:33–46. https://doi.org/10.1016/j.neucom.2021.03.131

    Article  MATH  Google Scholar 

  71. Wang F, Peng G (2022) Graph construction by incorporating local and global affinity graphs for saliency detection. Signal Processing: Image Communication 105:116712. https://doi.org/10.1016/j.image.2022.116712

    Article  MATH  Google Scholar 

  72. Zhang L, Sun J, Wang T, Min Y, Lu H (23020) Visual saliency detection via kernelized subspace ranking with active learning. IEEE Transactions on Image Processing 29:2258–2270. https://doi.org/10.1109/TIP.2019.2945679

  73. Zeng Y, Zhuge Y, Lu H, Zhang L, Qian M, Yu Y (2019) Multi-source weak supervision for saliency detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (Grant No. 52104031 and No. 12401673) and the Natural Science Basic Research Program of Shaanxi (2024JC-YBQN-0670).

Author information

Authors and Affiliations

Authors

Contributions

Fan Wang and Mingxian Wang wrote the main manuscript text and Fan Wang prepared all figures. All authors reviewed the manuscript.

Corresponding author

Correspondence to Fan Wang.

Ethics declarations

Conflicts of Interest

The authors declare that there is no conflict of interests regarding the publication of this paper.

Ethical Approval

This article complies with ethical standards.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, F., Wang, M. & Peng, G. Multiview diffusion-based affinity graph learning with good neighbourhoods for salient object detection. Appl Intell 55, 37 (2025). https://doi.org/10.1007/s10489-024-05847-7

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10489-024-05847-7

Keywords