Skip to main content
SpringerLink
Log in

Reverse collaborative fusion model for co-saliency detection

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

The purpose of co-saliency detection is to find out the salient and common objects of related images. This paper proposes a novel reverse collaborative fusion model (RCFM) for co-saliency detection. The model is mainly composed of two parts: reverse message fusion module (RMFM) and collaborative consistency learning module (CCLM). Specifically, we first aggregate the features in high-level layers as global guidance by using the cascaded decoder (CD). Then, we propose repeated RMFMs on each side output to complete the complementary fusion of deep and shallow information. Then, we fuse multi-scale feature maps as initial co-saliency maps. Finally, the CCLM extracts the collaborative information between images to improve the quality of the initial co-saliency map to obtain the final co-saliency map. The model fully considers the semantic features of high-level and the boundary features of low-level, thereby correcting some deviation predictions and improving the accuracy of co-saliency detection. Compared to the state-of-the-art approaches, experimental results demonstrate that our proposed approach achieves the best performance on four evaluation indicators of three datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Achanta, R., Hemami, S., Estrada, F., Susstrunk, S.: Frequency-tuned salient region detection. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1597–1604. IEEE (2009)

  2. Arbelaez, P., Maire, M., Fowlkes, C., Malik, J.: Contour detection and hierarchical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 33(5), 898–916 (2010)

    Article  Google Scholar 

  3. Bai, C., Chen, J., Huang, L., Kpalma, K., Chen, S.: Saliency-based multi-feature modeling for semantic image retrieval. J. Vis. Commun. Image Represent. 50, 199–204 (2018)

    Article  Google Scholar 

  4. Batra, D., Kowdle, A., Parikh, D., Luo, J., Chen, T.: icoseg: Interactive co-segmentation with intelligent scribble guidance. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 3169–3176. IEEE (2010)

  5. Bi, H., Wang, K., Lu, D., Wu, C., Wang, W., Yang, L: C (2) net: a complementary co-saliency detection network. In: Visual Computer (2020)

  6. Borji, A., Cheng, M.-M., Jiang, H., Li, J.: Salient object detection: a benchmark. IEEE Trans. Image Process. 24(12), 5706–5722 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  7. Borji, A., Frintrop, S., Sihite, D.N., Itti, L.: Adaptive object tracking by learning background context. In: 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 23–30. IEEE (2012)

  8. Cao, X., Tao, Z., Zhang, B., Huazhu, F., Feng, W.: Self-adaptively weighted co-saliency detection via rank constraint. IEEE Trans. Image Process. 23(9), 4175–4186 (2014)

    MathSciNet  MATH  Google Scholar 

  9. Chen, J., Bai, C., Huang, L., Liu, Z., Chen, S.: Visual saliency fusion based multi-feature for semantic image retrieval. In: CCF Chinese Conference on Computer Vision, pp. 126–136. Springer (2017)

  10. Chen, Y.-L., Hsu, C.-T.: Implicit rank-sparsity decomposition: applications to saliency/co-saliency detection. In: 2014 22nd International Conference on Pattern Recognition, pp. 2305–2310. IEEE (2014)

  11. Cheng, M.-M., Mitra, N.J., Huang, X., Hu, S.-M.: Salientshape: group saliency in image collections. Vis. Comput. 30(4), 443–453 (2014)

    Article  Google Scholar 

  12. Cheng, M.-M., Mitra, N., Huang, X., Torr, P.H.S., Hu, S.-M.: Global contrast based salient region detection. IEEE Trans. Pattern Anal. Mach. Intell. 37(3), 569–582 (2014)

    Article  Google Scholar 

  13. Cong, R., Lei, J., Huazhu, F., Cheng, M.-M., Lin, W., Huang, Q.: Review of visual saliency detection with comprehensive information. IEEE Trans Circuits Syst. Video Technol. 29(10), 2941–2959 (2018)

    Article  Google Scholar 

  14. Fan, D.-P., Li, T., Lin, Z., Ji, G.-P., Zhang, D., Cheng, M.-M., Fu, H., Shen, J.: Re-thinking co-salient object detection. Preprint arXiv:2007.03380

  15. Fan, D.-P., Cheng, M.-M., Liu, J.-J., Gao, S.-H., Hou, Q., Borji, A.: Salient objects in clutter: bringing salient object detection to the foreground. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 3–19 (2018)

  16. Fan, D.-P., Cheng, M.-M., Liu, Y., Li, T., Borji, A.: Structure-measure: a new way to evaluate foreground maps. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4548–4557 (2017)

  17. Fan, D.-P., Gong, C., Cao, Y., Ren, B., Cheng, M.-M., Borji, A.: Enhanced-alignment measure for binary foreground map evaluation. Preprint arXiv:1805.10421

  18. Fan, D.-P., Lin, Z., Ji, G.-P., Zhang, D., Fu, H., Cheng, M.-M.: Taking a deeper look at co-salient object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2919–2929 (2020)

  19. Fan, D.-P., Zhai, Y., Borji, A., Yang, J., Shao, L.: Bbs-net: Rgb-d salient object detection with a bifurcated backbone strategy network. In: European Conference on Computer Vision, pp. 275–292. Springer (2020)

  20. Fu, H., Cao, X., Tu, Z.: Cluster-based co-saliency detection. IEEE Trans. Image Process. 22(10), 3766–3778 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  21. Fu, H., Xu, D., Zhang, B., Lin, S., Ward, R.K.: Object-based multiple foreground video co-segmentation via multi-state selection graph. IEEE Trans. Image Process. 24(11), 3415–3424 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  22. Fu, K., Gong, C., Gu, I.Y.-H., Yang, J.: Normalized cut-based saliency detection by adaptive multi-level region merging. IEEE Trans. Image Process. 24(12), 5671–5683 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  23. Gao, S.-H., Tan, Y.-Q., Cheng, M.-M., Lu, C., Chen, Y., Yan, S.: Highly efficient salient object detection with 100k parameters (2020). Preprint arXiv:2003.05643

  24. Ge, C., Keren, F., Liu, F., Bai, L., Yang, J.: Co-saliency detection via inter and intra saliency propagation. Signal Process. Image Commun. 44, 69–83 (2016)

    Article  Google Scholar 

  25. Gong, C., Tao, D., Liu, W., Maybank, S.J., Fang, M., Fu, K., Yang, J.: Saliency propagation from simple to difficult. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2531–2539 (2015)

  26. Han, J., Cheng, G., Li, Z., Zhang, D.: A unified metric learning-based framework for co-saliency detection. IEEE Trans. Circuits Syst. Video Technol. 28(10), 2473–2483 (2017)

    Article  Google Scholar 

  27. Han, J., Quan, R., Zhang, D., Nie, F.: Robust object co-segmentation using background prior. IEEE Trans. Image Process. 27(4), 1639–1651 (2017)

    Article  MathSciNet  Google Scholar 

  28. Hsu, K.-J., Tsai, C.-C., Lin, Y.-Y., Qian X., Chuang, Y.-Y.: Unsupervised cnn-based co-saliency detection with graphical optimization. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 485–501 (2018)

  29. Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20(11), 1254–1259 (1998)

    Article  Google Scholar 

  30. Jeong, D., Hwang, I., Cho, N.I.: Co-salient object detection based on deep saliency networks and seed propagation over an integrated graph. IEEE Trans. Image Process. 27(12), 5866–5879 (2018)

    Article  MathSciNet  Google Scholar 

  31. Jiang, B., Jiang, X., Zhou, A., Tang, J., Luo, B.: A unified multiple graph learning and convolutional network model for co-saliency estimation. In: Proceedings of the 27th ACM International Conference on Multimedia, pp. 1375–1382 (2019)

  32. Klein, D.A, Frintrop, S.: Center-surround divergence of feature statistics for salient object detection. In: 2011 International Conference on Computer Vision, pp. 2214–2219. IEEE (2011)

  33. Lee, G., Tai, Y.-W., Kim, J.: Eld-net: an efficient deep learning architecture for accurate saliency detection. IEEE Trans. Pattern Anal. Mach. Intell. 40(7), 1599–1610 (2017)

    Article  Google Scholar 

  34. Li, B., Sun, Z., Tang, L., Sun, Y., Shi. J.: Detecting robust co-saliency with recurrent co-attention neural network. In: IJCAI, pp. 818–825 (2019)

  35. Li, G., Yu, Y.: Visual saliency based on multiscale deep features. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5455–5463 (2015)

  36. Li, H., Meng, F., Ngan, K.N.: Co-salient object detection from multiple images. IEEE Trans. Multimedia 15(8), 1869–1909 (2013)

    Article  Google Scholar 

  37. Li, H., Ngan, K.N.: A co-saliency model of image pairs. IEEE Trans. Image Process. 20(12), 3365–3375 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  38. Li, H., Chen, J., Huchuan, L., Chi, Z.: Cnn for saliency detection with low-level feature integration. Neurocomputing 226, 212–220 (2017)

    Article  Google Scholar 

  39. Li, L., Liu, Z., Zhang, J.: Unsupervised image co-segmentation via guidance of simple images. Neurocomputing 275, 1650–1661 (2018)

    Article  Google Scholar 

  40. Li, L., Liu, Z., Zou, W., Zhang, X., Le Meur, O.: Co-saliency detection based on region-level fusion and pixel-level refinement. In: 2014 IEEE International Conference on Multimedia and Expo (ICME), pp 1–6. IEEE (2014)

  41. Li, X., Lu, H., Zhang, L., Ruan, X., Yang, M.-H.: Saliency detection via dense and sparse reconstruction. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2976–2983 (2013)

  42. Li, Y., Keren, F., Liu, Z., Yang, J.: Efficient saliency-model-guided visual co-saliency detection. IEEE Signal Process. Lett. 22(5), 588–592 (2014)

    Article  Google Scholar 

  43. Liu, Z., Zou, W., Le Meur, O.: Saliency tree: a novel saliency detection framework. IEEE Trans. Image Process. 23(5), 1937–1952 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  44. Liu, Z., Zou, W., Li, L., Shen, L., Le Meur, O.: Co-saliency detection based on hierarchical segmentation. IEEE Signal Process. Lett. 21(1), 88–92 (2013)

    Article  Google Scholar 

  45. Ma, Y.-F., Zhang, H.-J.: Contrast-based image attention analysis by using fuzzy growing. In: Proceedings of the 11th ACM International Conference on Multimedia, pp. 374–381 (2003)

  46. Mahadevan, V., Vasconcelos, N.: Saliency-based discriminant tracking. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1007–1013. IEEE (2009)

  47. Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al.: Pytorch: an imperative style, high-performance deep learning library. In: Advances in Neural Information Processing Systems, pp. 8026–8037 (2019)

  48. Peng, H., Li, B., Ling, H., Hu, W., Xiong, W., Maybank, S.J.: Salient object detection via structured matrix decomposition. IEEE Trans. Pattern Anal. Mach. Intell. 39(4), 818–832 (2016)

    Article  Google Scholar 

  49. Ren, J., Liu, Z., Zhou, X., Sun, G., Bai, C.: Saliency integration driven by similar images. J. Vis. Commun. Image Represent. 50, 227–236 (2018)

    Article  Google Scholar 

  50. Shen, X., Wu, Y.: A unified approach to salient object detection via low rank matrix recovery. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 853–860. IEEE (2012)

  51. Simonyan, K., Zisserman A.: Very deep convolutional networks for large-scale image recognition (2014). Preprint arXiv:1409.1556

  52. Tang, K., Joulin, A., Li, L.-J., Li, F.-F.: Co-localization in real-world images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1464–1471 (2014)

  53. Toshev, A., Shi, J., Daniilidis, K.: Image matching via saliency region correspondences. In: 2007 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8. IEEE (2007)

  54. Tsai, C.-C., Li, W., Hsu, K.-J., Qian, X., Lin, Y.-Y.: Image co-saliency detection and co-segmentation via progressive joint optimization. IEEE Trans. Image Process. 28(1), 56–71 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  55. Wei, L., Zhao, S., Bourahla, O. E. F., Li, X., Wu, F.: Group-wise deep co-saliency detection (2017). Preprint arXiv:1707.07381

  56. Wei, L., Zhao, S., El Farouk, O., Bourahla, X.L., Fei, W., Zhuang, Y.: Deep group-wise fully convolutional network for co-saliency detection with graph propagation. IEEE Trans. Image Process. 28(10), 5052–5063 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  57. Winn, J., Criminisi, A., Minka, T.: Object categorization by learned universal visual dictionary. In: 10th IEEE International Conference on Computer Vision (ICCV’05), vol. 1, vol. 2, pp. 1800–1807. IEEE (2005)

  58. Woo, S., Park, J., Lee, J.Y., Kweon, I.S.: Cbam: Convolutional block attention module. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 3–19 (2018)

  59. Wu, Z., Su, L., Huang, Q.: Cascaded partial decoder for fast and accurate salient object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3907–3916 (2019)

  60. Yan, Q., Xu, L., Shi, J., Jia, J.: Hierarchical saliency detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1155–1162 (2013)

  61. Yao, X., Han, J., Zhang, D., Nie, F.: Revisiting co-saliency detection: a novel approach based on two-stage multi-view spectral rotation co-clustering. IEEE Trans. Image Process. 26(7), 3196–3209 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  62. Ye, L., Liu, Z., Li, J., Zhao, W.-L., Shen, L.: Co-saliency detection via co-salient object discovery and recovery. IEEE Signal Process. Lett. 22(11), 2073–2077 (2015)

    Article  Google Scholar 

  63. Zha, Z.-J., Wang, C., Liu, D., Xie, H., Zhang, Y.: Robust deep co-saliency detection with group semantic and pyramid attention. In: IEEE Transactions on Neural Networks and Learning Systems (2020)

  64. Zhai, Y., Shah, M.: Visual attention detection in video sequences using spatiotemporal cues. In: Proceedings of the 14th ACM International Conference on Multimedia, pp. 815–824 (2006)

  65. Zhang, D., Han, J., Han, J., Shao, L.: Cosaliency detection based on intrasaliency prior transfer and deep intersaliency mining. IEEE Trans. Neural Netw. Learn. Syst. 27(6), 1163–1176 (2015)

    Article  MathSciNet  Google Scholar 

  66. Zhang, D., Han, J., Li, C., Wang, J: Co-saliency detection via looking deep and wide. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2994–3002 (2015)

  67. Zhang, D., Han, J., Li, C., Wang, J., Li, X.: Detection of co-salient objects by looking deep and wide. Int. J. Comput. Vis. 120(2), 215–232 (2016)

    Article  MathSciNet  Google Scholar 

  68. Zhang, D., Meng, D., Han, J.: Co-saliency detection via a self-paced multiple-instance learning framework. IEEE Trans. Pattern Anal. Mach. Intell. 39(5), 865–878 (2016)

    Article  Google Scholar 

  69. Zhang, F., Bo, D., Zhang, L.: Saliency-guided unsupervised feature learning for scene classification. IEEE Trans. Geosci. Remote Sens. 53(4), 2175–2184 (2014)

    Article  Google Scholar 

  70. Zhang, K., Li, T., Liu, B.,Liu, Q.: Co-saliency detection via mask-guided fully convolutional networks with multi-scale label smoothing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3095–3104 (2019)

  71. Zhang, K., Li, T., Shen, S., Liu, B., Chen, J., Liu, Q.: Adaptive graph convolutional network with attention graph clustering for co-saliency detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9050–9059 (2020)

  72. Zhang, P., Wang, D., Lu, H., Wang, H., Yin, B.: Learning uncertain convolutional features for accurate saliency detection. In: Proceedings of the IEEE International Conference on computer vision, pp. 212–221 (2017)

  73. Zhang, Y., Li, L., Cong, R., Guo, X., Xu, H., Zhang, J.: Co-saliency detection via hierarchical consistency measure. In: 2018 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6. IEEE (2018)

  74. Zhao, J.X., Liu, J.J., Fan, D.P., Cao, Y., Yang, J., Cheng, M.M.: Egnet: Edge guidance network for salient object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 8779–8788 (2019)

  75. Zhao, R., Ouyang, W., Wang, X.: Unsupervised salience learning for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3586–3593 (2013)

  76. Zhou, X., Liu, Z., Sun, G., Wang, X.: Adaptive saliency fusion based on quality assessment. Multimedia Tools Appl. 76(22), 23187–23211 (2017)

    Article  Google Scholar 

  77. Zhou, X., Liu, Z., Sun, G., Ye, L., Wang, X.: Improving saliency detection via multiple kernel boosting and adaptive fusion. IEEE Signal Process. Lett. 23(4), 517–521 (2016)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Northeast Petroleum University, Daqing, China

    Xiufang Wang, Wei Wang, Hongbo Bi & Kang Wang

Authors
  1. Xiufang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hongbo Bi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hongbo Bi.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, X., Wang, W., Bi, H. et al. Reverse collaborative fusion model for co-saliency detection. Vis Comput 38, 3911–3921 (2022). https://doi.org/10.1007/s00371-021-02231-1

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-021-02231-1

Keywords

Search

Navigation