Skip to main content
Log in

RGBD co-saliency detection via multiple kernel boosting and fusion

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

RGBD co-saliency detection, which aims at extracting common salient objects from a group of RGBD images with the additional depth information, has become an emerging branch of saliency detection. In this regard, this paper proposes a novel framework via multiple kernel boosting (MKB) and co-saliency quality based fusion. First, on the basis of pre-segmented regions at multiple scales, the regional clustering by feature bagging is exploited to generate the base co-saliency maps. Then the clustering-based samples selection is performed to select the most similar regions with high saliency from different images in the image set. The selected samples are utilized to learn a MKB-based regressor, which is applied to all regions at multiple scales to generate the MKB-based co-saliency maps. Finally, to make full use of both MKB and clustering-based co-saliency maps, a co-saliency quality criterion is proposed for adaptive fusion to generate the final co-saliency maps. Experimental results on a public RGBD co-saliency detection dataset demonstrate that the proposed co-saliency model outperforms the state-of-the-art co-saliency models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Arbelaez P, Maire M, Fowlkes C, Malik J (2011) Contour detection and hierarchical image segmentation. IEEE Trans Pattern Anal Mach Intell 33(5):898–916

    Article  Google Scholar 

  2. Bezdek JC (2013) Pattern recognition with fuzzy objective function algorithms. Springer Science & Business Media, New York, pp 99–108

    Google Scholar 

  3. Borji A, Itti L (2012) Exploiting local and global patch rarities for saliency detection. In: Proc. of IEEE conference on computer vision pattern recognition, pp 478–485

    Google Scholar 

  4. Cao X, Tao Z, Zhang B, Fu H, Feng W (2014) Self-adaptively weighted co-saliency detection via rank constraint. IEEE Trans Image Process 23(9):4175–4186

    MathSciNet  MATH  Google Scholar 

  5. Catanzaro B, BY S, Sundaram N, Lee Y, Murphy M, Keutzer K (2009) Efficient, high-quality image contour detection. In: Proc. of IEEE conference on computer vision, pp 2381–2388

    Google Scholar 

  6. Cheng MM, Mitra NJ, Huang X, Torr P, SM H (2015) Global contrast based salient region detection. IEEE Trans Pattern Anal Mach Intell 37(3):569–582

    Article  Google Scholar 

  7. Cho M, Kwak S, Schmid C, Ponce J (2015) Unsupervised object discovery and localization in the wild: part-based matching with bottom-up region proposals. In: Proc. of IEEE conference on computer vision and pattern recognition, pp 1201–1210

    Google Scholar 

  8. Fu H, Cao X, Tu Z (2013) Cluster-based co-saliency detection. IEEE Trans Image Process 22(10):3766–3778

    Article  MathSciNet  MATH  Google Scholar 

  9. Fu H, Xu D, Lin S, Liu J (2015) Object-based RGBD image co-segmentation with mutex constraint. In: Proc. of IEEE conference on computer vision and pattern recognition, pp 4428–4436

    Google Scholar 

  10. Guo J, Ren T, Bei J (2016) Salient object detection for RGB-D image via saliency evolution. In: Proc. of IEEE international conference multimedia and expo, pp 1–6

    Google Scholar 

  11. Huang X, Shen C, Boix X, Zhao Q (2015) SALICON: reducing the semantic gap in saliency prediction by adapting deep neural networks. In: Proc. of IEEE international conference on computer vision, pp 262–270

    Google Scholar 

  12. Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell 20(11):1254–1259

    Article  Google Scholar 

  13. Jacobs DE, Goldman DB, Shechtman E (2010) Cosaliency: where people look when comparing images. In: Proc. of ACM symposium on user interface software and technology, pp 219–228

    Google Scholar 

  14. Jiang H, Wang J, Yuan Z, Wu Y, Zheng N, Li S (2013) Salient object detection: a discriminative regional feature integration approach. In: Proc. of IEEE conference on computer vision pattern recognition, pp 2083–2090

    Google Scholar 

  15. Ju R, Liu Y, Ren T, Ge L, Wu G (2015) Depth-aware salient object detection using anisotropic center-surround difference. Signal Process Image Commun 38:115–126

    Article  Google Scholar 

  16. Kolmogorov V, Zabin R (2004) What energy functions can be minimized via graph cuts? IEEE Trans Pattern Anal Mach Intell 26(2):147–159

    Article  Google Scholar 

  17. Lang C, Nguyen T, Katti H, Yadati K, Kankanhalli M, Yan S (2012) Depth matters: influence of depth cues on visual saliency. In: Proc. of European conference on computer vision, pp 101–115

    Google Scholar 

  18. Li H, Ngan KN (2011) A co-saliency model of image pairs. IEEE Trans Image Process 20(12):3365–3375

    Article  MathSciNet  MATH  Google Scholar 

  19. Li H, Meng F, Ngan K (2013) Co-salient object detection from multiple images. IEEE Trans Multimedia 15(8):1896–1909

    Article  Google Scholar 

  20. Liu Z, Zou W, Le Meur O (2014) Saliency tree: a novel saliency detection framework. IEEE Trans Image Process 23(5):1937–1952

    Article  MathSciNet  MATH  Google Scholar 

  21. Liu Z, Zou W, Li L, Shen L, Le Meur O (2014) Co-saliency detection based on hierarchical segmentation. IEEE Signal Process. Lett. 21(1):88–92

    Article  Google Scholar 

  22. Liu N, Han J, Zhang D, Wen S, Liu T (2015) Predicting eye fixations using convolutional neural networks. In: Proc. of IEEE conference on computer vision and pattern recognition, pp 362–370

    Google Scholar 

  23. MacKay D (2003) Information theory, inference and learning algorithms. Cambridge University Press, Cambridge, pp 284–292

    MATH  Google Scholar 

  24. Siva P, Xiang T (2011) Weakly supervised object detector learning with model drift detection. In: Proc. of IEEE international conference on computer vision, pp 343–350

    Google Scholar 

  25. Song H, Liu Z, Du H, Sun G (2016) Depth-aware saliency detection using discriminative saliency fusion. In: Proc. of IEEE international conference on acoustics, speech and signal processing, pp 1626–1630

    Google Scholar 

  26. Song H, Liu Z, Xie Y, Wu L, Huang M (2016) RGBD co-saliency detection via bagging-based clustering. IEEE Signal Process. Lett. 23(12):1722–1726

    Article  Google Scholar 

  27. Song H, Liu Z, Du H, Sun G, Le Meur O, Ren T (2017) Depth-aware salient object detection and segmentation via multiscale discriminative saliency fusion and bootstrap learning. IEEE Trans Image Process 26(9):4204–4216

    Article  MathSciNet  Google Scholar 

  28. Tong N, Lu H, Ruan X, Yang M (2015) Salient object detection via bootstrap learning. In: Proc. of IEEE conference on computer vision pattern recognition, pp 1884–1892

  29. Wang Z, Liu R (2013) Semi-supervised learning for large scale image cosegmentation. In: Proc. of IEEE international conference on computer vision, pp 393–400

    Google Scholar 

  30. Wang W, Shen J, Li X, Porikli F (2015) Robust video object cosegmentation. IEEE Trans Image Process 24(10):3137–3148

    Article  MathSciNet  Google Scholar 

  31. Ye L, Liu Z, Li J, Zhao WL, Shen L (2015) Co-saliency detection via co-salient object discovery and recovery. IEEE Signal Process. Lett. 22(11):2073–2077

    Article  Google Scholar 

  32. Zhang D, Fu H, Han J, Wu F (2016) A review of co-saliency detection technique: fundamentals, applications, and challenges. arXiv preprint, arXiv:1604.07090

  33. Zhou X, Liu Z, Sun G, Ye L, Wang X (2016) Improving saliency detection via multiple kernel boosting and adaptive fusion. IEEE Signal Process Lett 23(4):517–521

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China under Grant No. 61771301, and by the Program for Professor of Special Appointment (Eastern Scholar) at Shanghai Institutions of Higher Learning.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Zhi Liu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, L., Liu, Z., Song, H. et al. RGBD co-saliency detection via multiple kernel boosting and fusion. Multimed Tools Appl 77, 21185–21199 (2018). https://doi.org/10.1007/s11042-017-5576-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-5576-y

Keywords

Navigation