Abstract
RGBD co-saliency detection, which aims at extracting common salient objects from a group of RGBD images with the additional depth information, has become an emerging branch of saliency detection. In this regard, this paper proposes a novel framework via multiple kernel boosting (MKB) and co-saliency quality based fusion. First, on the basis of pre-segmented regions at multiple scales, the regional clustering by feature bagging is exploited to generate the base co-saliency maps. Then the clustering-based samples selection is performed to select the most similar regions with high saliency from different images in the image set. The selected samples are utilized to learn a MKB-based regressor, which is applied to all regions at multiple scales to generate the MKB-based co-saliency maps. Finally, to make full use of both MKB and clustering-based co-saliency maps, a co-saliency quality criterion is proposed for adaptive fusion to generate the final co-saliency maps. Experimental results on a public RGBD co-saliency detection dataset demonstrate that the proposed co-saliency model outperforms the state-of-the-art co-saliency models.
Similar content being viewed by others
References
Arbelaez P, Maire M, Fowlkes C, Malik J (2011) Contour detection and hierarchical image segmentation. IEEE Trans Pattern Anal Mach Intell 33(5):898–916
Bezdek JC (2013) Pattern recognition with fuzzy objective function algorithms. Springer Science & Business Media, New York, pp 99–108
Borji A, Itti L (2012) Exploiting local and global patch rarities for saliency detection. In: Proc. of IEEE conference on computer vision pattern recognition, pp 478–485
Cao X, Tao Z, Zhang B, Fu H, Feng W (2014) Self-adaptively weighted co-saliency detection via rank constraint. IEEE Trans Image Process 23(9):4175–4186
Catanzaro B, BY S, Sundaram N, Lee Y, Murphy M, Keutzer K (2009) Efficient, high-quality image contour detection. In: Proc. of IEEE conference on computer vision, pp 2381–2388
Cheng MM, Mitra NJ, Huang X, Torr P, SM H (2015) Global contrast based salient region detection. IEEE Trans Pattern Anal Mach Intell 37(3):569–582
Cho M, Kwak S, Schmid C, Ponce J (2015) Unsupervised object discovery and localization in the wild: part-based matching with bottom-up region proposals. In: Proc. of IEEE conference on computer vision and pattern recognition, pp 1201–1210
Fu H, Cao X, Tu Z (2013) Cluster-based co-saliency detection. IEEE Trans Image Process 22(10):3766–3778
Fu H, Xu D, Lin S, Liu J (2015) Object-based RGBD image co-segmentation with mutex constraint. In: Proc. of IEEE conference on computer vision and pattern recognition, pp 4428–4436
Guo J, Ren T, Bei J (2016) Salient object detection for RGB-D image via saliency evolution. In: Proc. of IEEE international conference multimedia and expo, pp 1–6
Huang X, Shen C, Boix X, Zhao Q (2015) SALICON: reducing the semantic gap in saliency prediction by adapting deep neural networks. In: Proc. of IEEE international conference on computer vision, pp 262–270
Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell 20(11):1254–1259
Jacobs DE, Goldman DB, Shechtman E (2010) Cosaliency: where people look when comparing images. In: Proc. of ACM symposium on user interface software and technology, pp 219–228
Jiang H, Wang J, Yuan Z, Wu Y, Zheng N, Li S (2013) Salient object detection: a discriminative regional feature integration approach. In: Proc. of IEEE conference on computer vision pattern recognition, pp 2083–2090
Ju R, Liu Y, Ren T, Ge L, Wu G (2015) Depth-aware salient object detection using anisotropic center-surround difference. Signal Process Image Commun 38:115–126
Kolmogorov V, Zabin R (2004) What energy functions can be minimized via graph cuts? IEEE Trans Pattern Anal Mach Intell 26(2):147–159
Lang C, Nguyen T, Katti H, Yadati K, Kankanhalli M, Yan S (2012) Depth matters: influence of depth cues on visual saliency. In: Proc. of European conference on computer vision, pp 101–115
Li H, Ngan KN (2011) A co-saliency model of image pairs. IEEE Trans Image Process 20(12):3365–3375
Li H, Meng F, Ngan K (2013) Co-salient object detection from multiple images. IEEE Trans Multimedia 15(8):1896–1909
Liu Z, Zou W, Le Meur O (2014) Saliency tree: a novel saliency detection framework. IEEE Trans Image Process 23(5):1937–1952
Liu Z, Zou W, Li L, Shen L, Le Meur O (2014) Co-saliency detection based on hierarchical segmentation. IEEE Signal Process. Lett. 21(1):88–92
Liu N, Han J, Zhang D, Wen S, Liu T (2015) Predicting eye fixations using convolutional neural networks. In: Proc. of IEEE conference on computer vision and pattern recognition, pp 362–370
MacKay D (2003) Information theory, inference and learning algorithms. Cambridge University Press, Cambridge, pp 284–292
Siva P, Xiang T (2011) Weakly supervised object detector learning with model drift detection. In: Proc. of IEEE international conference on computer vision, pp 343–350
Song H, Liu Z, Du H, Sun G (2016) Depth-aware saliency detection using discriminative saliency fusion. In: Proc. of IEEE international conference on acoustics, speech and signal processing, pp 1626–1630
Song H, Liu Z, Xie Y, Wu L, Huang M (2016) RGBD co-saliency detection via bagging-based clustering. IEEE Signal Process. Lett. 23(12):1722–1726
Song H, Liu Z, Du H, Sun G, Le Meur O, Ren T (2017) Depth-aware salient object detection and segmentation via multiscale discriminative saliency fusion and bootstrap learning. IEEE Trans Image Process 26(9):4204–4216
Tong N, Lu H, Ruan X, Yang M (2015) Salient object detection via bootstrap learning. In: Proc. of IEEE conference on computer vision pattern recognition, pp 1884–1892
Wang Z, Liu R (2013) Semi-supervised learning for large scale image cosegmentation. In: Proc. of IEEE international conference on computer vision, pp 393–400
Wang W, Shen J, Li X, Porikli F (2015) Robust video object cosegmentation. IEEE Trans Image Process 24(10):3137–3148
Ye L, Liu Z, Li J, Zhao WL, Shen L (2015) Co-saliency detection via co-salient object discovery and recovery. IEEE Signal Process. Lett. 22(11):2073–2077
Zhang D, Fu H, Han J, Wu F (2016) A review of co-saliency detection technique: fundamentals, applications, and challenges. arXiv preprint, arXiv:1604.07090
Zhou X, Liu Z, Sun G, Ye L, Wang X (2016) Improving saliency detection via multiple kernel boosting and adaptive fusion. IEEE Signal Process Lett 23(4):517–521
Acknowledgements
This work was supported by the National Natural Science Foundation of China under Grant No. 61771301, and by the Program for Professor of Special Appointment (Eastern Scholar) at Shanghai Institutions of Higher Learning.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Wu, L., Liu, Z., Song, H. et al. RGBD co-saliency detection via multiple kernel boosting and fusion. Multimed Tools Appl 77, 21185–21199 (2018). https://doi.org/10.1007/s11042-017-5576-y
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-017-5576-y