Abstract
Salient object detection in a real-time environment demands high accuracy with less computation time. It is a changeling task to investigate a saliency model which improves saliency detection accuracy as well as reduces computation time. In this paper, we propose a hybrid model that improves detection accuracy with low computational time. The input image is simplified and clustered at multiple scales using SLIC and K-means clustering algorithms respectively. Gaussian Mixture Model (GMM) is developed on various color components of the digital image at multiple scales. The parameters of GMM are learnt using Expectation Maximization (EM) algorithm. The spatial variance of each color component is determined using GMM parameters and hence object position is estimated. Further, spatial variance of color components and object position is exploited to compute saliency map at a scale level. Afterwards, all the saliency maps generated across various scales are linearly combined to produce the final saliency map. The performance of the proposed model is compared in terms of Precision, Recall, F-Measure, Area under the Curve (AUC), Receiver Operating Characteristics (ROC) and Mean Absolute Error (MAE). Extensive experiments on six publicly available datasets viz. MSRA10K, DUT-OMRON, ECSSD, PASCAL-S, SED2, and THUR15K show that the proposed model outperforms or comparable against 11 state-of-the-art methods of the last decade. The key features of the proposed method are object completeness and efficiency in terms of computational time.
Similar content being viewed by others
References
Achanta R, Hemami S, Estrada F, Süsstrunk S (2009) Frequency-tuned salient region detection. In: IEEE International conference on computer vision and pattern recognition (CVPR 2009), CONF, pp 1597–1604
Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Süsstrunk S, et al. (2010) Slic superpixels. Ecole Polytechnique fédéral de Lausssanne (EPFL). Tech Rep 149300:155–162
Alpert S, Galun M, Basri R, Brandt A (2007) Image segmentation by probabilistic bottom-up aggregation and cue integration. In: IEEE Conference on computer vision and pattern recognition, 2007. CVPRn’07. IEEE, pp 1–8
Avidan S, Shamir A (2007) Seam carving for content-aware image resizing. In: ACM Transactions on graphics (TOG), vol 26, ACM, pp 10
Bishop CM (2006) Pattern recognition and machine learning. Springer
Borji A, Cheng MM, Jiang H, Li J (2015) Salient object detection: a benchmark. IEEE Trans Image Process 12(24):5706–5722
Borji A, Itti L (2013) State-of-the-art in visual attention modeling. IEEE Trans Pat Anal Mach Intel 35(1):185–207
Chang KY, Liu TL, Chen HT, Lai SH (2011) Fusing generic objectness and visual saliency for salient object detection. In: 2011 International conference on computer vision, IEEE, pp 914–921
Chen T, Cheng MM, Tan P, Shamir A, Hu SM (2009) Sketch2photo: Internet image montage. In: ACM Transactions on graphics (TOG), vol 28, ACM, pp 124
Cheng MM, Mitra NJ, Huang X, Hu SM (2014) Salientshape: Group saliency in image collections. Vis Comput 30(4):443–453
Cheng MM, Mitra NJ, Huang X, Torr PH, Hu SM (2015) Global contrast based salient region detection. IEEE Trans Pat Anal Mach Intel 37(3):569–582
Duan L, Wu C, Miao J, Qing L, Fu Y (2011) Visual saliency detection by spatially weighted dissimilarity. In: 2011 IEEE conference on Computer vision and pattern recognition (CVPR), IEEE, pp 473–480
Erdem E, Erdem A (2013) Visual saliency estimation by nonlinearly integrating features using region covariances. J Vis 13(4):11–11
Everingham M, Van Gool L, Williams CK, Winn J, Zisserman A (2010) The pascal visual object classes (voc) challenge. Int J Comput Vis 88 (2):303–338
GAO D (2007) The discriminant center-surround hypothesis for bottom-up saliency. In: Proc Conference on neural information processing systems (NIPS), 2007
GOFERMAN S (2010) Context-aware saliency detection. In: Proc IEEE Conf Computer vision and pattern recognition, 2010, pp 2376–2383
Gopalakrishnan V, Hu Y, Rajan D (2010) Random walks on graphs for salient object detection in images. IEEE Trans Image Process 19(12):3232–3242
Harel J, Koch C, Perona P (2007) Graph-based visual saliency. In: Advances in neural information processing systems, pp 545–552
He S, Lau RW, Liu W, Huang Z, Yang Q (2015) Supercnn: a superpixelwise convolutional neural network for salient object detection. Int J Comput Vis 115(3):330–344
Hou X, Zhang L (2007) Saliency detection: a spectral residual approach. In: IEEE conference on computer vision and pattern recognition, 2007. CVPR’07. IEEE, pp 1–8
Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pat Anal Mach Intel 20(11):1254–1259
Jiang B, Zhang L, Lu H, Yang C, Yang MH (2013) Saliency detection via absorbing markov chain. In: Proceedings of the IEEE international conference on computer vision, pp 1665–1672
Jiang H, Wang J, Yuan Z, Liu T, Zheng N, Li S (2011) Automatic salient object segmentation based on context and shape prior. In: BMVC, vol 6, pp 9
Jiang H, Wang J, Yuan Z, Wu Y, Zheng N, Li S (2013) Salient object detection: a discriminative regional feature integration approach
Judd T, Durand F, Torralba A (2012) A benchmark of computational models of saliency to predict human fixations. Technical report, MIT tech report
Ko BC, Nam JY (2006) Object-of-interest image segmentation based on human attention and semantic region clustering. JOSA A 23(10):2462–2470
Kumar N (2018) Thresholding in salient object detection: a survey. Multimedia Tools and Applications 77(15):19,139–19,170
Li G, Yu Y (2015) Visual saliency based on multiscale deep features. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5455–5463
Li J, Levine MD, An X, Xu X, He H (2013) Visual saliency based on scale-space analysis in the frequency domain. IEEE Trans Pat Anal Mach Intel 35(4):996–1010. https://doi.org/10.1109/TPAMI.2012.147
Li X, Lu H, Zhang L, Ruan X, Yang MH (2013) Saliency detection via dense and sparse reconstruction. In: Proceedings of the IEEE international conference on computer vision, pp 2976–2983
Li Y, Hou X, Koch C, Rehg JM, Yuille AL (2014) The secrets of salient object segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 280–287
Liu GH, Yang JY (2019) Exploiting color volume and color difference for salient region detection. IEEE Trans Image Process 28(1):6–16
Liu T, Yuan Z, Sun J, Wang J, Zheng N, Tang X, Shum HY (2011) Learning to detect a salient object. IEEE Trans Pat Anal Mach Intel 33 (2):353–367
Ma YF, Zhang HJ (2003) Contrast-based image attention analysis by using fuzzy growing. In: Proceedings of the eleventh ACM international conference on Multimedia, ACM, pp 374–381
Marat S, Guironnet M, Pellerin D (2007) Video summarization using a visual attention model. In: 2007 15Th european signal processing conference, IEEE, pp 1784–1788
Margolin R, Tal A, Zelnik-Manor L (2013) What makes a patch distinct?. Proceedings of the IEEE Conference on computer vision and pattern recognition, pp 1139–1146
Moon TK (1996) The expectation-maximization algorithm. IEEE Signal Processing Magazine 13(6):47–60
Murray N, Vanrell M, Otazu X, Parraga CA (2011) Saliency estimation using a non-parametric low-level vision model. In: CVPR 2011, IEEE, pp 433–440
Park M, Kumar M, Loui AC (2011) Saliency detection using region-based incremental center-surround distance. In: 2011 IEEE International symposium on multimedia, IEEE, pp 249–256
Perazzi F, Krähenbühl P, Pritch Y, Hornung A (2012) Saliency filters: Contrast based filtering for salient region detection. In: 2012 IEEE Conference on computer vision and pattern recognition, IEEE, pp 733–740
Ren Z, Gao S, Chia LT, Tsang IWH (2013) Region-based saliency detection and its application in object recognition. IEEE Trans Circ Syst Vid Technol 24(5):769–779
Ren Z, Hu Y, Chia LT, Rajan D (2010) Improved saliency detection based on superpixel clustering and saliency propagation. In: Proceedings of the 18th ACM international conference on Multimedia, ACM, pp 1099–1102
Roy S, Das S (2013) Spatial variance of color and boundary statistics for salient object detection. In: 2013 Fourth national conference on computer vision, pattern recognition, image processing and graphics (NCVPRIPG), IEEE , pp 1–4
Seo HJ, Milanfar P (2009) Static and space-time visual saliency detection by self-resemblance. J Vis 9(12):15–15
Siagian C, Itti L (2007) Rapid biologically-inspired scene classification using features shared with visual attention. IEEE Trans Pat Anal Mach Intel 29(2):300–312
Singh N, Arya R, Agrawal R (2016) A convex hull approach in conjunction with gaussian mixture model for salient object detection. Digital Signal Processing 55:22–31
Singh N, Arya R, Agrawal R (2017) A novel position prior using fusion of rule of thirds and image center for salient object detection. Multimedia Tools and Applications 76(8):10,521–10,538
Singh N, Arya R, Agrawal R (2018) Performance enhancement of salient object detection using superpixel based gaussian mixture model. Multimedia Tools and Applications, pp 1–19
Snowden RJ (2002) Visual attention to color: Parvocellular guidance of attentional resources? Psychol Sci 13(2):180–184
Yan Q, Xu L, Shi J, Jia J (2013) Hierarchical saliency detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1155–1162
Yang C, Zhang L, Lu H, Ruan X, Yang MH (2013) Saliency detection via graph-based manifold ranking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3166–3173
Zeng Y, Feng M, Lu H, Yang G, Borji A (2018) An unsupervised game-theoretic approach to saliency detection. IEEE Trans Image Process 27(9):4545–4554
Zhai Y, Shah M (2006) Visual attention detection in video sequences using spatiotemporal cues. In: Proceedings of the 14th ACM international conference on Multimedia, ACM, pp 815–824
Zhang J, Sclaroff S (2016) Exploiting surroundedness for saliency detection: a boolean map approach. IEEE Trans Pat Anal Mach Intel 38(5):889–902
Zhang L, Tong MH, Marks TK, Shan H, Cottrell GW (2008) Sun: a bayesian framework for saliency using natural statistics. J Vis 8(7):32–32
Zhang W, Wu QJ, Wang G, Yin H (2010) An adaptive computational model for salient object detection. IEEE Transactions on Multimedia 12 (4):300–316
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Singh, V.K., Kumar, N. & Singh, N. A hybrid approach using color spatial variance and novel object position prior for salient object detection. Multimed Tools Appl 79, 30045–30067 (2020). https://doi.org/10.1007/s11042-020-09467-4
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-020-09467-4