Skip to main content
SpringerLink
Log in

Combining background information and a top-down model for computing salient objects

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Predicting the salient object region in real scenes has progressed significantly in recent years. In this work, we propose a novel method for computing salient object regions by combining background information and a top-down visual saliency model, which is well-suited for locating category-specific salient objects in cluttered real scenes. First, we used a robust background measure to acquire clean saliency maps by optimizing background information. Second, we learned a top-down saliency object model by combining a class-specific codebook and conditional random fields (CRFs) during the training phase. Furthermore, our model used the locality-constrained linear codes as latent CRF variables. Finally, we computed salient object regions by combining the robust background measure and top-down model. Experimental results on the Graz-02 and PASCAL VOC2007 datasets show that our method creates much better saliency maps than current state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  1. Achanta R, Hemami S, Estrada F, Susstrunk S (2009) Frequency-tuned salient region detection. In: IEEE conference on computer vision and pattern recognition, 2009. CVPR 2009, pp 1597–1604

  2. Aytekin C, Ozan EC, Kiranyaz S, Gabbouj M (2016) Extended quantum cuts for unsupervised salient object extraction. Multimedia Tools and Application:1–21

  3. Bodesheim P (2011) Spectral clustering of rois for object discovery. In: Proceedings of the 33rd international conference on pattern recognition, pp 450–455

  4. Borji A (2015) What is a salient object? A dataset and a baseline model for salient object detection. IEEE Trans Image Process 24(2):742–756

    Article  MathSciNet  Google Scholar 

  5. Borji A, Cheng M, Jiang H, Li J (2015) Salient object detection: a benchmark. IEEE Trans Image Process 24(12):5706–5722

    Article  MathSciNet  Google Scholar 

  6. Cheng MM, Warrell J, Lin WY, Zheng S, Vineet V, Crook N (2013) Efficient salient region detection with soft image abstraction. In: IEEE international conference on computer vision 2013. ICCV 2013, pp 1529–1536

  7. Ding Y, Xiao J, Yu J (2011) Importance filtering for image retargeting. In: IEEE conference on computer vision and pattern recognition. CVPR 2011, pp 89–96

  8. Donoser M, Urschler M, Hirzer M, Bischof H (2009) Saliency driven total variation segmentation. In: 2009 IEEE 12th international conference on computer vision. ICCV 2009, pp 817–824

  9. Ehinger KA, Hidalgo-Sotelo B, Torralba A, Oliva A (2009) Modeling search for people in 900 scenes: a combined source model of eye guidance. Vis Cogn 17(6-7):945–978

    Article  Google Scholar 

  10. Everingham M, Gool LV, Williams CKI, Winn J, Zisserman A (2010) The pascal visual object classes (voc) challenge. Int J Comput Vis 88(2):303–338

    Article  Google Scholar 

  11. Fei-Fei L, Perona P (2005) A bayesian hierarchical model for learning natural scene categories. In: IEEE computer society conference on computer vision and pattern recognition. CVPR 2005, vol 2, pp 524–531

  12. Gao D, Han S, Vasconcelos N (2009) Discriminant saliency, the detection of suspicious coincidences, and applications to visual recognition. IEEE Trans Pattern Anal Mach Intell 31(6):989–1005

    Article  Google Scholar 

  13. Goferman S, Zelnikmanor L, Tal A (2012) Context-aware saliency detection. IEEE Trans Pattern Anal Mach Intell 34(10):1915–1926

    Article  Google Scholar 

  14. Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell 20(11):1254–1259

    Article  Google Scholar 

  15. Jiang H, Wang J, Yuan Z, Wu Y, Zheng N, Li S (2014) Salient object detection: a discriminative regional feature integration approach. In: IEEE conference on computer vision and pattern recognitio. CVPR 2014, pp 2083–2090

  16. Joachims T, Finley T, Yu CNJ (2009) Cutting-plane training of structural svms. mach learn. Mach Learn 77(1):27–59

    Article  MATH  Google Scholar 

  17. Judd T, Ehinger K, Durand F, Torralba A (2009) Learning to predict where humans look. In: 2009 IEEE 12th international conference on computer vision. ICCV 2009, pp 2106–2113

  18. Kanan C, Tong MH, Zhang L, Cottrell GW (2009) Sun: top-down saliency using natural statistics. Vis Cogn 17(6-7):979–1003

    Article  Google Scholar 

  19. Khan N, TappenMF (2013) Discriminative dictionary learning with spatial priors. In: IEEE international conference on image processing. ICIP 2013, pp 166–170

  20. Kocak A, Cizmeciler K, Erdem A, Erkut E (2014) Top down saliency estimation via superpixel-based discriminative dictionaries. In: Proceedings of the british machine vision conference. BMVC 2014, BMVA press

  21. Lazebnik S, Schmid C, Ponce J (2006) Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: IEEE computer society conference on computer vision and pattern recognition. CVPR 2006, pp 2169–2178

  22. Liang Z, Wang M, Zhou X, Lin L, Li W (2012) Salient object detection based on regions. Multimedia Tools and Applications 68(3):517–544

    Article  Google Scholar 

  23. Liu T, Yuan Z, Sun J, Wang J, Zheng N, Tang X, Shum H (2011) Learning to detect a salient object. IEEE Trans Pattern Anal Mach Intell 33(2):353–367

    Article  Google Scholar 

  24. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(60):91–110

    Article  Google Scholar 

  25. Ma Y, Hua X, Lu L, Zhang H (2005) A generic framework of user attention model and its application in video summarization. IEEE Transactions on Multimedia 7(5):907–919

    Article  Google Scholar 

  26. Marchesotti L, Cifarelli C, Csurka G (2009) A framework for visual saliency detection with applications to image thumbnailing. In: International conference on computer vision. IEEE international conference on computer vision. ICCV 2009, pp 2232–2239

  27. Nguyen TV, Kankanhalli M (2016) As-similar-as-possible saliency fusion. Multimedia Tools and Applications:1–19

  28. Opelt A, Pinz A, Fussenegger M, Auer P (2006) Generic object recognition with boosting. IEEE Trans Pattern Anal Mach Intell 28(3):416–31

    Article  MATH  Google Scholar 

  29. Perazzi F, Kr?henbhl P, Pritch Y, Hornung A (2012) Saliency filters: contrast based filtering for salient region detection. In: IEEE conference on computer vision and pattern recognition. CVPR 2012, pp 733–740

  30. Rother C, Kolmogorov V, Blake A (2004) grabcut: interactive foreground extraction using iterated graph cuts 23:309–314

  31. Schölkopf B, Platt J, Hofmann T (2007) Fast discriminative visual codebooks using randomized clustering forests. In: Conference on neural information processing systems. NIPS 2007, pp 985–992

  32. Sun J, Ling H (2013) Scale and object aware image thumbnailing. Int J Comput Vis 104(2):135–153

    Article  MathSciNet  Google Scholar 

  33. Tang C, Hou C, Wang P, Song Z (2016) Salient object detection using color spatial distribution and minimum spanning tree weight. Multimedia Tools and Applications 75(12):6963–6978

    Article  Google Scholar 

  34. Torralba A, Oliva A (2006) Contextual guidance of eye movements and attention in real-world scenes: the role of global features in object search. Psychol Rev 113(4):766–86

    Article  Google Scholar 

  35. Wang J, Yang J, Yu K, Lv F, Huang T, Gong Y (2010) Locality-constrained linear coding for image classification. In: IEEE computer society conference on computer vision and pattern recognition. CVPR 2010, pp 3360–3367

  36. Wei Y, Wen F, Zhu W, Sun J (2012) Geodesic saliency using background priors. In: European conference on computer vision. ECCV 2012, pp 29–42

  37. Wu B, Xu L (2014) Integrating bottom-up and top-down visual stimulus for saliency detection in news video. Multimedia Tools and Applications 73(3):1053–1075

    Article  Google Scholar 

  38. Xu X, Mu N, Chen L, Zhang X (2015) Hierarchical salient object detection model using contrast-based saliency and color spatial distribution. Multimedia Tools and Applications 75(5):2667–2679

    Article  Google Scholar 

  39. Yan Q, Xu L, Shi J, Jia J (2013) Hierarchical saliency detection 9(4):1155–1162

  40. Yang C, Zhang L, Lu H, Ruan X, Yang MH (2013) Saliency detection via graph-based manifold ranking. In: IEEE conference on computer vision and pattern recognition. CVPR 2013, pp 3166–3173

  41. Yang J, Yu K, Gong Y, Huang T (2009) Linear spatial pyramid matching using sparse coding for image classification. In: IEEE conference on computer vision and pattern recognition. CVPR 2009, pp 1794–1801

  42. Yang MH, Yang J (2012) Top-down visual saliency via joint crf and dictionary learning. In: IEEE Conference on computer vision and pattern recognition. CVPR 2012, pp 2296–2303

  43. Yang Z, Xiong H (2015) Image classification based on saliency coding with category-specific codebooks. Neurocomputing 184:188–195

    Article  Google Scholar 

  44. Yang Z, Xiong H (2016) Computing object-based saliency via locality-constrained linear coding and conditional random fields. Vis Comput:1–11

  45. Yu K, Zhang T, Gong Y (2009) Nonlinear learning using local coordinate coding. In: Advances in neural information processing systems 22: conference on neural information processing systems. NIPS 2009, pp 2223–2231

  46. Zhu W, Liang S, Wei Y, Sun J (2014) Saliency optimization from robust background detection. In: Computer vision and pattern recognition. CVPR 2014, pp 2814–2821

Download references

Acknowledgments

This work was supported by the National Natural Foundation of China under grant no. 61375008. We thank LetPub (www.letpub.com) for its linguistic assistance during the preparation of this manuscript.

Author information

Authors and Affiliations

  1. Department of Automation, Shanghai Jiao Tong University, 800 Dongchuan Rd., Shanghai, 200240, China

    Zhen Yang & Huilin Xiong

  2. School of Communication and Electronics, Jiangxi Science and Technology Normal University, 605 Fenglin Rd., Nanchang, China

    Fan Yang

Authors
  1. Zhen Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fan Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Huilin Xiong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhen Yang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, Z., Yang, F. & Xiong, H. Combining background information and a top-down model for computing salient objects. Multimed Tools Appl 76, 20815–20832 (2017). https://doi.org/10.1007/s11042-016-4005-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-016-4005-y

Keywords

Search

Navigation