Skip to main content
SpringerLink
Log in

Learning adaptive contrast combinations for visual saliency detection

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Visual saliency detection plays a significant role in the fields of computer vision. In this paper, we introduce a novel saliency detection method based on weighted linear multiple kernel learning (WLMKL) framework, which is able to adaptively combine different contrast measurements in a supervised manner. As most influential factor is contrast operation in bottom-up visual saliency, an average weighted corner-surround contrast (AWCSC) is first designed to measure local visual saliency. Combined with common-used center-surrounding contrast (CESC) and global contrast (GC), three types of contrast operations are fed into our WLMKL framework to produce the final saliency map. We show that the assigned weights for each contrast feature maps are always normalized in our WLMKL formulation. In addition, the proposed approach benefits from the advantages of the contribution of each individual contrast feature maps, yielding more robust and accurate saliency maps. We evaluated our method for two main visual saliency detection tasks: human fixed eye prediction and salient object detection. The extensive experimental results show the effectiveness of the proposed model, and demonstrate the integration is superior than individual subcomponent.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Achanta R, et al (2009) Frequency-tuned salient region detection. In: CVPR, pp 1597–1604

  2. Alexe B, Deselaers T, Ferrari V (2010) What is an object?. In: CVPR, pp 73–80

  3. Bach FR, et al (2004) Multiple kernel learning, conic duality, and the SMO algorithm. In: ICML, pp 6–14

  4. Bonnans F (2006) Optimisation continue. Dunod

  5. Borji A (2012) Boosting bottom-up and top-down visual features for saliency estimation. In: CVPR, pp 438–445

  6. Borji A, et al (2015) Salient object detection: a benchmark. TIP 24(12):5706–5722

    MathSciNet  MATH  Google Scholar 

  7. Bruce N, Tsotsos J (2006) Saliency based on information maximization. In: NIPS, pp 155–162

  8. Bucak S, et al (2014) Multiple kernel learning for visual object recognition: a review. TIP 36(7):1354–1369

    Google Scholar 

  9. Chang KY, et al (2011) Fusing generic objectness and visual saliency for salient object detection. In: ICCV, pp 914–921

  10. Cheng MM, et al (2011) Global contrast based salient region detection. In: CVPR, pp 409–416

  11. Cornia M, et al (2016) A deep multi-level network for saliency prediction. In: ICPR, pp 3488–3493

  12. Duan L, et al (2011) Visual saliency detection by spatially weighted dissimilarity

  13. Fernandez-Carbajales V, et al (2016) Visual attention based on a joint perceptual space of color and brightness for improved video tracking. Pattern Recogn 60:571–584

    Google Scholar 

  14. Fu Y, et al (2008) Saliency cuts: an automatic approach to object segmentation. In: ICPR, pp 1–4

  15. Gao DS, et al (2008) On the plausibility of the discriminant center-surround hypothesis for visual saliency. J Vis 8(7):13–25

    Google Scholar 

  16. Gao DH, Han S, Vasconcelos N (2009) Discriminant saliency, the detection of suspicious coincidences, and applications to visual recognition. TPAMI 31(6):989–1005

    Google Scholar 

  17. Goferman S, et al (2012) Context-aware saliency detection. TPAMI 34 (10):1915–1926

    Google Scholar 

  18. Gopalakrishnan V, et al (2009) Salient region detection by modeling distributions of color and orientation. TMM 11(5):892–905

    Google Scholar 

  19. Guo C, Zhang L (2010) A novel multiresolution spatiotemporal saliency detection model and its applications in image and video compression. TIP 19(1):185–198

    MathSciNet  MATH  Google Scholar 

  20. Harel J, et al (2006) Graph-based visual saliency. In: NIPS, pp 545–552

  21. Hou X, Zhang L (2007) Saliency detection: a spectral residual approach. In: CVPR, pp 1–8

  22. Hou X, Zhang L (2008) Dynamic visual attention: searching for coding length increments. In: NIPS, pp 681–688

  23. Huang X, et al (2015) SALICON: reducing the semantic gap in saliency prediction by adapting deep neural networks. In: ICCV, pp 262–270

  24. Huchuan L, et al (2017) Co-bootstrapping saliency. TIP 26(1):414–425

    MathSciNet  MATH  Google Scholar 

  25. Itti L (1998) Others: a model of saliency-based visual attention for rapid scene analysis. TPAMI 20(11):1254–1259

    Google Scholar 

  26. Itti L, Koch C (2001) Computational modelling of visual attention. Nat Rev Neurosci 2(3):194–203

    Google Scholar 

  27. Jetley S, et al (2016) End-to-end saliency mapping via probability distribution prediction. In: CVPR, pp 5753–5761

  28. Jing PG, Su YT, Nie LQ, Bai X, Liu J, Wang M (2018) Low-rank multi-view embedding learning for micro-video popularity prediction. TKDE 30(8):1519–1532

    Google Scholar 

  29. Jing PG, Su YT, Nie LQ, Gu HM, Liu J, Wang M (2018) A framework of joint low-rank and sparse regression for image memorability prediction CSVT. https://doi.org/10.1109/TCSVT.2018.2832095

  30. John ST, Cristianini N (2004) Kernel methods for pattern analysis. Cambridge University Press, Cambridge

    Google Scholar 

  31. Judd T, et al (2009) Learning to predict where humans look. In: ICCV, pp 2106–2113

  32. Kadir T, Brady M (2001) Saliency, scale and image description. IJCV 45 (2):83–105

    MATH  Google Scholar 

  33. Koch C, Ullman S (1985) Shifts in selective visual attention: towards the underlying neural circuitry. Matters Int 4(1):219–227

    Google Scholar 

  34. Kruthiventi SSS, et al (2017) Deepfix: a fully convolutional neural network for predicting human eye fixations. TIP 26(9):4446–4456

    MathSciNet  MATH  Google Scholar 

  35. Kummerer M, et al (2015) Deep gaze I: boosting saliency prediction with feature maps trained on ImageNet. In: ICLRW, pp 262–270

  36. Liu N, Han JJ (2016) DHSNet: Deep hierarchical saliency network for salient object detection. In: CVPR, pp 678–686

  37. Liu W, et al (2018) Learning to predict eye fixations via multiresolution convolutional neural networks. TNNLS 29(2):392–404

    MathSciNet  Google Scholar 

  38. Lu H, et al (2017) Wound intensity correction and segmentation with convolutional neural networks. Concurrency and Computation: Practice and Experience 29(6):3927–3737

    Google Scholar 

  39. Lu H, et al (2018) Brain intelligence: go beyond artificial intelligence. Mobile Netw Appl 23(2):368–375

    Google Scholar 

  40. Lu H, et al (2018) Low illumination underwater light field images reconstruction using deep convolutional neural networks. Futur Gener Comput Syst 82(6):142–148

    Google Scholar 

  41. Ma YF, Zhang HJ (2003) Contrast-based image attention analysis by using fuzzy growing. In: ACMMM, pp 374–381

  42. Ma Q, Zhang L (2008) Image quality assessment with visual attention. In: ICPR, pp 1–4

  43. Mairal J, et al (2010) Online learning for matrix factorization and sparse coding. J Mach Learn Res 11(1):19–60

    MathSciNet  MATH  Google Scholar 

  44. Marchesotti L, et al (2009) A framework for visual saliency detection with applications to image thumbnailing. In: ICCV, pp 2232–2239

  45. Mehmet G, et al (2011) Multiple kernel learning algorithms. JMLR 12 (7):2211–2268

    MathSciNet  MATH  Google Scholar 

  46. Nuthmann A, Henderson JM (2010) Object-based attentional selection in scene viewing. J Vis 10(8):2237–2242

    Google Scholar 

  47. Olshausen BA, et al (1996) Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381(7):607–609

    Google Scholar 

  48. Pan J, et al (2016) Shallow and deep convolutional networks for saliency prediction. In: CVPR, pp 598–606

  49. Perazzi F, et al (2012) Saliency filters: contrast based filtering for salient region detection. In: CVPR, pp 733–740

  50. Quan Z, et al (2018) Weighted linear multiple kernel learning for saliency detection. In: ROSENET

  51. Shen X, Wu Y (2012) A unified approach to salient object detection via low rank matrix recovery. In: CVPR, pp 853–860

  52. Simoncelli EP, et al (2001) Natural image statistics and neural representation. Annu Rev Neurosci 24(1):1193–1216

    MathSciNet  Google Scholar 

  53. Sonnenburg S, et al (2006) Large scale multiple kernel learning. JMLR 7 (1):1531–1565

    MathSciNet  MATH  Google Scholar 

  54. Sonnenburg S, et al (2012) Object detection with dog scale-space: a multiple kernel learning approach. TIP 21(8):3744–3756

    MathSciNet  MATH  Google Scholar 

  55. Srinivas SS, et al (2016) Saliency unified: a deep architecture for simultaneous eye fixation prediction and salient object segmentation. In: CVPR, pp 5781–5790

  56. Tatler BW, et al (2015) Visual correlates of fixation selection: effects of scale and time. Vis Res 45(5):643–659

    Google Scholar 

  57. Thiagarajan J, et al (2014) Multiple kernel sparse representations for supervised and unsupervised learning. TIP 23(7):2905–2915

    MathSciNet  MATH  Google Scholar 

  58. Tian H, et al (2014) Salient region detection by fusing bottom-up and top-down features extracted from a single image. TIP 23(10):4389–4398

    MathSciNet  MATH  Google Scholar 

  59. Torralba A, et al (2003) Modeling global scene factors in attention. JOSA A 20(7):1407–1418

    Google Scholar 

  60. Treisman AM, Gelade G (1980) A feature-integration theory of attention. Cogn Psychol 12(1):97–136

    Google Scholar 

  61. Varma M, Ray D (2007) Learning the discriminative power-invariance trade-off. In: ICCV, pp 1–8

  62. Vladimir V (1993) The nature of statistical learning theory. Springer, Berlin

    Google Scholar 

  63. Wang LJ, Lu HC, Wang YF, Feng MY, Wang D, Yin BC, Ruan X (2017) Learning to detect salient objects with image-level supervision. In: CVPR, pp 3796–3805

  64. Wang W, et al (2018) Video salient object detection via fully convolutional networks. TIP 27(1):38–49

    MathSciNet  MATH  Google Scholar 

  65. Xu ZL, et al (2010) Simple and efficient multiple kernel learning by group lasso. In: ICML, pp 1175–1182

  66. Yin L, et al (2014) The secrets of salient object segmentation. In: CVPR, pp 280–287

  67. Yu JG, et al (2016) A computational model for object-based visual saliency: spreading attention along gestalt cues. TMM 18(2):273–286

    Google Scholar 

  68. Zhang LY, et al (2008) SUN: A Bayesian framework for saliency using natural statistics. J Vis 8(7):32–42

    Google Scholar 

  69. Zhang PP, Wang D, Lu HC, Wang HY, Ruan X (2017) Amulet: aggregating multi-level convolutional features for salient object detection. In: ICCV, pp 202–211

  70. Zhang PP, Wang D, Lu HC, Wang HY, Yin BC (2017) Learning uncertain convolutional features for accurate saliency detection. In: ICCV, pp 212–221

  71. Zhou Q, et al (2013) On contrast combinations for visual saliency detection. In: ICIP, pp 2665–2669

  72. Zhou Q, et al (2014) Salient object detection using window mask transferring with multi-layer background contrast. In: ACCV, pp 221–235

Download references

Acknowledgements

The authors would like to thank all the anonymous reviewers for their valuable comments and suggestions. This work was partly supported by the National Science Foundation (Grant No. IIS-1302164), the National Natural Science Foundation of China (Grant No. 61876093, 61881240048, 61671253, 61701252, 61762021), Natural Science Foundation of Jiangsu Province (Grant No. BK20181393, BK20150849, BK20160908), Huawei Innovation Research Program (HIRP2018), and Natural Science Foundation of Guizhou Province (Grant No.[2017]1130).

Author information

Authors and Affiliations

  1. National Engineering Research Center of Communications and Networking, Nanjing University of Posts & Telecommunications, Nanjing, People’s Republic of China

    Quan Zhou, Yawen Fan, Xiaofu Wu & Baoyu Zheng

  2. State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, People’s Republic of China

    Quan Zhou

  3. Huawei Technologies Co. Ltd., ShenZhen, People’s Republic of China

    Jie Cheng

  4. Department of Mechanical and Control Engineering, Kyushu Institute of Technology, Kitakyushu, Japan

    Huimin Lu

  5. School of Internet of Things, Nanjing University of Posts & Telecommunications, Nanjing, People’s Republic of China

    Suofei Zhang

  6. School of Big Data and Computer Science, Guizhou Normal University, Guiyang, People’s Republic of China

    Weihua Ou

  7. Department of Computer and Information Sciences, Temple University, Philadelphia, USA

    Longin Jan Latecki

Authors
  1. Quan Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jie Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Huimin Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yawen Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Suofei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Xiaofu Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Baoyu Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Weihua Ou
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Longin Jan Latecki
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Quan Zhou or Suofei Zhang.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, Q., Cheng, J., Lu, H. et al. Learning adaptive contrast combinations for visual saliency detection. Multimed Tools Appl 79, 14419–14447 (2020). https://doi.org/10.1007/s11042-018-6770-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-6770-2

Keywords

Search

Navigation