Skip to main content
Springer Nature Link
Log in

Spatiotemporal Saliency: Towards a Hierarchical Representation of Visual Saliency

  • Conference paper
Attention in Cognitive Systems (WAPCV 2008)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 5395))

Included in the following conference series:

Abstract

In prior work, we put forth a model of visual saliency motivated by information theoretic considerations [1]. In this effort we consider how this proposal extends to explain saliency in the spatiotemporal domain and further, propose a distributed representation for visual saliency comprised of localized hierarchical saliency computation. Evidence for the efficacy of the proposal in capturing aspects of human behavior is achieved via comparison with eye tracking data and a discussion of the role of neural coding in the determination of saliency suggests avenues for future research.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Bruce, N.D.B., Tsotsos, J.K.: Saliency Based on Information Maximization. In: Advances in Neural Information Processing Systems, vol. 18, pp. 155–162 (June 2006)

    Google Scholar 

  2. Itti, L., Koch, C., Niebur, E.: A Model of Saliency-Based Visual Attention for Rapid Scene Analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(11), 1254–1259 (1998)

    Article  Google Scholar 

  3. Bruce, N.D.B., Tsotsos, J.K.: An information theoretic model of saliency and visual search. In: Paletta, L., Rome, E. (eds.) WAPCV 2007. LNCS, vol. 4840, pp. 171–183. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  4. Tsotsos, J.K., Culhane, S., Yan Kei Wai, W., Lai, Y., Davis, N., Nuflo, F.: Modeling visual attention via selective tuning. Artificial intelligence 78, 507–545 (1995)

    Article  Google Scholar 

  5. Bell, A.J., Sejnowski, T.J.: The ‘Independent Components’ of Natural Scenes are Edge Filters. Vision Research 37(23), 3327–3338 (1997)

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  6. Olshausen, B.A., Field, D.J.: Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–609 (1996)

    Article  CAS  PubMed  Google Scholar 

  7. Wachtler, T., Lee, T.-W., Sejnowski, T.J.: The chromatic structure of natural scenes. J. Opt. Soc. Amer. A 18(1), 65–77 (2001)

    Article  CAS  Google Scholar 

  8. van Hateren, J.H., van der Schaaf, A.: Independent component filters of natural images compared with simple cells in primary visual cortex. Proc. R. Soc. Lond. B 265, 359–366 (1998)

    Article  Google Scholar 

  9. Lee, T.W., Girolami, M., Sejnowski, T.J.: Independent component analysis using an extended infomax algorithm for mixed subgaussian and supergaussian sources. Neural Computation 11(2), 417–441 (1999)

    Article  CAS  PubMed  Google Scholar 

  10. Itti, L., Baldi, P.: Bayesian Surprise Attracts Human Attention. In: Advances in Neural Information Processing Systems, vol. 19, pp. 547–554 (2006)

    Google Scholar 

  11. Yu, C., Levi, D.M.: Surround modulation in human vision unmasked by masking experiments. Nature 3(7), 724–728 (2000)

    CAS  Google Scholar 

  12. Williams, A.L., Singh, K.D., Smith, A.T.: Surround modulation measured with fMRI in the visual cortex. Journal of Neurophysiology 89(1), 525–533 (2003)

    Article  PubMed  Google Scholar 

  13. Xing, J., Heeger, D.J.: Measurement and Modeling of Centre-Surround Suppression and Enhancement. Vision Research 41, 571–583 (2001)

    Article  CAS  PubMed  Google Scholar 

  14. Shen, Z.M., Xu, W.F., Li, C.Y.: Cue-invariant detection of centre surround discontinuity by V1 neurons in awake macaque monkey. Journal of Physiology 583, 581–592 (2007)

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  15. Yu, C., Klein, A.K., Levi, D.M.: Cross-and Iso-oriented surrounds modulate the contrast response function: The effect of surround contrast. Journal of Vision 3, 527–540 (2003)

    Article  PubMed  Google Scholar 

  16. Petrov, Y., McKee, S.P.: The effect of spatial configuration on surround suppression of contrast sensitivity. Journal of Vision 6(3), 224–238 (2006)

    Article  PubMed  PubMed Central  Google Scholar 

  17. Adini, Y., Sagi, D.: Recurrent networks in human visual cortex: psychophysical evidence. Journal of the Optical Society of America A 18(8), 2228–2236 (2001)

    Article  CAS  Google Scholar 

  18. Olzak, L.A., Laurinen, P.I.: Contextual Effects in fine spatial discriminations. Nature 381(6583), 607–609 (2005)

    Google Scholar 

  19. Cannon, M.W., Fullencamp, S.C.: A model for inhibitory lateral interaction effects in perceived contrast. Vision Research 36(8), 1115–1125 (1996)

    Article  CAS  PubMed  Google Scholar 

  20. Xing, J., Heeger, D.J.: Centre-surround interactions in foveal and peripheral vision. Vision Research 40, 3065–3072 (2000)

    Article  CAS  PubMed  Google Scholar 

  21. Yu, C., Klein, A.K., Levi, D.M.: Surround modulation of perceived contrast and the role of brightness induction. Journal of Vision 1, 18–31 (2001)

    Article  CAS  PubMed  Google Scholar 

  22. Zhang, B., Zheng, J., Watanabe, I., Maruko, I., Bi, H., Smith, E.L., Chino, Y.: Delayed maturation of receptive field centre/surround mechanisms in V2. Proceedings of the National Academy of Sciences 102(16), 5862–5867 (2005)

    Google Scholar 

  23. Solomon, S.G., Pierce, J.W., Lennie, P.: The impact of suppressive surrounds on chromatic properties of cortical neurons. Journal of Neuroscience 24(1), 148–160 (2004)

    Article  CAS  PubMed  Google Scholar 

  24. Schein, S.J., Desimone, R.: Spectral properties of V4 Neurons in the macaque. Journal of Neuroscience 10(10), 3369–3389 (1990)

    CAS  PubMed  Google Scholar 

  25. Kondo, H., Komatsu, H.: Suppression on neuronal responses by a metacontrast masking stimulus. Neuroscience Research 36(1), 27–33 (2000)

    Article  CAS  PubMed  Google Scholar 

  26. Tadin, D., Lappin, J.S.: Optimal Size for perceiving motion decreases with contrast. Vision Research 45, 2059–2064 (2005)

    Article  PubMed  Google Scholar 

  27. Born, R.T., Bradley, D.C.: Structure and Function of Visual Area MT. Annual Review of Neuroscience 28, 157–189 (2005)

    Article  CAS  PubMed  Google Scholar 

  28. Huang, X., Albright, T.D., Stoner, G.R.: Adaptive Surround Modulation in Cortical Area MT. Neuron 53(5), 761–770 (2007)

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  29. Eifuku, S., Wurtz, R.H.: Response to Motion in Extrastriate Area MSTI: Centre-Surround Interactions. Journal of Neurophysiology 80(11), 282–296 (1998)

    CAS  PubMed  Google Scholar 

  30. Foldiak, P., Young, M.: Sparse coding in the primate cortex. In: Arbib, M.A. (ed.) The Handbook of Brain Theory and Neural Networks, pp. 895–898 (1995)

    Google Scholar 

  31. David, S.V., Vinje, W.E., Gallant, J.L.: Natural stimulus statistics alter the receptive field structure of v1 neurons. Journal of Neuroscience 24(31), 6991–7006 (2004)

    Article  CAS  PubMed  Google Scholar 

  32. Simoncelli, E.P., Olshausen, B.A.: Natural image statistics and neural representation. Annual Review Neuroscience 24, 1193–1216 (2001)

    Article  CAS  Google Scholar 

  33. Quian Quiroga, R., Reddy, L., Kreiman, G., Koch, C., Fried, I.: Invariant visual representation by single neurons in the human brain. Proceedings of the National Academy of Science 102(16), 5862–5867 (2005)

    Google Scholar 

  34. Kreiman, G.: Neural coding: computational and biophysical perspectives. Physics of Life Reviews 2, 71–102 (2004)

    Article  Google Scholar 

  35. Sagi, D.: The combination of spatial frequency and orientation is effortlessly perceived. Perception and Psychophysics 43, 601–603 (1988)

    Article  CAS  PubMed  Google Scholar 

  36. Wolfe, J.M., Horowitz, T.S.: What attributes guide the deployment of visual attention and how do they do it? Nature Reviews Neuroscience 5, 1–7 (2004)

    Article  Google Scholar 

  37. Enns, J.T., Rensink, R.A.: Sensitivity to three-dimensional orientation in visual search. Psychological Science 1, 323–326 (1990)

    Article  Google Scholar 

  38. Ramachandran, V.S.: Perception of Shape from Shading. Nature, 163–166 (1988)

    Google Scholar 

  39. Hershler, O., Hochstein, S.: At first sight: a high-level pop out effect for faces. Vision Research 45(13), 1707–1724 (2005)

    Article  PubMed  Google Scholar 

  40. Sergent, J., Ohta, S., MacDonald, B.: Functional neuroanatomy of face and object processing. A positron emission tomography study. Brain 115(1), 15–36 (1992)

    PubMed  Google Scholar 

  41. Kanwisher, N., McDermott, J., Chun, M.M.: The fusiform face area: a module in human extrastriate cortex specialized for face perception. Journal of Neuroscience 17(11), 4302–4311 (2006)

    Google Scholar 

  42. Grill-Spector, K., Sayres, R., Ress, D.: High-resolution imaging reveals highly selective nonface clusters in the fusiform face area. Nature Neuroscience 9(9), 1177–1185 (2006)

    Article  CAS  PubMed  Google Scholar 

  43. Wang, Q., Cavanagh, P., Green, M.: Familiarity and pop-out in visual search. Perception and Psychophysics 56(5), 495–500 (1994)

    Article  CAS  PubMed  Google Scholar 

  44. Shen, J., Reingold, E.M.: Visual search asymmetry: the influence of stimulus familiarity and low-level features. Perception and Psychophysics 63(3), 464–475 (2001)

    Article  CAS  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering and Centre for Vision Research, York University, Toronto, ON, Canada

    Neil D. B. Bruce & John K. Tsotsos

Authors
  1. Neil D. B. Bruce
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. John K. Tsotsos
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Joanneum Research, Graz, Austria

    Lucas Paletta

  2. Center for Vision Research (CVR) and, Department of Computer Science and Engineering, York University, 4700 Keele St., ON M3J 1P3, Toronto, Canada

    John K. Tsotsos

Rights and permissions

Reprints and permissions

Copyright information

© 2009 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Bruce, N.D.B., Tsotsos, J.K. (2009). Spatiotemporal Saliency: Towards a Hierarchical Representation of Visual Saliency. In: Paletta, L., Tsotsos, J.K. (eds) Attention in Cognitive Systems. WAPCV 2008. Lecture Notes in Computer Science(), vol 5395. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-00582-4_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-00582-4_8

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-00581-7

  • Online ISBN: 978-3-642-00582-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics