Skip to main content
SpringerLink
Log in

Cognitive Processes in Eye Guidance: Algorithms for Attention in Image Processing

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

When inspecting an image for the first time, how does the viewer decide where to look next? The saliency map hypothesis proposes that viewers initially analyse the image for variations in low-level visual features including intensity, colour, and edge orientation, and that their eyes are guided towards the most salient region. The saliency of objects in scenes may provide an explanation of why some experiments find that incongruent objects attract attention whilst other studies do not find this effect. Experiments that have monitored eye movements during scene inspection have found some support for the saliency map hypothesis, particularly when pictures are inspected in anticipation of a memory test. Under some circumstances the hypothesis fails to account for inspection patterns. When scenes are inspected to check the presence or absence of a named object, or when two images are compared to determine whether they are identical, or when the viewer has specialised domain knowledge of the scene depicted, then saliency has little influence. This paper evaluates the saliency map hypothesis of scene perception using evidence of eye movements made when images are first inspected, and concludes that visual saliency can be used by viewers, but that its use is both task-dependent and knowledge-dependent.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Biederman I, Mezzanotte RJ, Rabinowitz JC. Scene perception: detecting and judging objects undergoing relational violations. Cognit Psychol. 1982;14:143–77.

    Article  PubMed  CAS  Google Scholar 

  2. Murphy GL, Wisniewski EJ. Categorizing objects in isolation and in scenes: what a superordinate is good for. J Exp Psychol Learn Mem Cogn. 1989;15:572–86.

    Article  PubMed  CAS  Google Scholar 

  3. Davenport JL, Potter MC. Scene consistency in object and background perception. Psychol Sci. 2004;15:559–64.

    Article  PubMed  Google Scholar 

  4. Mackworth NH, Morandi AJ. The gaze selects informative details within pictures. Percept Psychophys. 1967;2:547–52.

    Google Scholar 

  5. Loftus GR, Mackworth NH. Cognitive determinants of fixation location during picture viewing. J Exp Psychol Hum Percept Perform. 1978;4:565–72.

    Article  PubMed  CAS  Google Scholar 

  6. Underwood G, Humphreys L, Cross E. Congruency, saliency and gist in the inspection of objects in natural scenes. In: van Gompel RPG, Fischer MH, Murray WS, Hill RL, editors. Eye movements: a window on mind and brain. Oxford: Elsevier; 2007. p. 561–77.

    Google Scholar 

  7. De Graef P, Christiaens D, d’Ydewalle G. Perceptual effects of scene context on object identification. Psychol Res. 1990;52:317–29.

    Article  PubMed  Google Scholar 

  8. Henderson JM, Weeks PA, Hollingworth A. The effects of semantic consistency on eye movements during scene viewing. J Exp Psychol Hum Percept Perform. 1999;25:210–28.

    Article  Google Scholar 

  9. Gareze L, Findlay JM. In: van Gompel RPG, Fischer MH, Murray WS, Hill RL, editors. Eye movements: a window on mind and brain. Oxford: Elsevier; 2007. p. 617–37.

    Google Scholar 

  10. Findlay JM, Walker R. A model of saccade generation base on parallel processing and competitive inhibition. Behav Brain Sci. 1999;4:661–721.

    Google Scholar 

  11. Torralba A, Castelhano MS, Oliva A, Henderson JM. Contextual guidance of eye movements and attention in real-world scenes: the role of global features on object search. Psychol Rev. 2006;113:766–86.

    Article  PubMed  Google Scholar 

  12. Navalpakkam V, Itti L. Modeling the influence of task on attention. Vision Res. 2005;45:205–31.

    Article  PubMed  Google Scholar 

  13. Itti L, Koch C. A saliency-based search mechanism for overt and covert shifts of visual attention. Vision Res. 2000;40:1489–506.

    Article  PubMed  CAS  Google Scholar 

  14. Koch C, Ullman S. Shifts in selective visual attention: towards the underlying neural circuitry. Hum Neurobiol. 1985;4:219–27.

    PubMed  CAS  Google Scholar 

  15. Itti L. Quantitative modelling of perceptual salience at human eye position. Vis Cogn. 2006;14:959–84.

    Article  Google Scholar 

  16. Elazary L, Itti L. Interesting objects are visually salient. J Vis. 2008;8(3):3.1–15.

    Article  Google Scholar 

  17. Russell BC, Torralba A, Murphy KP, Freeman WT. LabelMe: a database and a web-based tool for image annotation. Int J Comput Vis. 2008;77:157–73.

    Article  Google Scholar 

  18. Treisman AM, Gelade G. A feature-integration theory of attention. Cognit Psychol. 1980;12:97–136.

    Article  PubMed  CAS  Google Scholar 

  19. Parkhurst D, Law K, Niebur E. Modelling the role of salience in the allocation of overt visual attention. Vision Res. 2002;42:107–23.

    Article  PubMed  Google Scholar 

  20. Underwood G, Foulsham T, van Loon E, Humphreys L, Bloyce J. Eye movements during scene inspection: a test of the saliency map hypothesis. Eur J Cogn Psychol. 2006;18:321–42.

    Article  Google Scholar 

  21. Underwood G, Foulsham T. Visual saliency and semantic incongruency influence eye movements when inspecting pictures. Q J Exp Psychol. 2006;59:1931–49.

    Article  Google Scholar 

  22. Underwood G, Jebbett L, Roberts K. Inspecting pictures for information to verify a sentence: eye movements in general encoding and in focused search. Q J Exp Psychol. 2004;57A:165–82.

    Google Scholar 

  23. Underwood G, Templeman E, Lamming L, Foulsham T. Is attention necessary for object identification? Evidence from eye movements during the inspection of real-world scenes. Conscious Cogn. 2008;17:159–70.

    Article  PubMed  Google Scholar 

  24. Stirk JA, Underwood G. Low-level visual saliency does not predict change detection in natural scenes. J Vis. 2007;7(10):3.1–10.

    Article  Google Scholar 

  25. Underwood J, Templeman E, Underwood G. Conspicuity and congruity in change detection. Lect Notes Comput Sci. 2008.

  26. Galpin AJ, Underwood G. Eye movements during search and detection in comparative visual search. Percept Psychophys. 2005;67:1313–31.

    PubMed  Google Scholar 

  27. Becker MW, Pashler H, Lubin J. Object-intrinsic oddities draw early saccades. J Exp Psychol Hum Percept Perform. 2007;33:20–30.

    Article  PubMed  Google Scholar 

  28. Tatler BW, Baddeley RJ, Gilchrist ID. Visual correlates of fixation selection: effects of scale and time. Vision Res. 2005;45:643–59.

    Article  PubMed  Google Scholar 

  29. Tatler BW. The central fixation bias in scene viewing: selecting an optimal viewing position independently of motor biases and image feature distributions. J Vis. 2007;7(14):4.1–17.

    Article  Google Scholar 

  30. Foulsham T, Underwood G. What can saliency models predict about eye-movements? Spatial and sequential aspects of fixations during encoding and recognition. J Vis. 2008;8(2):6.1–17.

    Article  Google Scholar 

  31. Humphrey K, Underwood G. Domain knowledge moderates the influence of visual saliency in scene recognition. Br J Psychol. 2009;100 (in press).

Download references

Acknowledgement

The U.K. Engineering and Physical Sciences Research Council (EPSRC) supported this work with project award EP/E006329/1. I am indebted to Laurent Itti for making available his program for determining saliency maps.

Author information

Authors and Affiliations

  1. School of Psychology, University of Nottingham, Nottingham, NG7 2RD, UK

    Geoffrey Underwood

Authors
  1. Geoffrey Underwood
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Geoffrey Underwood.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Underwood, G. Cognitive Processes in Eye Guidance: Algorithms for Attention in Image Processing. Cogn Comput 1, 64–76 (2009). https://doi.org/10.1007/s12559-008-9002-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-008-9002-7

Keywords

Search

Navigation