Skip to main content
SpringerLink
Log in

See What I’m Saying? Expertise and Verbalisation in Perception and Imagery of Complex Scenes

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

How does describing a previously viewed picture affect our memory for it? Does verbalisation affect our eye movements even when the picture has disappeared? When viewing a photograph, the sequences of eye movements we make (‘scanpaths’) are influenced by both bottom-up visual saliency and top-down cognitive knowledge. Recognition memory is enhanced and the similarity of scanpaths at encoding and recognition is greater for domain-specific pictures. A similarity in scanpaths is also observed during imagery but to a greatly reduced degree. This study explored whether scanpath similarity could be improved by verbalising one’s memory of the picture and whether the previously observed domain-specific advantage was still present when no bottom-up information was available. Specialists and controls were shown a set of photographs, and after each one had to either visualise it or describe it from memory. The stimuli were complex scenes, half of which contained a domain-specific object. Recognition accuracy was increased by post-stimulus verbalisation, and specialists demonstrated an advantage for stimuli that contained domain-relevant information. Saliency influenced both verbal feedback and eye movements but was moderated by domain expertise. Scanpaths were more similar when pictures were described compared to when imagined, and specialists produced more similar scanpaths when describing domain-specific pictures, compared to control pictures and control participants.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Foulsham T, Underwood G. What can saliency models predict about eye-movements? Spatial and sequential aspects of fixations during encoding and recognition. J Vis. 2008;8(2):6, 1–17.

    Google Scholar 

  2. Humphrey K, Underwood G. Domain knowledge moderates the influence of visual saliency in scene recognition. Br J Psychol. 2009;100(2):377–98.

    Article  PubMed  Google Scholar 

  3. Stark L, Ellis SR. Scanpaths revisited: cognitive models direct active looking. In: Fisher DF, Monty RA, Senders JW, editors. Eye movements: cognition and visual perception. Hillsdale, NJ: Lawrence Erlbaum; 1981. p. 193–227.

    Google Scholar 

  4. Walker-Smith GJ, Gale AG, Findlay JM. Eye movement strategies involved in face perception. Perception. 1977;6(3):313–26.

    Article  PubMed  CAS  Google Scholar 

  5. Underwood G, Foulsham T, Humphrey K. Saliency and scan patterns in the inspection of real-world scenes: eye movements during encoding and recognition. Vis Cogn. 2009;17(6):812–34.

    Article  Google Scholar 

  6. Underwood G, Humphrey K, Foulsham T. Knowledge-based patterns of remembering: eye movement scanpaths reflect domain experience. Lect Notes Comput Sci. 2008;5298:125–44.

    Article  Google Scholar 

  7. Harding G, Bloj M. Real and predicted influence of image manipulations on eye movements during scene recognition. J Vis. 2010;10(2):8, 1–17.

    Google Scholar 

  8. Itti L, Koch C. A saliency-based search mechanism for overt and covert shifts of visual attention. Vis Res. 2000;40(10–12):1489–506.

    Article  PubMed  CAS  Google Scholar 

  9. Koch C, Ullman S. Shifts in selective visual attention: towards the underlying neural circuitry. Hum Neurobiol. 1985;4:219–27.

    PubMed  CAS  Google Scholar 

  10. Carmi R, Itti L. Visual causes versus correlates of attentional selection in dynamic scenes. Vis Res. 2006;46(26):4333–45.

    Article  PubMed  Google Scholar 

  11. Parkhurst D, Law K, Niebur E. Modelling the role of salience in the allocation of overt visual attention. Vis Res. 2002;42:107–23.

    Article  PubMed  Google Scholar 

  12. Sheth BR, Shimojo S. Compression of space in visual memory. Vis Res. 2001;41:328–41.

    Article  Google Scholar 

  13. Tatler BW. The central fixation bias in scene viewing: selecting an optimal viewing position independently of motor biases and image feature distributions. J Vis. 2007;7(14):1–17.

    Article  PubMed  Google Scholar 

  14. Tatler BW, Baddeley RJ, Gilchrist ID. Visual correlates of fixation selection: effects of scale and time. Vis Res. 2005;45(5):643–59.

    Article  PubMed  Google Scholar 

  15. Underwood G, Foulsham T. Visual saliency and semantic incongruency influence eye movements when inspecting pictures. Q J Exp Psychol. 2006;59(11):1931–49.

    Article  Google Scholar 

  16. Underwood G, Foulsham T, van Loon E, Humphreys L, Bloyce J. Eye movements during scene inspection: a test of the saliency map hypothesis. Eur J Cogn Psychol. 2006;18(3):321–42.

    Article  Google Scholar 

  17. Straube S, Fahle M. The electrophysiological correlate of saliency: evidence from a figure-detection task. Brain Res. 2010;1307:89–102.

    Article  PubMed  CAS  Google Scholar 

  18. Straube S, Grimsen C, Fahle M. Electrophysiological correlates of figure-ground segregation directly reflect perceptual saliency. Vis Res. 2010;50(5):509–21.

    Article  PubMed  Google Scholar 

  19. Brandt SA, Stark LW. Spontaneous eye movements during visual imagery reflect the content of the visual scene. J Cogn Neurosci. 1997;9:27–38.

    Article  Google Scholar 

  20. Holsanova J, Hedberg B, Nilsson N. Visual and verbal focus patterns when describing pictures. In: Becker W, Deubel H, Mergner T, editors. Current oculomotor research: physiological and psychological aspects. New York: Plenum; 1998. p. 303–4.

    Google Scholar 

  21. Humphrey K, Underwood G. Fixation sequences in imagery and in recognition during the processing of pictures of real-world scenes. J Eye Mov Res. 2008;2(2:3):1–15.

    Google Scholar 

  22. Pieters R, Rosbergen E, Wedel M. Visual attention to repeated print advertising: a test of scanpath theory. J Mark Res. 1999;36:424–38.

    Article  Google Scholar 

  23. Underwood G. Cognitive processes in eye guidance: algorithms for attention in image processing. Cognit Comput. 2009;1(1):64–76.

    Article  Google Scholar 

  24. Underwood G, Templeman E, Lamming L, Foulsham T. Is attention necessary for object identification? Evidence from eye movements during the inspection of real-world scenes. Conscious Cogn. 2008;17:159–70.

    Article  PubMed  Google Scholar 

  25. Einhauser W, Rutishauser U, Koch C. Task-demands can immediately reverse the effects of sensory-driven saliency in complex visual stimuli. J Vis. 2008;8:1–19.

    Google Scholar 

  26. Hegarty M, Canham M, Fabrikant SI. Thinking about the weather: how display salience and knowledge affect performance in a graphic inference task. J Exp Psychol Learn Mem Cogn. 2010;36(1):37–53.

    Article  PubMed  Google Scholar 

  27. Egan DE, Schwartz EJ. Chunking in recall of symbolic drawings. Mem Cogn. 1979;7:149–58.

    Article  CAS  Google Scholar 

  28. Gobet F, Lane PCR, Croker S, Cheng PCH, Jones G, Oliver I, Pine JM. Chunking mechanisms in human learning. Trends Cogn Sci. 2001;5:236–43.

    Article  PubMed  Google Scholar 

  29. Saariluoma P, Kalakoski V. Skilled imagery and long-term working memory. Am J Psychol. 1997;110(2):177–201.

    Article  Google Scholar 

  30. Hishitani S. Imagery experts: how do expert abacus operators process imagery? Appl Cogn Psychol. 1988;4(1):33–46.

    Article  Google Scholar 

  31. Calabrese L, Marucci FS. The influence of expertise level on the visuo-spatial ability: differences between experts and novices in imagery and drawing abilities. Cogn Process. 2006;7(1):118–20.

    Article  Google Scholar 

  32. Sims VK, Mayer RE. Domain specificity of spatial expertise: the case of video game players. Appl Cogn Psychol. 2002;16:97–115.

    Article  Google Scholar 

  33. Schooler JW, Ohlson S, Brooks K. Thoughts beyond words: when language overshadows insight. J Exp Psychol Gen. 1993;122:166–83.

    Article  Google Scholar 

  34. Fiore SM, Schooler JW. How did you get here from there? Verbal overshadowing of spatial mental models. Appl Cogn Psychol. 2002;16:897–909.

    Article  Google Scholar 

  35. Diamond R, Carey S. Why faces are and are not special: an effect of expertise. J Exp Psychol Gen. 1986;115:107–17.

    Article  PubMed  CAS  Google Scholar 

  36. Fallshore M, Schooler JW. Verbal vulnerability of perceptual expertise. J Exp Psychol Learn Mem Cogn. 1995;21(6):1608–23.

    Article  PubMed  CAS  Google Scholar 

  37. Ryan RS, Schooler JW. Whom do words hurt? Individual differences in susceptibility to verbal overshadowing. Appl Cogn Psychol. 1998;12:105–25.

    Article  Google Scholar 

  38. Melcher JM, Schooler JW. The mis-rememberance of wines past: verbal and perceptual expertise differentially mediate verbal overshadowing of taste memory. J Memory Lang. 1996;35:231–45.

    Article  Google Scholar 

  39. Chance J, Goldstein AG. Recognition of faces and verbal labels. Bull Psychon Soc. 1976;7:384–6.

    Google Scholar 

  40. Kitagami S, Sato W, Yoshikawa S. The influence of test-set similarity in verbal overshadowing. Appl Cogn Psychol. 2002;16:963–72.

    Article  Google Scholar 

  41. McKelvie SJ. The effects of verbal labelling on recognition memory for schematic faces. Q J Exp Psychol. 1976;28:459–74.

    Article  Google Scholar 

  42. Meissner CA, Brigham JC, Kelley CM. The influence of retrieval processes in verbal overshadowing. Mem Cogn. 2001;29:176–86.

    Article  CAS  Google Scholar 

  43. Read JD. Rehearsal and recognition of human faces. Am J Psychol. 1979;92:71–85.

    Article  Google Scholar 

  44. Yu CJ, Geiselman RE. Effects of constructing identi-kit composites on photospread identification performance. Crim Justice Behav. 1993;20:280–92.

    Article  Google Scholar 

  45. Bartlett JC, Till RE, Levy JC. Retrieval characteristics of complex pictures: effect of verbal encoding. J Verbal Learn Verbal Behav. 1980;19:430–9.

    Article  Google Scholar 

  46. Meissner CA, Brigham JC. A meta-analysis of the verbal overshadowing effect in face identification. Appl Cogn Psychol. 2001;15(6):603–16.

    Article  Google Scholar 

  47. Itoh Y. The facilitating effect of verbalization on the recognition memory of incidentally learned faces. Appl Cogn Psychol. 2005;19:421–33.

    Article  Google Scholar 

  48. Huff M, Schwan S. Verbalizing events: overshadowing or facilitation? Mem Cogn. 2008;36(2):392–402.

    Article  Google Scholar 

  49. Choi YS, Mosley AD, Stark L. Sting editing analysis of human visual search. Optom Vis Sci. 1995;72:439–51.

    Article  PubMed  CAS  Google Scholar 

  50. Hacisalihzade SS, Allen JS, Stark L. Visual perception and sequences of eye movement fixations: a stochastic modelling approach. IEEE Trans Syst Man Cybern. 1992;22:474–81.

    Article  Google Scholar 

  51. Privitera CM, Stark LW, Zangemeister WH. Bonnard’s representation of the perception of substance. J Eye Mov Res. 2007;1(1):3, 1–6.

    Google Scholar 

  52. Laeng B, Teodorescu DS. Eye scanpaths during visual imagery re-enact those of perception of the same visual scene. Cogn Sci. 2002;26:207–31.

    Article  Google Scholar 

  53. Myles-Worsley M, Johnston WA, Simons MA. The influence of expertise on X-ray image processing. J Exp Psychol Learn Mem Cogn. 1988;14:553–7.

    Article  PubMed  CAS  Google Scholar 

  54. Dodson CS, Johnson MK, Schooler JW. The verbal overshadowing effect: why descriptions impair face recognition. Mem Cogn. 1997;25:129–39.

    Article  CAS  Google Scholar 

  55. Westerman DL, Larsen JD. Verbal-overshadowing effect: evidence for a general shift in processing. Am J Psychol. 1997;110:117–28.

    Article  Google Scholar 

  56. Cutsuridis V. A cognitive model of saliency, attention, and picture scanning. Cogn Comput. 2009;1:292–9.

    Article  Google Scholar 

  57. Lowe RK. Extracting information from an animation during complex visual learning. Eur J Psychol Educ. 1999;14(2):225–44.

    Article  Google Scholar 

  58. Findlay JM, Walker R. A model of saccade generation based on parallel processing and competitive inhibition. Behav Brain Sci. 1999;22:661–74.

    PubMed  CAS  Google Scholar 

  59. Evans KK, Treisman A. Perception of objects in natural scenes: is it really attention free? J Exp Psychol Hum Percept Perform. 2005;31:1476–92.

    Article  PubMed  Google Scholar 

  60. Rayner K, Castelhano MS, Yang J. Eye movements when looking at unusual/weird scenes: are there cultural differences? J Exp Psychol Learn Mem Cogn. 2009;35:254–9.

    Article  PubMed  Google Scholar 

  61. Brown ID. A review of the ‘looked but failed to see’ accident causation factor. Behav Res Road Saf. 2002;11:116–24.

    Google Scholar 

  62. DeAngelus M, Pelz JB. Top-down control of eye movements: Yarbus revisited. Vis Cogn. 2009;99999(1):1–22.

    Google Scholar 

  63. Hayhoe MM, Shrivastava A, Mruczek R, Pelz JB. Visual memory and motor planning in a natural task. J Vis. 2003;3:49–63.

    Article  PubMed  Google Scholar 

  64. Yarbus A. Eye movements and vision. New York: Plenum Press; 1967.

  65. Faloutsos C, Flickner M, Niblack W, Petkovic D, Equitz W, Barber R. Efficient and effective querying by image content. Tech report, IBM research report; 1993.

  66. Flickner M, Sawhney H, Niblack W, Ashley J, Huang Q, Dom B, et al. Query by image and video content: the QBIC system. IEEE Comput. 1995; 23–32.

  67. Niblack W, Barber R. The QBIC project: querying images by content using colour, texture and shape. Proceedings of the SPIE Storage and Retrieval for Image and Video Databases; 1994.

  68. Amadasun M, King RA. Low-level segmentation of multispectral images via agglomerative clustering of uniform neighborhoods. Pattern Recogn. 1998;21(3):261–8.

    Article  Google Scholar 

  69. Howarth P, Ruger S. Evaluation of texture features for content-based image retrieval. LNCS. 2004;3115:326–34.

    Google Scholar 

Download references

Acknowledgments

We are grateful to the UK Engineering and Physical Sciences Research Council (EPSRC) for support (award EP/E006329/1) and to Laurent Itti for the use of his saliency software. Thanks also go to Tom Foulsham for use of his string-editing scanpath computer program.

Author information

Authors and Affiliations

  1. Division of Psychology, Nottingham Trent University, Chaucer building, Nottingham, NG1 4BU, UK

    Katherine Humphrey

  2. School of Psychology, University of Nottingham, Nottingham, NG7 2RD, UK

    Geoffrey Underwood

Authors
  1. Katherine Humphrey
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Geoffrey Underwood
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Katherine Humphrey.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Humphrey, K., Underwood, G. See What I’m Saying? Expertise and Verbalisation in Perception and Imagery of Complex Scenes. Cogn Comput 3, 64–78 (2011). https://doi.org/10.1007/s12559-010-9065-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-010-9065-0

Keywords

Search

Navigation