skip to main content
10.1145/1344471.1344508acmconferencesArticle/Chapter ViewAbstractPublication PagesetraConference Proceedingsconference-collections
poster

Using semantic content as cues for better scanpath prediction

Published: 26 March 2008 Publication History

Abstract

Under natural viewing conditions, human observers use shifts in gaze to allocate processing resources to subsets of the visual input. There are many computational models that try to predict these shifts in eye movement and attention. Although the important role of high level stimulus properties (e.g., semantic information) stands undisputed, most models are based solely on low-level image properties. We here demonstrate that a combined model of high-level object detection and low-level saliency significantly outperforms a low-level saliency model in predicting locations humans fixate on. The data is based on eye-movement recordings of humans observing photographs of natural scenes, which contained one of the following high-level stimuli: faces, text, scrambled text or cell phones. We show that observers - even when not instructed to look for anything particular, fixate on a face with a probability of over 80% within their first two fixations, on text and scrambled text with a probability of over 65.1% and 57.9% respectively, and on cell phones with probability of 8.3%. This suggests that content with meaningful semantic information is significantly more likely to be seen earlier. Adding regions of interest (ROI), which depict the locations of the high-level meaningful features, significantly improves the prediction of a saliency model for stimuli with high semantic importance, while it has little effect for an object with no semantic meaning.

References

[1]
Cerf, M., Harel, J., Einhäuser, W., and Koch, C. 2008. Predicting human gaze using low-level saliency combined with face detection. In Advances in Neural Information Processing Systems 20, J. Platt, D. Koller, Y. Singer, and S. Roweis, Eds. MIT Press, Cambridge, MA.
[2]
Einhäuser, W., Rutishauser, U., Frady, E., Nadler, S., König, P., and Koch, C. 2006. The relation of phase noise and luminance contrast to overt attention in complex visual stimuli. Journal of Vision 6, 11, 1148--1158.
[3]
Hershler, O., and Hochstein, S. 2005. At first sight: a high-level pop out effect for faces. Vision Res 45, 13, 1707--24.
[4]
Itti, L., and Koch, C. 2000. A saliency-based search mechanism for overt and covert shifts of visual attention. Vision Research 40, 10--12, 1489--1506.
[5]
Itti, L., and Koch, C. 2001. Computational modeling of visual attention. Nature Rev. Neurosci. 2, 3, 194--203.
[6]
Itti, L., Koch, C., Niebur, E., et al. 1998. A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 20, 11, 1254--1259.
[7]
James, W. 1950. The Principles of Psychology. Dover Publications.
[8]
Johnson, M., Dziurawiec, S., Ellis, H., and Morton, J. 1991. Newborns' preferential tracking of face-like stimuli and its subsequent decline. Cognition 40, 1--2, 1--19.
[9]
Oliva, A., Torralba, A., Castelhano, M., and Henderson, J. 2003. Top-down control of visual attention in object detection. Image Processing, 2003. Proceedings. 2003 International Conference on I.
[10]
Peters, R., Iyer, A., Itti, L., and Koch, C. 2005. Components of bottom-up gaze allocation in natural images. Vision Research 45, 18, 2397--2416.
[11]
Vanrullen, R. 2006. On second glance: Still no high-level popout effect for faces. Vision Res 46, 18, 3017--3027.
[12]
Viola, P., and Jones, M. 2001. Rapid object detection using a boosted cascade of simple features. Computer Vision and Pattern Recognition 1, 511--518.

Cited By

View all

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ETRA '08: Proceedings of the 2008 symposium on Eye tracking research & applications
March 2008
285 pages
ISBN:9781595939821
DOI:10.1145/1344471
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 March 2008

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. eye tracking
  2. natural scenes
  3. psychophysics

Qualifiers

  • Poster

Conference

ETRA '08
ETRA '08: Eye Tracking Research and Applications
March 26 - 28, 2008
Georgia, Savannah

Acceptance Rates

Overall Acceptance Rate 69 of 137 submissions, 50%

Upcoming Conference

ETRA '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)10
  • Downloads (Last 6 weeks)2
Reflects downloads up to 13 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2023)Out-of-Home und Digital-Out-of-Home aus der Perspektive der kognitiven InformationsverarbeitungOut-of-Home-Kommunikation10.1007/978-3-658-38119-6_4(53-78)Online publication date: 30-Apr-2023
  • (2023)SMST: A Saliency Map to Scanpath TransformerDatabases Theory and Applications10.1007/978-3-031-47843-7_10(136-149)Online publication date: 7-Nov-2023
  • (2020)Gravitational Laws of Focus of AttentionIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2019.292063642:12(2983-2995)Online publication date: 1-Dec-2020
  • (2019)DeepAttent: Saliency Prediction with Deep Multi-scale Residual NetworkProceedings of 3rd International Conference on Computer Vision and Image Processing10.1007/978-981-32-9291-8_6(65-73)Online publication date: 20-Sep-2019
  • (2018)Computational Models of Visual AttentionComputer Vision10.4018/978-1-5225-5204-8.ch001(1-26)Online publication date: 2018
  • (2018)Investigating overt and covert shifts of attention within social naturalistic scenesJournal of Vision10.1167/18.12.1118:12(11)Online publication date: 20-Nov-2018
  • (2017)Top-down influence on gaze patterns in the presence of social featuresPLOS ONE10.1371/journal.pone.018379912:8(e0183799)Online publication date: 24-Aug-2017
  • (2015)First attention then intentionInternational Journal of Advertising10.2501/S026504870808003727:3(381-398)Online publication date: 7-Jan-2015
  • (2014)Computational Models of Visual AttentionResearch Developments in Computer Vision and Image Processing10.4018/978-1-4666-4558-5.ch004(54-76)Online publication date: 2014
  • (2012)Improving Visual Saliency by Adding ‘Face Feature Map’ and ‘Center Bias’Cognitive Computation10.1007/s12559-012-9146-35:1(63-75)Online publication date: 1-Jun-2012
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media