research-article

Exploring the influence of audio in directing visual attention during dynamic content

Authors:

Brooke E. Wooley,

David S. MarchAuthors Info & Claims

ETRA '14: Proceedings of the Symposium on Eye Tracking Research and Applications

Pages 187 - 190

https://doi.org/10.1145/2578153.2578180

Published: 26 March 2014 Publication History

Abstract

The mechanisms underlying the allocation of visual attention toward dynamic content are still largely unexplored. Due to the number of variables present during dynamic content, it is often difficult to confidently determine what components direct visual attention. In this study, we manipulated the presence of audio in an attempt to explore the contribution of audio in driving visual attention during dynamic content. Participants viewed a reel of non-global commercials while their eye movements were recorded. Participants were either exposed to content containing the original audio track or content in which the audio track was edited out. Dynamic heat maps were created for each ad in order to identify areas of high visual attention between the conditions. Fixation durations and fixation counts for each area of interest were then computed. Analyses showed that the presence of audio has an influence on the allocation of visual attention during dynamic content, most notably in regard to on-screen text. Understanding the influence of audio in directing visual attention may help future researchers control for the extraneous influence of audio in eye-tracking methodologies.

References

[1]

Buswell, G. T. (1935). How people look at pictures: a study of the psychology and perception in art.

[2]

Coutrot, A., Guyader, N., Ionescu, G., & Caplier, A. (2012). Influence of soundtrack on eye movements during video exploration. Journal of Eye Movement Research, 5(4).

[3]

Egeth, H. E., & Yantis, S. (1997). Visual attention: Control, representation, and time course. Annual review of psychology, 48(1), 269--297.

[4]

Frens, M. A., Van Opstal, A. J. & Van Der Willigen, R. F. (1995). Spatial and temporal factors determine auditory-visual interactions in human saccadic eye movements. Perception & Psychophysics, 57 (6), 802--816.

[5]

Goldberg, J. H., & Kotval, X. P. (1999). Computer interface evaluation using eye movements: methods and constructs. International Journal of Industrial Ergonomics, 24(6), 631--645.

[6]

Goldstein, R. B., Woods, R. L., Peli, E. (2006). Where people look when watching movies: Do all viewers look at the same place? Computers in Biology and Medicine, 37 (7), 957--964.

Digital Library

[7]

Hillstrom, A. P., & Yantis, S. (1994). Visual motion and attentional capture. Perception & Psychophysics, 55(4), 399--411.

[8]

Itti, L., Koch, C., & Niebur, E. (1998). A model of saliency-based visual attention for rapid scene analysis. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 20(11), 1254--1259.

Digital Library

[9]

Jacob, R. J., & Karn, K. S. (2003). Eye-tracking in human-computer interaction and usability research: Ready to deliver the promises. Mind, 2(3), 4.

[10]

Krejtz, I., Szarkowska, A., Krejtz, K., Walczak, A., & Duchowski, A. (2012, March). Audio description as an aural guide of children's visual attention: evidence from an eye-tracking study. In Proceedings of the Symposium on Eye-tracking Research and Applications (pp. 99--106). ACM.

Digital Library

[11]

Loftus, G. R., & Mackworth, N. H. (1978). Cognitive determinants of fixation location during picture viewing. Journal of Experimental Psychology: Human Perception and Performance, 4(4), 565.

[12]

Noton, D., & Stark, L. (1971). Scanpaths in saccadic eye movements while viewing and recognizing patterns. Vision research, 11(9), 929-IN8.

[13]

Pieters, R., Rosbergen, E., & Wedel, M. (1999). Visual attention to repeated print advertising: A test of scanpath theory. Journal of Marketing Research, 424--438.

[14]

Pieters, R., & Wedel, M. (2007). Goal control of attention to advertising: The Yarbus implication. Journal of Consumer Research, 34(2), 224--233.

[15]

Tatler, B. W. (2007). The central fixation bias in scene viewing: Selecting an optimal viewing position independently of motor biases and image feature distributions. Journal of Vision, 7(14).

[16]

Vilaró, A., Duchowski, A. T., Orero, P., Grindinger, T., Tetreault, S., & di Giovanni, E. (2012). How sound is the Pear Tree Story? Testing the effect of varying audio stimuli on visual attention distribution. Perspectives, 20(1), 55--65.

[17]

Wedel, M., & Pieters, R. (2006). Eye tracking for visual marketing, Foundations and Trends in Marketing, Vol. 1. Issue, 4, 231--320.

[18]

Wiebe, E., & Annetta, L. (2008). Influences on visual attentional distribution in multimedia instruction. Journal of Educational Multimedia and Hypermedia, 17(2), 259--277.

Cited By

Ji XZhao JRau P(2022)Designing a Voice Intelligent Agent: A Review and AppraisalHuman-Automation Interaction10.1007/978-3-031-10788-7_8(143-162)Online publication date: 15-Dec-2022
https://doi.org/10.1007/978-3-031-10788-7_8
Wooley BMarch D(2016)Unique Object Characteristics Differentially Affect Visual Attention During Viewing of Dynamic Stimuli: The Influence of Location and LuminosityUniversal Access in Human-Computer Interaction. Methods, Techniques, and Best Practices10.1007/978-3-319-40250-5_46(481-492)Online publication date: 21-Jun-2016
https://doi.org/10.1007/978-3-319-40250-5_46

Index Terms

Exploring the influence of audio in directing visual attention during dynamic content
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Gaze is more than just a point: Rethinking visual attention analysis using peripheral vision-based gaze mapping
ETRA '23: Proceedings of the 2023 Symposium on Eye Tracking Research and Applications

In mobile eye-tracking, visual attention is commonly evaluated using fixation-based measures, which can be mapped to predefined objects of interest for task-specific attention analysis. Even though attention can be directed independently from the fovea, ...
Audio-visual attention: Eye-tracking dataset and analysis toolbox
2017 IEEE International Conference on Image Processing (ICIP)
Although many visual attention models have been proposed, very few saliency models investigated the impact of audio information. To develop audio-visual attention models, researchers need to have a ground truth of eye movements recorded while exploring ...
Audio-Visual Salieny Network with Audio Attention Module
ICAIIS 2021: 2021 2nd International Conference on Artificial Intelligence and Information Systems

Recently, inspired by the fact that people rely on visual and audio modalities to observe the world, the audio-visual multi-modal method for computer vision tasks in deep learning networks has attracted more and more attention, and a lot of work has been ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ETRA '14: Proceedings of the Symposium on Eye Tracking Research and Applications

March 2014

394 pages

ISBN:9781450327510

DOI:10.1145/2578153

Conference Chairs:
Pernilla Qvarfordt
FX Palo Alto Laboratory
,
Dan Witzner Hansen
IT University, Denmark

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 March 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ETRA '14

Sponsor:

ETRA '14: Eye Tracking Research and Applications

March 26 - 28, 2014

Florida, Safety Harbor

Acceptance Rates

Overall Acceptance Rate 69 of 137 submissions, 50%

Upcoming Conference

ETRA '25

Sponsor:
sigchi
sigchi

2025 Symposium on Eye Tracking Research and Applications

May 26 - 29, 2025

Tokyo , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
245
Total Downloads

Downloads (Last 12 months)5
Downloads (Last 6 weeks)1

Reflects downloads up to 01 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Ji XZhao JRau P(2022)Designing a Voice Intelligent Agent: A Review and AppraisalHuman-Automation Interaction10.1007/978-3-031-10788-7_8(143-162)Online publication date: 15-Dec-2022
https://doi.org/10.1007/978-3-031-10788-7_8
Wooley BMarch D(2016)Unique Object Characteristics Differentially Affect Visual Attention During Viewing of Dynamic Stimuli: The Influence of Location and LuminosityUniversal Access in Human-Computer Interaction. Methods, Techniques, and Best Practices10.1007/978-3-319-40250-5_46(481-492)Online publication date: 21-Jun-2016
https://doi.org/10.1007/978-3-319-40250-5_46

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten