research-article

Inferring word relevance from eye-movements of readers

Authors:

Tomasz D. Loboda,

Peter Brusilovsky,

Jöerg BrunsteinAuthors Info & Claims

IUI '11: Proceedings of the 16th international conference on Intelligent user interfaces

Pages 175 - 184

https://doi.org/10.1145/1943403.1943431

Published: 13 February 2011 Publication History

Abstract

Reading is one of the most important skills in today's society. The ubiquity of this activity has naturally affected many information systems; the only goal of some is the presentation of textual information. One concrete task often performed on a computer and involving reading is finding relevant parts of text. In the current study, we investigated if word-level relevance, defined as a binary measure of an individual word being congruent with the reader's current informational needs, could be inferred given only the text and eye movements of readers. We found that the number of fixations, first-pass fixations, and the total viewing time can be used to predict the relevance of sentence-terminal words. In light of what is known about eye movements of readers, knowing which sentence-terminal words are relevant can help in an unobtrusive identification of relevant sentences.

References

[1]

Ajanki, A., Hardoon, D., Kaski, S., Puolamaki, K., & Taylor, J. (2009) Can eyes reveal interest? Implicit queries from gaze patterns. User Modeling and User-Adapted Interaction, 19(4), 307--339.

Digital Library

[2]

Babaian, T., Grosz, B., & Shieber, S. (2002) A writer's collaborative assistant. Proceedings of International Conference on Intelligent User Interfaces (IUI), 7--14.

Digital Library

[3]

Billsus, D. & Pazzani, M.J. (2000) A learning agent for wireless news access. Proceedings of International Conference on Intelligent User Interfaces (IUI), 94--97.

Digital Library

[4]

Buscher, G., Dengel, A., & van Elst, L. (2008) Query expansion using gaze-based feedback on the subdocument level. Proceedings of the 31st International ACM SIGIR Conference on Research and Development in Information Retrieval, 387--394.

Digital Library

[5]

Claypool, M., Le, P., Wased, M., & Brown, D. (2002) Implicit interest indicators. Proceedings of International Conference on Intelligent User Interfaces (IUI), 33--40.

Digital Library

[6]

Clifton, C., Staub, A., & Rayner, K. (2007) Eye movements in reading words and sentences. In Eye movements: A window on mind and brain (pp. 341--371). New York: Elsevier.

[7]

Fox, S., Karnawat, K., Mydland, M., Dumais, S., & White, T. (2005) Evaluating implicit measures to improve web search. ACM Transactions on Information Systems, 23(2), 147--168.

Digital Library

[8]

Fu, X., Budzik, J., & Hammond, K. J. (2000) Mining navigation history for recommendation. Proceedings of International Conference on Intelligent User Interfaces (IUI), 106--112.

Digital Library

[9]

Goecks, J. & Shavlik, J. (2000) Learning user's interests by unobtrusively observing their normal behavior. Proceedings of International Conference on Intelligent User Interfaces (IUI), 129--132.

Digital Library

[10]

Harris C.M., Hainline L., Abramov I., Lemerise E., & Camenzuli C. (1988) The distribution of fixation durations in infants and naive adults. Vision Research, 28, 419--32.

[11]

Hijikata, Y. (2004) Implicit user profiling for on demand relevance feedback. Proceedings of the International Conference on Intelligent User Interfaces (IUI), 198--205.

Digital Library

[12]

Hyöna, J. & Olson, R.K. (1995) Eye movement patterns among dyslexic and normal readers: Effects of word length and word frequency. Journal of Experimental Psychology: Learning, Memory, & Cognition, 21, 1430--1340.

[13]

Joachims, T., Granka, L., Bing Pan, Hembrooke, H., & Gay, G. (2005) Accurately interpreting clickthrough data as implicit feedback. Proceedings of Annual International ACM SIGIR Conference, 154--161.

Digital Library

[14]

Juhasz, B.J. & Rayner, K. (2003) Investigating the effects of a set of intercorrelated variables on eye fixation durations in reading. Journal of Experimental Psychology: Learning, Memory, & Cognition, 29, 1312--1318.

[15]

Juhasz, B.J. & Rayner, K. (2006) The role of age-of-acquisition and word frequency in reading: Evidence from eye fixation durations. Visual Cognition, 13 (7 & 8), 846--863.

[16]

Just, M.A. & Carpenter, P.A. (1980) A theory of reading: From eye fixations to comprehension. Psychological Review, 87, 329--354.

[17]

Just, M.A., Carpenter, P.A., & Woolley J.D. (1982) Paradigms and processes in reading comprehension. Journal of Experimental Psychology: General, 3(2), 228--238.

[18]

Kaakinen, J.K. & Hyona, J. (2005) Perspective effects on expository text comprehension: Evidence from think-aloud protocols, eyetracking, and recall. Discourse Processes, 40(3), 239--257.

[19]

Kim, J., Oard, D., & Romanik, K. (2000) Using implicit feedback for user modeling in internet and intranet searching, Technical Report, College of Library and Information Services, University of Maryland at College Park.

[20]

Littell, R.C., Milliken, G.A., Stroup, W.W., Wolfinger, R.D., & Schabenberger, O. (2006) SAS for Mixed Models. 2nd Edition. SAS Publishing, Cary.

Digital Library

[21]

Loboda, T.D. (2009) Pegasus {computer software}. Pittsburgh, PA.

[22]

Marlow, C., Naaman, M., Boyd, D., & Davis, M. (2006) HT06, tagging paper, taxonomy, Flickr, academic article, to read. Proceedings of Conference on Hypertext and Hypermedia, 31--40.

Digital Library

[23]

McConkie, G.W. & Rayner, K. (1976) Asymmetry of the perceptual span in reading. Bulletin of the Psychonomic Society, 8, 365--368.

[24]

O'Regan, J.K. (1979) Eye guidance in reading: Evidence for the linguistic control hypothesis. Perception & Psychophysics, 25, 501--509.

[25]

O'Regan, J.K. (1980) The control of saccade size fixation duration in reading: The limits of linguistic control. Perception & Psychophysics, 28, 112--117.

[26]

Puolamaki, K., Ajanki, A., & Kaski, S. (2008) Learning to learn implicit queries from gaze patterns. Proceedings of International Conference on Machine Learning, 760--767.

Digital Library

[27]

Rayner, K. & Raney, G.E. (1996) Eye movement control in reading and visual search: Effects of word frequency. Psychonomic Bulletin & Review, 3(2), 245--248.

[28]

Rayner, K. (1998) Eye Movement in Reading and Information Processing: 20 Years of Research. Psychological Bulletin, 124(3), 372--422.

[29]

Rayner, K., Kambe, G., & Duffy, S.A. (2000). The effect of clause wrap-up on eye movements during reading. The Quarterly Journal of Experimental Psychology, 53A(4), 1061--1080.

[30]

Reichle, E.D., Pollatsek, A., Fisher, D.L., & Rayner, K. (1998) Toward a model of eye movement control in reading. Psychological Review, 105, 125--157.

[31]

Reichle, E.D., Rayner, K., & Pollatsek, A. (2003) The E-Z Reader model of eye-movement control in reading: Comparisons to other models. Behavioral and Brain Sciences, 26, 445--476.

[32]

Salojarvi, J., Kojo, I., Simola, J., & Kaski, S. (2003) Can relevance be inferred from eye movements in information retrieval? Proceedings of Workshop on Self-organizing Maps.

[33]

Salojarvi, J., Puolamaki, K., and Kaski, S. (2005) Implicit relevance feedback from eye movements. Proceedings of International Conference on Artificial Neural Networks (ICANN), 513--518.

Digital Library

[34]

SAS Institute Inc. (2008) SAS 9.2 help and documentation.

[35]

Seo, Y.-W. & Zhang, B.-T. (2000) A reinforcement learning agent for personalized information filtering. Proceedings of International Conference on Intelligent User Interfaces (IUI), 248--251.

Digital Library

[36]

Sugiyama, K., Hatano, K., & Yoshikawa, M. (2004) Adaptive Web search based on user profile constructed without any effort from users. Proceedings of International World Wide Web Conference (WWW), 675--684.

Digital Library

[37]

Swidale, M., Gruzd, A., & Nichols, D. (2008) Writing in the library: Exploring tighter integration of digital library use with the writing process. Information Processing and Management, 44(2), 558--580.

Digital Library

[38]

Underwood, G. (2005) Cognitive Processess in Eye Guidance. Oxford University Press.

[39]

Williams, R.S. & Morris, R.K. (2004) Eye movements, word familiarity, and vocabulary acquisition. European Journal of Cognitive Psychology, 16, 312--339.

[40]

Warren, T., White, S.J., & Reichle, E.D. (2009) Investigating the causes of wrap-up effects: Evidence from eye movements and E-Z Reader. Cognition, 111, 132--137.

[41]

Zhao, S., Du, N., Nauerz, A., Zhang, X., Yuan, Q., & Fu, R. (2008) Improved recommendation based on collaborative tagging behaviors. Proceedings of International Conference on Intelligent User Interfaces (IUI), 413--416.

Digital Library

Cited By

Tahmid ILisle LDavidson KWhitley KNorth CBowman D(2023)Evaluating the Feasibility of Predicting Information Relevance During Sensemaking with Eye Gaze Data2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR59233.2023.00086(713-722)Online publication date: 16-Oct-2023
https://doi.org/10.1109/ISMAR59233.2023.00086
Taieb-Maimon MRomanovski-Chernik ALast MLitvak MElhadad M(2023)Mining Eye-Tracking Data for Text SummarizationInternational Journal of Human–Computer Interaction10.1080/10447318.2023.222782740:17(4887-4905)Online publication date: 21-Jul-2023
https://doi.org/10.1080/10447318.2023.2227827
Southwell RMills CCaruso MD’Mello S(2022)Gaze-based predictive models of deep reading comprehensionUser Modeling and User-Adapted Interaction10.1007/s11257-022-09346-733:3(687-725)Online publication date: 17-Nov-2022
https://doi.org/10.1007/s11257-022-09346-7
Show More Cited By

Index Terms

Inferring word relevance from eye-movements of readers
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods
    2. Interaction devices
      1. Touch screens

Recommendations

Eye movements as implicit relevance feedback
CHI EA '08: CHI '08 Extended Abstracts on Human Factors in Computing Systems

Reading detection is an important step in the process of automatic relevance feedback generation based on eye movements for information retrieval tasks. We describe a reading detection algorithm and present a preliminary study to find expressive eye ...
User Eye Fatigue Detection via Eye Movement Behavior
CHI EA '15: Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems

In this study we propose and evaluate a novel approach that allows detection of physical eye fatigue. The proposed approach is based on the analysis of the recorded eye movements via what is called behavioral scores. These easy-to-compute scores can be ...
Analysing EOG signal features for the discrimination of eye movements with wearable devices
PETMEI '11: Proceedings of the 1st international workshop on pervasive eye tracking & mobile eye-based interaction

Eye tracking research in human-computer interaction and experimental psychology traditionally focuses on stationary devices and a small number of common eye movements. The advent of pervasive eye tracking promises new applications, such as eye-based ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

IUI '11: Proceedings of the 16th international conference on Intelligent user interfaces

February 2011

504 pages

ISBN:9781450304191

DOI:10.1145/1943403

General Chairs:
Pearl Pu
EPFL, Switzerland
,
Michael Pazzani
Rutgers University, USA
,
Program Chairs:
Elisabeth André
Augsburg University, Germany
,
Doug Riecken
CCLS Columbia University, USA

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 February 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

IUI '11

Sponsor:

IUI '11: 16th International Conference on Intelligent User Interfaces

February 13 - 16, 2011

CA, Palo Alto, USA

Acceptance Rates

Overall Acceptance Rate 746 of 2,811 submissions, 27%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

27
Total Citations
View Citations
421
Total Downloads

Downloads (Last 12 months)11
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Tahmid ILisle LDavidson KWhitley KNorth CBowman D(2023)Evaluating the Feasibility of Predicting Information Relevance During Sensemaking with Eye Gaze Data2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR59233.2023.00086(713-722)Online publication date: 16-Oct-2023
https://doi.org/10.1109/ISMAR59233.2023.00086
Taieb-Maimon MRomanovski-Chernik ALast MLitvak MElhadad M(2023)Mining Eye-Tracking Data for Text SummarizationInternational Journal of Human–Computer Interaction10.1080/10447318.2023.222782740:17(4887-4905)Online publication date: 21-Jul-2023
https://doi.org/10.1080/10447318.2023.2227827
Southwell RMills CCaruso MD’Mello S(2022)Gaze-based predictive models of deep reading comprehensionUser Modeling and User-Adapted Interaction10.1007/s11257-022-09346-733:3(687-725)Online publication date: 17-Nov-2022
https://doi.org/10.1007/s11257-022-09346-7
Bhattacharya NRakshit SGwizdka J(2020)Towards Real-time Webpage Relevance Prediction UsingConvex Hull Based Eye-tracking FeaturesACM Symposium on Eye Tracking Research and Applications10.1145/3379157.3391302(1-10)Online publication date: 2-Jun-2020
https://dl.acm.org/doi/10.1145/3379157.3391302
Bhattacharya NRakshit SGwizdka JKogut PO'Brien HFreund LArapakis IHoeber OLopatovska I(2020)Relevance Prediction from Eye-movements Using Semi-interpretable Convolutional Neural NetworksProceedings of the 2020 Conference on Human Information Interaction and Retrieval10.1145/3343413.3377960(223-233)Online publication date: 14-Mar-2020
https://dl.acm.org/doi/10.1145/3343413.3377960
Toker DMoro RSimko JBielikova MConati CPapadopoulos GSamaras GWeibelzahl SJannach DSantos O(2019)Impact of English Reading Comprehension Abilities on Processing Magazine Style Narrative Visualizations and Implications for PersonalizationProceedings of the 27th ACM Conference on User Modeling, Adaptation and Personalization10.1145/3320435.3320447(309-317)Online publication date: 7-Jun-2019
https://dl.acm.org/doi/10.1145/3320435.3320447
Toker DConati CCarenini G(2019)Gaze analysis of user characteristics in magazine style narrative visualizationsUser Modeling and User-Adapted Interaction10.1007/s11257-019-09244-529:5(977-1011)Online publication date: 30-Aug-2019
https://doi.org/10.1007/s11257-019-09244-5
Li XLiu YMao JHe ZZhang MMa SCuzzocrea AAllan JPaton NSrivastava DAgrawal RBroder AZaki MCandan SLabrinidis ASchuster AWang H(2018)Understanding Reading Attention Distribution during Relevance JudgementProceedings of the 27th ACM International Conference on Information and Knowledge Management10.1145/3269206.3271764(733-742)Online publication date: 17-Oct-2018
https://dl.acm.org/doi/10.1145/3269206.3271764
Jayarathna SShanmugathasan SChen JGonçalves MAllen JFox EKan MPetras V(2018)Evaluating Saccade-Bounded Eye Movement Features for the User ModelingProceedings of the 18th ACM/IEEE on Joint Conference on Digital Libraries10.1145/3197026.3197072(21-24)Online publication date: 23-May-2018
https://dl.acm.org/doi/10.1145/3197026.3197072
Toker DConati CCarenini GBerkovsky SHijikata YRekimoto JBurnett MBillinghurst MQuigley A(2018)User-adaptive Support for Processing Magazine Style Narrative Visualizations23rd International Conference on Intelligent User Interfaces10.1145/3172944.3173009(199-204)Online publication date: 5-Mar-2018
https://dl.acm.org/doi/10.1145/3172944.3173009
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten