research-article

Attentive documents: Eye tracking as implicit feedback for information retrieval and beyond

Authors:

Andreas Dengel,

Ludger V. ElstAuthors Info & Claims

ACM Transactions on Interactive Intelligent Systems (TiiS), Volume 1, Issue 2

Article No.: 9, Pages 1 - 30

https://doi.org/10.1145/2070719.2070722

Published: 13 January 2012 Publication History

Abstract

Reading is one of the most frequent activities of knowledge workers. Eye tracking can provide information on what document parts users read, and how they were read. This article aims at generating implicit relevance feedback from eye movements that can be used for information retrieval personalization and further applications.

We report the findings from two studies which examine the relation between several eye movement measures and user-perceived relevance of read text passages. The results show that the measures are generally noisy, but after personalizing them we find clear relations between the measures and relevance. In addition, the second study demonstrates the effect of using reading behavior as implicit relevance feedback for personalizing search. The results indicate that gaze-based feedback is very useful and can greatly improve the quality of Web search. The article concludes with an outlook introducing attentive documents keeping track of how users consume them. Based on eye movement feedback, we describe a number of possible applications to make working with documents more effective.

References

[1]

Agichtein, E., Brill, E., and Dumais, S. 2006. Improving web search ranking by incorporating user behavior information. In Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'06). ACM, New York, NY, 19--26.

Digital Library

[2]

Ahn, J. W., Brusilovsky, P., He, D., Grady, J., and Li, Q. 2008. Personalized web exploration with task models. In Proceedings of the 17th International Conference on World Wide Web (WWW'08). ACM, New York, NY, 1--10.

Digital Library

[3]

Ajanki, A., Hardoon, D., Kaski, S., Puolamäki, K., and Shawe-Taylor, J. 2009. Can eyes reveal interest&quest; Implicit queries from gaze patterns. User Model. User-Adapt. Interact. 19, 307--339.

Digital Library

[4]

Balatsoukas, P. and Ruthven, I. 2010. The use of relevance criteria during predictive judgment: An eye tracking approach. Proc. Amer. Soc. Info. Sci. Techn. 47, 1, 1--10.

Digital Library

[5]

Biedert, R., Buscher, G., and Dengel, A. 2010a. The eyebook -- using eye tracking to enhance the reading experience. Informatik-Spektrum 33, 3, 272--281.

[6]

Biedert, R., Buscher, G., Schwarz, S., Hees, J., and Dengel, A. 2010b. Text 2.0. In CHI'10: Extended Abstracts on Human Factors in Computing Systems. ACM Press, New York, NY, 4003--4008.

Digital Library

[7]

Brooks, P., Phang, K. Y., Bradley, R., Oard, D., White, R., and Guimbretiere, F. 2006. Measuring the utility of gaze detection for task modeling: A preliminary study. In Proceedings of the International Conference on Intelligent User Interfaces (IUI'06). (Workshop on Intelligent User Interfaces for Intelligence Analysis).

[8]

Buscher, G., Cutrell, E., and Morris, M. R. 2009a. What do you see when you're surfing&quest;: using eye tracking to predict salient regions of web pages. In Proceedings of the 27th International Conference on Human Factors in Computing Systems (CHI'09). ACM, New York, NY, 21--30.

Digital Library

[9]

Buscher, G., Dengel, A., and van Elst, L. 2008a. Eye movements as implicit relevance feedback. In CHI'08: Extended Abstracts on Human Factors in Computing Systems. ACM, New York, NY, 2991--2996.

Digital Library

[10]

Buscher, G., Dengel, A., and van Elst, L. 2008b. Query expansion using gaze-based feedback on the subdocument level. In Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'08). ACM, New York, NY, 387--394.

Digital Library

[11]

Buscher, G., van Elst, L., and Dengel, A. 2009b. Segment-level display time as implicit feedback: a comparison to eye tracking. In Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'09). ACM, New York, NY, 67--74.

Digital Library

[12]

Chen, Z. and Xu, Y. 2005. User-oriented relevance judgment: A conceptual model. In Proceedings of the 38th Annual Hawaii International Conference on System Sciences (HICSS'05). IEEE Computer Society, Los Alamitos, CA, 101.2.

Digital Library

[13]

Claypool, M., Le, P., Wased, M., and Brown, D. 2001. Implicit interest indicators. In Proceedings of the 6th International Conference on Intelligent User Interfaces (IUI'01). ACM Press, New York, NY, 33--40.

Digital Library

[14]

Cole, M. J., Gwizdka, J., Bierig, R., Belkin, N. J., Liu, J., Liu, C., and Zhang, X. 2010. Linking search tasks with low-level eye movement patterns. In Proceedings of the 28th Annual European Conference on Cognitive Ergonomics (ECCE'10). ACM, New York, NY, 109--116.

Digital Library

[15]

Cutrell, E. and Guan, Z. 2007. What are you looking for&quest;: an eye-tracking study of information usage in web search. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'07). ACM Press, New York, NY, 407--416.

Digital Library

[16]

Davenport, T. H. and Beck, J. C. 2001. The Attention Economy: Understanding the New Currency of Business. Harvard Business School Press.

[17]

van Elst, L., Kiesel, M., Schwarz, S., Buscher, G., and Lauer, A. 2008. Contextualized Knowledge Acquisition in a Personal Semantic Wiki. In Proceedings of the 16th International Conference on Knowledge Engineering and Knowledge Management (EKAW'08). Springer, Lecture Notes in Computer Science vol. 5268, 172--187.

Digital Library

[18]

Fox, S., Karnawat, K., Mydland, M., Dumais, S., and White, T. 2005. Evaluating implicit measures to improve web search. ACM Trans. Inform. Syst. 23, 2, 147--168.

Digital Library

[19]

Golovchinsky, G., Price, M. N., and Schilit, B. N. 1999. From reading to retrieval: freeform ink annotations as queries. In Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'99). ACM Press, New York, NY, 19--25.

Digital Library

[20]

Gyllstrom, K. 2009. Passages through time: chronicling users' information interaction history by recording when and what they read. In Proceedings of the 13th International Conference on Intelligent User Interfaces (IUI'09). ACM, New York, NY, 147--156.

Digital Library

[21]

Hill, W. C., Hollan, J. D., Wroblewski, D., and McCandless, T. 1992. Edit wear and read wear. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'92). ACM Press, New York, NY, 3--9.

Digital Library

[22]

Hyrskykari, A., Majaranta, P., and Räihä, K.-J. 2003. Proactive response to eye movements. In Proceedings of the International Conferece on Human-Computer Interaction (INTERACT'03). 129--136.

[23]

Järvelin, K. and Kekäläinen, J. 2000. Ir evaluation methods for retrieving highly relevant documents. In Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'00). ACM, New York, NY, 41--48.

Digital Library

[24]

Joachims, T., Granka, L., Pan, B., Hembrooke, H., Radlinski, F., and Gay, G. 2007. Evaluating the accuracy of implicit feedback from clicks and query reformulations in web search. ACM Trans. Inform. Syst. 25, 2.

Digital Library

[25]

Kelly, D. and Belkin, N. J. 2001. Reading time, scrolling and interaction: exploring implicit sources of user preferences for relevance feedback. In Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'01). ACM, New York, NY, 408--409.

Digital Library

[26]

Kelly, D. and Belkin, N. J. 2004. Display time as implicit feedback: understanding task effects. In Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'04). ACM Press, New York, NY, 377--384.

Digital Library

[27]

Kelly, D. and Teevan, J. 2003. Implicit feedback for inferring user preference: a bibliography. SIGIR Forum 37, 2, 18--28.

Digital Library

[28]

Kiesel, M., Schwarz, S., van Elst, L., and Buscher, G. 2008. Using attention and context information for annotations in a semantic wiki. In Proceedings of the 3rd Semantic Wiki Workshop (SemWiki'08).

[29]

Liversedge, S. P. and Findlay, J. M. 2000. Saccadic eye movements and cognition. Trends Cogn. Sci. 4, 1, 6--14.

[30]

Loboda, T. D., Brusilovsky, P., and Brunstein, J. 2011. Inferring word relevance from eye-movements of readers. In Proceedings of the 16th International Conference on Intelligent User Interfaces (IUI'11). ACM, New York, NY, 175--184.

Digital Library

[31]

Majaranta, P. and Räihä, K.-J. 2002. Twenty years of eye typing: systems and design issues. In Proceedings of the Symposium on Eye Tracking Research & Applications (ETRA'02). ACM, New York, NY, 15--22.

Digital Library

[32]

Melucci, M. and White, R. W. 2007. Discovering hidden contextual factors for implicit feedback. In Proceedings of the CIR'07 Workshop on Context-Based Information Retrieval (in conjunction with CONTEXT'07).

[33]

Moe, K. K., Jensen, J. M., and Larsen, B. 2007. A qualitative look at eye-tracking for implicit relevance feedback. In Proceedings of the 2nd International Workshop on Context-Based Information Retrieval. B.-L. Doan, J. Jose, and M. Melucci, Eds., 36--47.

[34]

Morita, M. and Shinoda, Y. 1994. Information filtering based on user behavior analysis and best match text retrieval. In Proceedings of the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'94). Springer, 272--281.

Digital Library

[35]

Ohno, T. 2004. Eyeprint: support of document browsing with eye gaze trace. In Proceedings of the 6th International Conference on Multimodal Interfaces (ICMI'04). ACM, New York, NY, 16--23.

Digital Library

[36]

Puolamäki, K., Salojärvi, J., Savia, E., Simola, J., and Kaski, S. 2005. Combining eye movements and collaborative filtering for proactive information retrieval. In Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'05). ACM Press, New York, NY, 146--153.

Digital Library

[37]

Rayner, K. 1998. Eye movements in reading and information processing: 20 years of research. Psych. Bull. 124, 3, 372--422.

[38]

Rocchio, J. 1971. The SMART Retrieval System: Experiments in Automatic Document Processing. Prentice Hall, 313--323.

Digital Library

[39]

Salton, G. and Buckley, C. 1990. Improving retrieval performance by relevance feedback. J. Amer. Soc. Inf. Sci. 41, 4, 288--297.

[40]

Simon, H. A. 1969. The Sciences of the Artificial. MIT Press.

[41]

Simon, H. A. 1971. Designing organizations for an information rich world. In Computers, Communications and the Public Interest. Johns Hopkins Press, 38--51.

[42]

White, R. W. and Kelly, D. 2006. A study on the effects of personalization and task information on implicit feedback performance. In Proceedings of the 15th ACM International Conference on Information and Knowledge Management (CIKM'06). ACM, New York, NY, 297--306.

Digital Library

[43]

Wilcox, R. 2005. Introduction to Robust Estimation and Hypothesis Testing, 2nd Ed. Elsevier Academic Press.

[44]

Xu, S., Jiang, H., and Lau, F. C. 2009. User-oriented document summarization through vision-based eye-tracking. In Proceedings of the 13th International Conference on Intelligent User Interfaces (IUI'09). ACM, New York, NY, 7--16.

Digital Library

Cited By

Zhu RCheng CSong YCai Z(2024)Modeling Attentive Interaction Behavior for Web Content Identification in Exploratory Information SeekingProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36997508:4(1-28)Online publication date: 21-Nov-2024
https://dl.acm.org/doi/10.1145/3699750
Hienert DSchmidt HKrämer TKern D(2024)EyeLiveMetrics: Real-time Analysis of Online Reading with Eye TrackingProceedings of the 2024 Symposium on Eye Tracking Research and Applications10.1145/3649902.3656495(1-7)Online publication date: 4-Jun-2024
https://dl.acm.org/doi/10.1145/3649902.3656495
Ji KPathiyan Cherumanal STrippas JHettiachchi DSalim FScholer FSpina D(2024)Towards Detecting and Mitigating Cognitive Bias in Spoken Conversational SearchAdjunct Proceedings of the 26th International Conference on Mobile Human-Computer Interaction10.1145/3640471.3680245(1-10)Online publication date: 21-Sep-2024
https://dl.acm.org/doi/10.1145/3640471.3680245
Show More Cited By

Index Terms

Attentive documents: Eye tracking as implicit feedback for information retrieval and beyond

Recommendations

Eye movements as implicit relevance feedback
CHI EA '08: CHI '08 Extended Abstracts on Human Factors in Computing Systems

Reading detection is an important step in the process of automatic relevance feedback generation based on eye movements for information retrieval tasks. We describe a reading detection algorithm and present a preliminary study to find expressive eye ...
The effect of subject familiarity on comprehension and eye movements during reading
OzCHI '13: Proceedings of the 25th Australian Computer-Human Interaction Conference: Augmentation, Application, Innovation, Collaboration

We investigate factors affecting reading and overall comprehension of the underlying meaning and concepts within a piece of text using eye movements. Our objective is to identify eye movement measures that will predict reading comprehension, and intend ...
Finding good feedback documents
CIKM '09: Proceedings of the 18th ACM conference on Information and knowledge management

Pseudo-relevance feedback finds useful expansion terms from a set of top-ranked documents. It is often crucial to identify those good feedback documents from which useful expansion terms can be added to the query. In this paper, we propose to detect ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Interactive Intelligent Systems

ACM Transactions on Interactive Intelligent Systems Volume 1, Issue 2

January 2012

157 pages

ISSN:2160-6455

EISSN:2160-6463

DOI:10.1145/2070719

Issue’s Table of Contents

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 January 2012

Accepted: 01 August 2011

Revised: 01 June 2011

Received: 01 December 2010

Published in TIIS Volume 1, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Project Perspecting
Bundesministerium für Bildung und Forschung

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

82
Total Citations
View Citations
1,404
Total Downloads

Downloads (Last 12 months)76
Downloads (Last 6 weeks)10

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhu RCheng CSong YCai Z(2024)Modeling Attentive Interaction Behavior for Web Content Identification in Exploratory Information SeekingProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36997508:4(1-28)Online publication date: 21-Nov-2024
https://dl.acm.org/doi/10.1145/3699750
Hienert DSchmidt HKrämer TKern D(2024)EyeLiveMetrics: Real-time Analysis of Online Reading with Eye TrackingProceedings of the 2024 Symposium on Eye Tracking Research and Applications10.1145/3649902.3656495(1-7)Online publication date: 4-Jun-2024
https://dl.acm.org/doi/10.1145/3649902.3656495
Ji KPathiyan Cherumanal STrippas JHettiachchi DSalim FScholer FSpina D(2024)Towards Detecting and Mitigating Cognitive Bias in Spoken Conversational SearchAdjunct Proceedings of the 26th International Conference on Mobile Human-Computer Interaction10.1145/3640471.3680245(1-10)Online publication date: 21-Sep-2024
https://dl.acm.org/doi/10.1145/3640471.3680245
Gienapp LScells HDeckers NBevendorff JWang SKiesel JSyed SFröbe MZuccon GStein BHagen MPotthast MHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Evaluating Generative Ad Hoc Information RetrievalProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657849(1916-1929)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657849
Gutnikova OYarosh OKalkova N(2023)Gluten-free products: Consumer perception of the functional properties and peculiarities of labellingUpravlenets10.29141/2218-5003-2023-14-4-614:4(87-99)Online publication date: 7-Sep-2023
https://doi.org/10.29141/2218-5003-2023-14-4-6
SAKAMOTO KSHIRAI STAKEMURA NORLOSKY JNAGATAKI HUEDA MURANISHI YTAKEMURA H(2023)Subjective Difficulty Estimation of Educational Comics Using Gaze FeaturesIEICE Transactions on Information and Systems10.1587/transinf.2022EDP7100E106.D:5(1038-1048)Online publication date: 1-May-2023
https://doi.org/10.1587/transinf.2022EDP7100
Ji KSpina DHettiachchi DSalim FScholer FChen HDuh WHuang HKato MMothe JPoblete B(2023)Examining the Impact of Uncontrolled Variables on Physiological Signals in User Studies for Information Processing ActivitiesProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3591981(1971-1975)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3539618.3591981
Davis KSpape MRuotsalo T(2023)Contradicted by the Brain: Predicting Individual and Group Preferences via Brain-Computer InterfacingIEEE Transactions on Affective Computing10.1109/TAFFC.2022.322588514:4(3094-3105)Online publication date: 1-Oct-2023
https://doi.org/10.1109/TAFFC.2022.3225885
Tahmid ILisle LDavidson KWhitley KNorth CBowman D(2023)Evaluating the Feasibility of Predicting Information Relevance During Sensemaking with Eye Gaze Data2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR59233.2023.00086(713-722)Online publication date: 16-Oct-2023
https://doi.org/10.1109/ISMAR59233.2023.00086
Taieb-Maimon MRomanovski-Chernik ALast MLitvak MElhadad M(2023)Mining Eye-Tracking Data for Text SummarizationInternational Journal of Human–Computer Interaction10.1080/10447318.2023.222782740:17(4887-4905)Online publication date: 21-Jul-2023
https://doi.org/10.1080/10447318.2023.2227827
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents