research-article

Query expansion using gaze-based feedback on the subdocument level

Authors:

Andreas Dengel,

Ludger van ElstAuthors Info & Claims

SIGIR '08: Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval

Pages 387 - 394

https://doi.org/10.1145/1390334.1390401

Published: 20 July 2008 Publication History

Abstract

We examine the effect of incorporating gaze-based attention feedback from the user on personalizing the search process. Employing eye tracking data, we keep track of document parts the user read in some way. We use this information on the subdocument level as implicit feedback for query expansion and reranking.

We evaluated three different variants incorporating gaze data on the subdocument level and compared them against a baseline based on context on the document level. Our results show that considering reading behavior as feedback yields powerful improvements of the search result accuracy of ca. 32% in the general case. However, the extent of the improvements varies depending on the internal structure of the viewed documents and the type of the current information need.

References

[1]

E. Agichtein, E. Brill, and S. Dumais. Improving web search ranking by incorporating user behavior information. In SIGIR '06, pages 19--26, 2006. ACM.

Digital Library

[2]

E. Agichtein, E. Brill, S. Dumais, and R. Ragno. Learning user interaction models for predicting web search result preferences. In SIGIR '06, pages 3--10, 2006. ACM.

Digital Library

[3]

P. Brooks, K. Y. Phang, R. Bradley, D. Oard, R. White, and F. Guimbreti`ere. Measuring the utility of gaze detection for task modeling: A preliminary study. In International Conference on Intelligent User Interfaces (IUI'06), Workshop on Intelligent User Interfaces for Intelligence Analysis, Sydney, 2006.

[4]

G. Buscher. Attention-based information retrieval. In SIGIR '07, pages 918--918, 2007. ACM.

Digital Library

[5]

G. Buscher, A. Dengel, and L. van Elst. Eye movements as implicit relevance feedback. In CHI '08 extended abstracts on Human factors in computing systems, pages 2991--2996, 2008. ACM.

Digital Library

[6]

G. Buscher, A. Dengel, and L. van Elst. Eye movements as implicit relevance feedback. In CHI '08 extended abstracts on Human factors in computing systems, pages 2991--2996, 2008. ACM.

Digital Library

[7]

P. A. Chirita, C. S. Firan, and W. Nejdl. Personalized query expansion for the web. In SIGIR '07, pages 7--14, 2007. ACM.

Digital Library

[8]

E. Cutrell and Z. Guan. What are you looking for?: an eye-tracking study of information usage in web search. In CHI '07: Proceedings of the SIGCHI conference on Human factors in computing systems, pages 407--416, 2007. ACM.

Digital Library

[9]

A. Dengel. Six thousand words about multiperspective personal document management. In Proc. IEEE-EDM: IEEE Internat. Workshop on the Electronic Document Management in an Enterprise Comp. Env., Key Note Paper, pages 1--10, 2006.

Digital Library

[10]

S. Fox, K. Karnawat, M. Mydland, S. Dumais, and T. White. Evaluating implicit measures to improve web search. ACM Transactions on Information Systems, 23(2):147--168, 2005.

Digital Library

[11]

G. Golovchinsky, M. N. Price, and B. N. Schilit. From reading to retrieval: freeform ink annotations as queries. In SIGIR '99, pages 19--25, 1999. ACM.

Digital Library

[12]

L. A. Granka, T. Joachims, and G. Gay. Eye-tracking analysis of user behavior in www search. In SIGIR '04, pages 478--479, 2004. ACM.

Digital Library

[13]

K. Järvelin and J. Kekäläinen. IR evaluation methods for retrieving highly relevant documents. In SIGIR '00, pages 41--48, 2000. ACM.

Digital Library

[14]

T. Joachims, L. Granka, B. Pan, H. Hembrooke, and G. Gay. Accurately interpreting clickthrough data as implicit feedback. In SIGIR '05, pages 154--161, 2005.

Digital Library

[15]

D. Kelly and N. J. Belkin. Display time as implicit feedback: understanding task effects. In SIGIR '04, pages 377--384, 2004. ACM.

Digital Library

[16]

D. Kelly and J. Teevan. Implicit feedback for inferring user preference: a bibliography. SIGIR Forum, 37(2):18--28, 2003.

Digital Library

[17]

P. P. Maglio, R. Barrett, C. S. Campbell, and T. Selker. Suitor: An attentive information system. In IUI-2000: International Conference on Intelligent User Interfaces, pages 169--176, 2000.

Digital Library

[18]

T. Miller and S. Agne. Attention-based information retrieval using eye tracker data. In K-CAP '05: Proceedings of the 3rd international conference on Knowledge capture, pages 209--210, 2005. ACM.

Digital Library

[19]

F. Qiu and J. Cho. Automatic identification of user interest for personalized search. In WWW '06, pages 727--736, 2006. ACM.

Digital Library

[20]

F. Radlinski and T. Joachims. Query chains: learning to rank from implicit feedback. In KDD '05: Proceeding of the eleventh ACM SIGKDD international conference on Knowledge discovery and data mining, pages 239--248, 2005. ACM.

Digital Library

[21]

J. Rocchio. The SMART Retrieval System: Experiments in Automatic Document Processing, chapter Relevance feedback in information retrieval, pages 313--323. Prentice Hall, 1971.

[22]

J. Salojärvi, K. Puolamäki, and S. Kaski. Implicit relevance feedback from eye movements. In ICANN'05, pages 513--518, 2005.

Digital Library

[23]

X. Shen, B. Tan, and C. Zhai. Implicit user modeling for personalized search. In CIKM '05: Proceedings of the 14th ACM international conference on Information and knowledge management, pages 824--831, 2005. ACM.

Digital Library

[24]

K. Spärck-Jones, S. Walker, and S. E. Robertson. A probabilistic model of information retrieval: development and status. Technical Report 446, University of Cambridge Computer Laboratory, 1998.

[25]

K. Sugiyama, K. Hatano, and M. Yoshikawa. Adaptive web search based on user profile constructed without any effort from users. In WWW '04, pages 675--684, 2004. ACM.

Digital Library

[26]

J. Teevan, S. T. Dumais, and E. Horvitz. Personalizing search via automated analysis of interests and activities. In SIGIR '05, pages 449--456, 2005. ACM.

Digital Library

[27]

K. Yang, K. L. Maglaughlin, and G. B. Newby. Passage feedback with IRIS. In Information Processing and Management, 37, 3, pages 521--541, 2001.

Digital Library

[28]

S. Yu, D. Cai, J.-R. Wen, and W.-Y. Ma. Improving pseudo-relevance feedback in web information retrieval using web page segmentation. In WWW '03, pages 11--18, 2003. ACM.

Digital Library

Cited By

Lobodenko LMatveeva IShesterkina LZagoskin E(2024)Eye-Tracking Study into Patterns of Attention to Environmental Media Texts among Youth Audiences in the Context of the Communicative Strategy2024 Communication Strategies in Digital Society Seminar (ComSDS)10.1109/ComSDS61892.2024.10502069(83-88)Online publication date: 10-Apr-2024
https://doi.org/10.1109/ComSDS61892.2024.10502069
Chen JZhang LQian W(2022)Cognitive differences between readers attentive and inattentive to task-related information: an eye-tracking studyAslib Journal of Information Management10.1108/AJIM-01-2022-000775:5(917-939)Online publication date: 4-Oct-2022
https://doi.org/10.1108/AJIM-01-2022-0007
Vuong TAndolina SJacucci GRuotsalo T(2021)Does More Context Help? Effects of Context Window and Application Source on Retrieval PerformanceACM Transactions on Information Systems10.1145/347405540:2(1-40)Online publication date: 27-Sep-2021
https://dl.acm.org/doi/10.1145/3474055
Show More Cited By

Index Terms

Query expansion using gaze-based feedback on the subdocument level
1. Information systems
  1. Information retrieval
    1. Evaluation of retrieval results
      1. Relevance assessment

Recommendations

Segment-level display time as implicit feedback: a comparison to eye tracking
SIGIR '09: Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval

We examine two basic sources for implicit relevance feedback on the segment level for search personalization: eye tracking and display time. A controlled study has been conducted where 32 participants had to view documents in front of an eye tracker, ...
Gaze-based interest detection on newspaper articles
PETMEI '18: Proceedings of the 7th Workshop on Pervasive Eye Tracking and Mobile Eye-Based Interaction

Eye tracking measures have been used to recognize cognitive states involving mental workload, comprehension, and self-confidence in the task of reading. In this paper, we present how these measures can be used to detect the interest of a reader. From the ...
Beyond gaze: preliminary analysis of pupil dilation and blink rates in an fMRI study of program comprehension
EMIP '18: Proceedings of the Workshop on Eye Movements in Programming

Researchers have been employing psycho-physiological measures to better understand program comprehension, for example simultaneous fMRI and eye tracking to validate top-down comprehension models. In this paper, we argue that there is additional value in ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR '08: Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval

July 2008

934 pages

ISBN:9781605581644

DOI:10.1145/1390334

General Chairs:
Tat-Seng Chua
National University of Singapore
,
Mun-Kew Leong
National Library Board, Singapore
,
Program Chairs:
Syung Hyon Myaeng
Information and Communications University, Korea
,
Douglas W. Oard
University of Maryland, College Park, USA
,
Fabrizio Sebastiani
Consiglio Nazionale delle Ricerche, Italy

Copyright © 2008 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 July 2008

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SIGIR '08

Sponsor:

SIGIR '08: The 31st Annual International ACM SIGIR Conference

July 20 - 24, 2008

Singapore, Singapore

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

62
Total Citations
View Citations
971
Total Downloads

Downloads (Last 12 months)10
Downloads (Last 6 weeks)1

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Lobodenko LMatveeva IShesterkina LZagoskin E(2024)Eye-Tracking Study into Patterns of Attention to Environmental Media Texts among Youth Audiences in the Context of the Communicative Strategy2024 Communication Strategies in Digital Society Seminar (ComSDS)10.1109/ComSDS61892.2024.10502069(83-88)Online publication date: 10-Apr-2024
https://doi.org/10.1109/ComSDS61892.2024.10502069
Chen JZhang LQian W(2022)Cognitive differences between readers attentive and inattentive to task-related information: an eye-tracking studyAslib Journal of Information Management10.1108/AJIM-01-2022-000775:5(917-939)Online publication date: 4-Oct-2022
https://doi.org/10.1108/AJIM-01-2022-0007
Vuong TAndolina SJacucci GRuotsalo T(2021)Does More Context Help? Effects of Context Window and Application Source on Retrieval PerformanceACM Transactions on Information Systems10.1145/347405540:2(1-40)Online publication date: 27-Sep-2021
https://dl.acm.org/doi/10.1145/3474055
Hansen CHansen CSimonsen JLarsen BAlstrup SLioma CHuang JChang YCheng XKamps JMurdock VWen JLiu Y(2020)Factuality Checking in News Headlines with Eye TrackingProceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3397271.3401221(2013-2016)Online publication date: 25-Jul-2020
https://doi.org/10.1145/3397271.3401221
Feit AVordemann LPark SBerube CHilliges O(2020)Detecting Relevance during Decision-Making from Eye Movements for UI AdaptationACM Symposium on Eye Tracking Research and Applications10.1145/3379155.3391321(1-11)Online publication date: 2-Jun-2020
https://dl.acm.org/doi/10.1145/3379155.3391321
Davari MHienert DKern DDietze SO'Brien HFreund LArapakis IHoeber OLopatovska I(2020)The Role of Word-Eye-Fixations for Query Term PredictionProceedings of the 2020 Conference on Human Information Interaction and Retrieval10.1145/3343413.3378010(422-426)Online publication date: 14-Mar-2020
https://dl.acm.org/doi/10.1145/3343413.3378010
Hienert DKern DMitsui MShah CBelkin NAzzopardi LHalvey MRuthven IJoho HMurdock VQvarfordt P(2019)Reading ProtocolProceedings of the 2019 Conference on Human Information Interaction and Retrieval10.1145/3295750.3298921(73-81)Online publication date: 8-Mar-2019
https://dl.acm.org/doi/10.1145/3295750.3298921
Serim BJacucci GBrewster SFitzpatrick GCox AKostakos V(2019)Explicating "Implicit Interaction"Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290605.3300647(1-16)Online publication date: 2-May-2019
https://dl.acm.org/doi/10.1145/3290605.3300647
Bui KNguyen TNguyen TDao T(2018)An Effective of Data Organizing Method Combines with Naïve Bayes for Vietnamese Document RetrievalContext-Aware Systems and Applications, and Nature of Computation and Communication10.1007/978-3-319-77818-1_20(205-213)Online publication date: 16-Mar-2018
https://doi.org/10.1007/978-3-319-77818-1_20
Pian WKhoo CChi J(2017)Automatic Classification of Users’ Health Information Need Context: Logistic Regression Analysis of Mouse-Click and Eye-Tracker DataJournal of Medical Internet Research10.2196/jmir.835419:12(e424)Online publication date: 21-Dec-2017
https://doi.org/10.2196/jmir.8354
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten