Dmitry Lagun
ABSTRACT
Previous studies of web search result examination have provided valuable insights in understanding and modelling searcher behavior. Yet, recent work (e.g., [3]) has been developed based on the assumption that the time a searcher spends examining a particular result abstract or snippet, correlates with result relevance. While this idea is intuitively attractive, to the best of our knowledge it has not been empirically tested. This poster investigates this hypothesis empirically, in a controlled setting, using eye tracking equipment to compare search result examination time with result relevance. Interestingly, while we replicate previous findings showing examination time to be indicative of whole-page relevance, we find that viewing time of individual results alone is a poor indicator of either absolute result relevance or even of pairwise preferences. Our results should not be taken as negating the usefulness of modeling searcher examination behavior, but rather to emphasize that snippet examination time is not in itself a good indicator of relevance.
- G. Buscher, A. Dengel, R. Biedert, and L. V. Elst. Attentive documents: Eye tracking as implicit feedback for information retrieval and beyond. ACM Trans. Interact. Intell. Syst., 1(2):9:1--9:30, 2012. Google Scholar
Digital Library
- Z. Guan and E. Cutrell. An eye tracking study of the effect of target rank on web search. In Proc. of CHI, pages 417--420, 2007. Google ScholarDigital Library
- Y. He and K. Wang. Inferring search behaviors using partially observable markov model with duration (pomd). In Proc. of WSDM, pages 415--424, 2011. Google ScholarDigital Library
- T. Joachims, L. Granka, B. Pan, H. Hembrooke, F. Radlinski, and G. Gay. Evaluating the accuracy of implicit feedback from clicks and query reformulations in web search. ACM TOIS, 25(2):7, 2007. Google ScholarDigital Library
- D. Lagun and E. Agichtein. Viewser: enabling large-scale remote user studies of web search examination and interaction. In Proceedings of the 34th international ACM SIGIR conference on Research and development in Information, pages 365--374. ACM, 2011. Google ScholarDigital Library
Index Terms
- Re-examining search result snippet examination time for relevance estimation
Recommendations
Snippet based relevance calculations
The purpose of Web Host Access Tool (WHAT) project is to formulate a search based on a user's present needs, and past search results. The tool gathers information from the user, sends out a query to various search engines, ranks the responses, and then ...
Relevance Estimation with Multiple Information Sources on Search Engine Result Pages
CIKM '18: Proceedings of the 27th ACM International Conference on Information and Knowledge ManagementRelevance estimation is among the most important tasks in the ranking of search results because most search engines follow the Probability Ranking Principle. Current relevance estimation methodologies mainly concentrate on text matching between the ...
Sponsored Search: Is Money a Motivator for Providing Relevant Results?
Analysis of data from a major metasearch engine reveals that sponsored-link click-through rates appear lower than previously reported. Combining sponsored and nonsponsored links in a single listing, while providing some benefits to users, does not ...
Comments