research-article

Influence of Vertical Result in Web Search Examination

Authors:

Shaoping MaAuthors Info & Claims

SIGIR '15: Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval

Pages 193 - 202

https://doi.org/10.1145/2766462.2767714

Published: 09 August 2015 Publication History

Abstract

Research in how users examine results on search engine result pages (SERPs) helps improve result ranking, advertisement placement, performance evaluation and search UI design. Although examination behavior on organic search results (also known as "ten blue links") has been well studied in existing works, there lacks a thorough investigation on how users examine SERPs with verticals. Considering the fact that a large fraction of SERPs are served with one or more verticals in the practical Web search scenario, it is of vital importance to understand the influence of vertical results on search examination behaviors. In this paper, we focus on five popular vertical types and try to study their influences on users' examination processes in both cases when they are relevant or irrelevant to the search queries. With examination behavior data collected with an eye-tracking device, we show the existence of vertical-aware user behavior effects including vertical attraction effect, examination cut-off effect in the presence of a relevant vertical, and examination spill-over effect in the presence of an irrelevant vertical. Furthermore, we are also among the first to systematically investigate the internal examination behavior within the vertical results. We believe that this work will promote our understanding of user interactions with federated search engines and bring benefit to the construction of search performance evaluations.

References

[1]

J. Arguello and R. Capra. The effects of vertical rank and border on aggregated search coherence and search behavior. In Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management, pages 539--548. ACM, 2014.

Digital Library

[2]

J. Arguello, F. Diaz, and J. Callan. Learning to aggregate vertical results into web search results. In Proceedings of the 20th ACM international conference on Information and knowledge management, pages 201--210. ACM, 2011.

Digital Library

[3]

J. Arguello, F. Diaz, J. Callan, and J.-F. Crespo. Sources of evidence for vertical selection. In Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval, pages 315--322. ACM, 2009.

Digital Library

[4]

A. Aula, P. Majaranta, and K.-J. Raiha. Eye-tracking reveals the personal styles for search result evaluation. In Human-Computer Interaction-INTERACT 2005, pages 1058--1061. Springer, 2005.

Digital Library

[5]

J. Bar-Ilan, K. Keenoy, M. Levene, and E. Yaari. Presentation bias is significant in determining user preference for search results - a user study. Journal of the American Society for Information Science and Technology, 60(1):135--149, 2009.

Digital Library

[6]

A. Broder. A taxonomy of web search. In ACM Sigir forum, volume 36, pages 3--10. ACM, 2002.

Digital Library

[7]

G. Buscher, A. Dengel, and L. van Elst. Eye movements as implicit relevance feedback. In CHI'08 extended abstracts on Human factors in computing systems, pages 2991--2996. ACM, 2008.

Digital Library

[8]

G. Buscher, S. T. Dumais, and E. Cutrell. The good, the bad, and the random: an eye-tracking study of ad quality in web search. In Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval, pages 42--49. ACM, 2010.

Digital Library

[9]

D. Chen, W. Chen, H. Wang, Z. Chen, and Q. Yang. Beyond ten blue links: enabling user click modeling in federated web search. In Proceedings of the fifth ACM international conference on Web search and data mining, pages 463--472. ACM, 2012.

Digital Library

[10]

E. Cutrell and Z. Guan. What are you looking for?: an eye-tracking study of information usage in web search. In Proceedings of the SIGCHI conference on Human factors in computing systems, pages 407--416. ACM, 2007.

Digital Library

[11]

F. Diaz, R. White, G. Buscher, and D. Liebling. Robust models of mouse movement on dynamic web search results pages. In Proceedings of the 22nd ACM international conference on Conference on information & knowledge management, pages 1451--1460. ACM, 2013.

Digital Library

[12]

S. T. Dumais, G. Buscher, and E. Cutrell. Individual differences in gaze patterns for web search. In Proceedings of the third symposium on Information interaction in context, pages 185--194. ACM, 2010.

Digital Library

[13]

J. Fernquist and E. H. Chi. Perception and understanding of social annotations in web search. In Proceedings of the 22nd international conference on World Wide Web, pages 403--412. International World Wide Web Conferences Steering Committee, 2013.

Digital Library

[14]

L. A. Granka, T. Joachims, and G. Gay. Eye-tracking analysis of user behavior in www search. In Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval, pages 478--479. ACM, 2004.

Digital Library

[15]

Q. Guo and E. Agichtein. Towards predicting web searcher gaze position from mouse movements. In CHI'10 Extended Abstracts on Human Factors in Computing Systems, pages 3601--3606. ACM, 2010.

Digital Library

[16]

K. Hofmann, B. Mitra, F. Radlinski, and M. Shokouhi. An eye-tracking study of user interactions with query auto completion. In Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management, pages 549--558. ACM, 2014.

Digital Library

[17]

J. Huang, R. White, and G. Buscher. User see, user point: gaze and cursor alignment in web search. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 1341--1350. ACM, 2012.

Digital Library

[18]

T. Joachims, L. Granka, B. Pan, H. Hembrooke, and G. Gay. Accurately interpreting clickthrough data as implicit feedback. In Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, pages 154--161. ACM, 2005.

Digital Library

[19]

T. Joachims, L. Granka, B. Pan, H. Hembrooke, F. Radlinski, and G. Gay. Evaluating the accuracy of implicit feedback from clicks and query reformulations in web search. ACM Transactions on Information Systems (TOIS), 25(2):7, 2007.

Digital Library

[20]

D. Lagun, C.-H. Hsieh, D. Webster, and V. Navalpakkam. Towards better measurement of attention and satisfaction in mobile search. In Proceedings of the 37th international ACM SIGIR conference on Research & development in information retrieval, pages 113--122. ACM, 2014.

Digital Library

[21]

X. Li, Y.-Y. Wang, and A. Acero. Learning query intent from regularized click graphs. In Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval, pages 339--346. ACM, 2008.

Digital Library

[22]

Y. Liu, C. Wang, K. Zhou, J. Nie, M. Zhang, and S. Ma. From skimming to reading: A two-stage examination model for web search. In Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management, pages 849--858. ACM, 2014.

Digital Library

[23]

V. Navalpakkam, L. Jentzsch, R. Sayres, S. Ravi, A. Ahmed, and A. Smola. Measurement and modeling of eye-mouse behavior in the presence of nonlinear page layouts. In Proceedings of the 22nd international conference on World Wide Web, pages 953--964. International World Wide Web Conferences Steering Committee, 2013.

Digital Library

[24]

M. Richardson, E. Dominowska, and R. Ragno. Predicting clicks: estimating the click-through rate for new ads. In Proceedings of the 16th international conference on World Wide Web, pages 521--530. ACM, 2007.

Digital Library

[25]

K. Rodden and X. Fu. Exploring how mouse movements relate to eye movements on web search results pages. Web Information Seeking and Interaction, pages 29--32, 2007.

[26]

K. Rodden, X. Fu, A. Aula, and I. Spiro. Eye-mouse coordination patterns on web search results pages. In CHI'08 Extended Abstracts on Human Factors in Computing Systems, pages 2997--3002. ACM, 2008.

Digital Library

[27]

S. Sushmita, H. Joho, M. Lalmas, and R. Villa. Factors affecting click-through behavior in aggregated search interfaces. In Proceedings of the 19th ACM international conference on Information and knowledge management, pages 519--528. ACM, 2010.

Digital Library

[28]

C. Wang, Y. Liu, M. Zhang, S. Ma, M. Zheng, J. Qian, and K. Zhang. Incorporating vertical results into search click models. In Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval, pages 503--512. ACM, 2013.

Digital Library

[29]

K. Zhou, R. Cummins, M. Lalmas, and J. M. Jose. Evaluating aggregated search pages. In Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval, pages 115--124. ACM, 2012.

Digital Library

[30]

K. Zhou, R. Cummins, M. Lalmas, and J. M. Jose. Which vertical search engines are relevant? In Proceedings of the 22nd international conference on World Wide Web, pages 1557--1568. International World Wide Web Conferences Steering Committee, 2013.

Digital Library

Cited By

Taieb-Maimon M(2025)Enhancing Snippet Visualizations to Improve Web SearchInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2443267(1-20)Online publication date: 15-Jan-2025
https://doi.org/10.1080/10447318.2024.2443267
Chen MLiu CLiu ZLi ZSun JSalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)Identifiability mattersProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3692342(7057-7080)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3692342
Momeni M(2024)Visualization-Enhanced Aggregated Search InterfacesProceedings of the 2024 Conference on Human Information Interaction and Retrieval10.1145/3627508.3638336(461-464)Online publication date: 10-Mar-2024
https://dl.acm.org/doi/10.1145/3627508.3638336
Show More Cited By

Index Terms

Influence of Vertical Result in Web Search Examination
1. Information systems
  1. Information retrieval

Recommendations

Predicting Search User Examination with Visual Saliency
SIGIR '16: Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval

Predicting users' examination of search results is one of the key concerns in Web search related studies. With more and more heterogeneous components federated into search engine result pages (SERPs), it becomes difficult for traditional position-based ...
Incorporating vertical results into search click models
SIGIR '13: Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval

In modern search engines, an increasing number of search result pages (SERPs) are federated from multiple specialized search engines (called verticals, such as Image or Video). As an effective approach to interpret users' click-through behavior as ...
Result List Actions in Fiction Search
JCDL '15: Proceedings of the 15th ACM/IEEE-CS Joint Conference on Digital Libraries

It is studied how users browse search results to find interesting novels for four search scenarios. It is evaluated in particular whether there are differences in search result page (SERP) browsing patterns and effectiveness between an enriched catalog ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR '15: Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval

August 2015

1198 pages

ISBN:9781450336215

DOI:10.1145/2766462

General Chair:
Ricardo Baeza-Yates
Yahoo Labs, USA
,
Program Chairs:
Mounia Lalmas
Yahoo Labs, UK
,
Alistair Moffat
University of Melbourne, Australia
,
Berthier Ribeiro-Neto
Google, Brazil, and UFMG, Brazil

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 August 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SIGIR '15

Sponsor:

SIGIR

SIGIR '15: The 38th International ACM SIGIR conference on research and development in Information Retrieval

August 9 - 13, 2015

Santiago, Chile

Acceptance Rates

SIGIR '15 Paper Acceptance Rate 70 of 351 submissions, 20%;

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

47
Total Citations
View Citations
592
Total Downloads

Downloads (Last 12 months)16
Downloads (Last 6 weeks)2

Reflects downloads up to 27 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Taieb-Maimon M(2025)Enhancing Snippet Visualizations to Improve Web SearchInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2443267(1-20)Online publication date: 15-Jan-2025
https://doi.org/10.1080/10447318.2024.2443267
Chen MLiu CLiu ZLi ZSun JSalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)Identifiability mattersProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3692342(7057-7080)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3692342
Momeni M(2024)Visualization-Enhanced Aggregated Search InterfacesProceedings of the 2024 Conference on Human Information Interaction and Retrieval10.1145/3627508.3638336(461-464)Online publication date: 10-Mar-2024
https://dl.acm.org/doi/10.1145/3627508.3638336
Mao HZou LZheng YTang JChu XZhao JWang QYin DChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Whole Page Unbiased Learning to RankProceedings of the ACM Web Conference 202410.1145/3589334.3645474(1431-1440)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645474
Taieb-Maimon MHarush H(2024)Web Search Engine Results Page Viewing Formats for Different Search TasksInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2376358(1-16)Online publication date: 29-Jul-2024
https://doi.org/10.1080/10447318.2024.2376358
Chen HAi QWu ZWang ZLiu YZhang MMa SHu JTan NChai HChen HDuh WHuang HKato MMothe JPoblete B(2023)Behavior Modeling for Point of Interest SearchProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3591955(1843-1847)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3539618.3591955
Liu JLiu J(2023)Behavioral Economics in IRA Behavioral Economics Approach to Interactive Information Retrieval10.1007/978-3-031-23229-9_6(155-180)Online publication date: 18-Feb-2023
https://doi.org/10.1007/978-3-031-23229-9_6
Liu JLiu J(2023)Formally Modeling Users in Information RetrievalA Behavioral Economics Approach to Interactive Information Retrieval10.1007/978-3-031-23229-9_2(23-64)Online publication date: 18-Feb-2023
https://doi.org/10.1007/978-3-031-23229-9_2
Chen MLiu CLiu ZSun JKoyejo SMohamed SAgarwal ABelgrave DCho KOh A(2022)LBDProceedings of the 36th International Conference on Neural Information Processing Systems10.5555/3600270.3602690(33400-33413)Online publication date: 28-Nov-2022
https://dl.acm.org/doi/10.5555/3600270.3602690
Chen MLiu CLiu ZSun JZhang ARangwala H(2022)Scalar is Not EnoughProceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3534678.3539468(136-145)Online publication date: 14-Aug-2022
https://dl.acm.org/doi/10.1145/3534678.3539468
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten