research-article

Investigation of User Search Behavior While Facing Heterogeneous Search Services

Authors:

Shaoping MaAuthors Info & Claims

WSDM '17: Proceedings of the Tenth ACM International Conference on Web Search and Data Mining

Pages 161 - 170

https://doi.org/10.1145/3018661.3018673

Published: 02 February 2017 Publication History

Abstract

With Web users' search tasks becoming increasingly complex, a single information source cannot necessarily satisfy their information needs. Searchers may rely on heterogeneous sources to complete their tasks, such as search engines, Community Question Answering (CQA), encyclopedia sites, and crowdsourcing platforms. Previous works focus on interaction behaviors with federated search results, including how to compose a federated Web search result page and what factors affect users' interaction behavior on aggregated search interfaces. However, little is known about which factors are crucial in determining users' search outcomes while facing multiple heterogeneous search services. In this paper, we design a lab-based user study to analyze what explicit and implicit factors affect search outcomes (information gain and user satisfaction) when users have access to heterogeneous information sources. In the study, each participant can access three different kinds of search services: a general search engine (Bing), a general CQA portal (Baidu Knows), and a high-quality CQA portal (Zhihu). Using questionnaires and interaction log data, we extract explicit and implicit signals to analyze how users' search outcomes are correlated with their behaviors on different information sources. Experimental results indicate that users' search experiences on CQA portals (such as users' perceived usefulness and number of result clicks) positively affect search outcome (information gain), while search satisfaction is significantly correlated with some other factors such as users' familiarity, interest and difficulty of the task. Besides, users' search satisfaction can be more accurately predicted by the implicit factors than search outcomes.

References

[1]

A. Afuah and C. L. Tucci. Crowdsourcing as a solution to distant search. Academy of Management Review, 37(3):355--375, 2012.

[2]

J. Arguello, F. Diaz, J. Callan, and B. Carterette. A methodology for evaluating aggregated search results. In European Conference on Information Retrieval, pages 141--152. Springer, 2011.

Digital Library

[3]

J. Arguello, F. Diaz, J. Callan, and J.-F. Crespo. Sources of evidence for vertical selection. In Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval, pages 315--322. ACM, 2009.

Digital Library

[4]

J. Arguello, W.-C. Wu, D. Kelly, and A. Edwards. Task complexity, vertical display and user interaction in aggregated search. In Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval, pages 435--444. ACM, 2012.

Digital Library

[5]

O. Boydell and B. Smyth. Capturing community search expertise for personalized web search using snippet-indexes. In Proceedings of the 15th ACM international conference on Information and knowledge management, pages 277--286. ACM, 2006.

Digital Library

[6]

A. Chuklin, K. Zhou, A. Schuth, F. Sietsma, and M. De Rijke. Evaluating intuitiveness of vertical-aware click models. In Proceedings of the 37th international ACM SIGIR conference on Research and development in information retrieval, pages 1075--1078. ACM, 2014.

Digital Library

[7]

K. Collins-Thompson, S. Y. Rieh, C. C. Haynes, and R. Syed. Assessing learning outcomes in web search: A comparison of tasks and query strategies. In Proceedings of the 2016 ACM on Conference on Human Information Interaction and Retrieval, pages 163--172. ACM, 2016.

Digital Library

[8]

Q. Guo, D. Lagun, and E. Agichtein. Predicting web search success with fine-grained interaction data. In Proceedings of the 21st ACM international conference on Information and knowledge management, pages 2050--2054. ACM, 2012.

Digital Library

[9]

R. Hahn, C. Bizer, C. Sahnwaldt, C. Herta, S. Robinson, M. Bürgle, H. Düwiger, and U. Scheel. Faceted wikipedia search. In Business Information Systems, pages 1--11. Springer, 2010.

[10]

A. Hassan, X. Shi, N. Craswell, and B. Ramsey. Beyond clicks: query reformulation as a predictor of search satisfaction. In Proceedings of the 22nd ACM international conference on Conference on information & knowledge management, pages 2019--2028. ACM, 2013.

Digital Library

[11]

J. Jiang, A. Hassan Awadallah, X. Shi, and R. W. White. Understanding and predicting graded search satisfaction. In Proceedings of the Eighth ACM International Conference on Web Search and Data Mining, pages 57--66. ACM, 2015.

Digital Library

[12]

Y. Liu, J. Bian, and E. Agichtein. Predicting information seeker satisfaction in community question answering. In Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval, pages 483--490. ACM, 2008.

Digital Library

[13]

Y. Liu, Y. Chen, J. Tang, J. Sun, M. Zhang, S. Ma, and X. Zhu. Different users, different opinions: Predicting search satisfaction with mouse movement information. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 493--502. ACM, 2015.

Digital Library

[14]

Y. Liu, Z. Liu, K. Zhou, M. Wang, H. Luan, C. Wang, M. Zhang, and S. Ma. Predicting search user examination with visual saliency. pages 619--628, 2016.

Digital Library

[15]

D. C. Montgomery. Design and analysis of experiments. John Wiley & Sons, 2008.

Digital Library

[16]

N. Moraveji. User interface designs to support the social transfer of web search expertise. In Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval, pages 915--915. ACM, 2010.

Digital Library

[17]

N. Moraveji, S. Ahmad, C. Kita, F. Chen, and S. Kamvar. Weblines: Enabling the social transfer of web search expertise using user-generated short-form timelines. In Proceedings of the 9th International Computer-Supported Collaborative Learning Conference, pages 112--119. ISLS, 2011.

[18]

A. K. Ponnuswami, K. Pattabiraman, Q. Wu, R. Gilad-Bachrach, and T. Kanungo. On composition of a federated web search result page: using online users to provide pairwise preference for heterogeneous verticals. In Proceedings of the fourth ACM international conference on Web search and data mining, pages 715--724. ACM, 2011.

Digital Library

[19]

Y. Sun and Y. Zhang. Individual differences and online health information source selection. In Proceedings of the 2016 ACM on Conference on Human Information Interaction and Retrieval, pages 321--324. ACM, 2016.

Digital Library

[20]

S. Sushmita, H. Joho, M. Lalmas, and R. Villa. Factors affecting click-through behavior in aggregated search interfaces. In Proceedings of the 19th ACM international conference on Information and knowledge management, pages 519--528. ACM, 2010.

Digital Library

[21]

H. Wang, Y. Song, M.-W. Chang, X. He, A. Hassan, and R. W. White. Modeling action-level satisfaction for search task satisfaction prediction. In Proceedings of the 37th international ACM SIGIR conference on Research & development in information retrieval, pages 123--132. ACM, 2014.

Digital Library

[22]

R. W. White and D. Morris. Investigating the querying and browsing behavior of advanced search engine users. In Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, pages 255--262. ACM, 2007.

Digital Library

[23]

K. Zhou, R. Cummins, M. Lalmas, and J. M. Jose. Evaluating aggregated search pages. In Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval, pages 115--124. ACM, 2012.

Digital Library

[24]

K. Zhou, R. Cummins, M. Lalmas, and J. M. Jose. Which vertical search engines are relevant? In Proceedings of the 22nd international conference on World Wide Web, pages 1557--1568. ACM, 2013.

Digital Library

[25]

K. Zhou, T. Sakai, M. Lalmas, Z. Dou, and J. M. Jose. Evaluating heterogeneous information access. In Proceedings of MUBE workshop, 2013.

Cited By

Khan ARashid UAhmed N(2022)An Explanatory Study on User Behavior in Discovering Aggregated Multimedia Web ContentIEEE Access10.1109/ACCESS.2022.317759710(56316-56330)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3177597
Khan ARashid USaleem KAhmed A(2021)An architecture for non-linear discovery of aggregated multimedia document web search resultsPeerJ Computer Science10.7717/peerj-cs.4497(e449)Online publication date: 21-Apr-2021
https://doi.org/10.7717/peerj-cs.449
Khan ARashid U(2021)A Relational Aggregated Disjoint Multimedia Search Results Approach using Semantics2021 International Conference on Artificial Intelligence (ICAI)10.1109/ICAI52203.2021.9445229(62-67)Online publication date: 5-Apr-2021
https://doi.org/10.1109/ICAI52203.2021.9445229
Show More Cited By

Index Terms

Investigation of User Search Behavior While Facing Heterogeneous Search Services
1. Information systems
  1. Information retrieval
    1. Users and interactive retrieval
      1. Task models

Recommendations

Patterns of Search Result Examination: Query to First Action
CIKM '19: Proceedings of the 28th ACM International Conference on Information and Knowledge Management

To determine key factors that affect a user's behavior with search results, we conducted a controlled eye-tracking study of users completing search tasks using both desktop and mobile devices. We focus our investigation on users' behavior from their ...
User Behavior Modeling for Web Image Search
WSDM '19: Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining

Web-based image search engines differ from Web search engines greatly. The intents or goals behind human interactions with image search engines are different. In image search, users mainly search images instead of Web pages or online services. It is ...
A Study of Immediate Requery Behavior in Search
CHIIR '18: Proceedings of the 2018 Conference on Human Information Interaction & Retrieval

When search results fail to satisfy users» information needs, users often reformulate their search query in the hopes of receiving better results. In many cases, users immediately requery without clicking on any search results. In this paper, we report ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WSDM '17: Proceedings of the Tenth ACM International Conference on Web Search and Data Mining

February 2017

868 pages

ISBN:9781450346757

DOI:10.1145/3018661

General Chairs:
Maarten de Rijke
University of Amsterdam
,
Milad Shokouhi
Microsoft
,
Program Chairs:
Andrew Tomkins
Google
,
Min Zhang
Tsinghua University

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 February 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

WSDM 2017

Sponsor:

WSDM 2017: Tenth ACM International Conference on Web Search and Data Mining

February 6 - 10, 2017

Cambridge, United Kingdom

Acceptance Rates

WSDM '17 Paper Acceptance Rate 80 of 505 submissions, 16%;

Overall Acceptance Rate 498 of 2,863 submissions, 17%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
294
Total Downloads

Downloads (Last 12 months)9
Downloads (Last 6 weeks)3

Reflects downloads up to 27 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Khan ARashid UAhmed N(2022)An Explanatory Study on User Behavior in Discovering Aggregated Multimedia Web ContentIEEE Access10.1109/ACCESS.2022.317759710(56316-56330)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3177597
Khan ARashid USaleem KAhmed A(2021)An architecture for non-linear discovery of aggregated multimedia document web search resultsPeerJ Computer Science10.7717/peerj-cs.4497(e449)Online publication date: 21-Apr-2021
https://doi.org/10.7717/peerj-cs.449
Khan ARashid U(2021)A Relational Aggregated Disjoint Multimedia Search Results Approach using Semantics2021 International Conference on Artificial Intelligence (ICAI)10.1109/ICAI52203.2021.9445229(62-67)Online publication date: 5-Apr-2021
https://doi.org/10.1109/ICAI52203.2021.9445229
Aldalur IPerez ALarrinaga F(2021)Web Augmentation as a Technique to Diminish User Interactions in Repetitive TasksIEEE Access10.1109/ACCESS.2021.31041879(112686-112704)Online publication date: 2021
https://doi.org/10.1109/ACCESS.2021.3104187
Wang WLi SRao YShen XJia L(2021)An Analysis on User Behaviors in Online Question and Answering CommunitiesComputer Supported Cooperative Work and Social Computing10.1007/978-981-16-2540-4_34(469-483)Online publication date: 7-May-2021
https://doi.org/10.1007/978-981-16-2540-4_34
Aldalur IPerez ALarrinaga F(2019)Customizing Websites Through Automatic Web SearchHuman-Computer Interaction – INTERACT 201910.1007/978-3-030-29384-0_36(598-618)Online publication date: 25-Aug-2019
https://doi.org/10.1007/978-3-030-29384-0_36
Wu DLiang S(2018)Mobile Search Behavious: An In-depth Analysis based on Contexts, APPs, and DevicesSynthesis Lectures on Information Concepts, Retrieval, and Services10.2200/S00831ED1V01Y201802ICR06310:2(i-159)Online publication date: 19-Mar-2018
https://doi.org/10.2200/S00831ED1V01Y201802ICR063
Mao JLiu YKando NZhang MMa S(2018)How Does Domain Expertise Affect Users’ Search Interaction and Outcome in Exploratory Search?ACM Transactions on Information Systems10.1145/322304536:4(1-30)Online publication date: 17-Jul-2018
https://dl.acm.org/doi/10.1145/3223045
Liu MLiu YMao JLuo CZhang MMa SChampin PGandon FMédini LLalmas MIpeirotis P(2018)"Satisfaction with Failure" or "Unsatisfied Success"Proceedings of the 2018 World Wide Web Conference10.1145/3178876.3186065(1533-1542)Online publication date: 10-Apr-2018
https://dl.acm.org/doi/10.1145/3178876.3186065

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten