research-article

Engagement Periodicity in Search Engine Usage: Analysis and its Application to Search Quality Evaluation

Authors:

Pavel SerdyukovAuthors Info & Claims

WSDM '15: Proceedings of the Eighth ACM International Conference on Web Search and Data Mining

Pages 27 - 36

https://doi.org/10.1145/2684822.2685318

Published: 02 February 2015 Publication History

Abstract

Nowadays, billions of people use the Web in connection with their daily needs. A significant part of the needs are constituted by search tasks that are usually addressed by search engines. Thus, daily search needs result in regular user engagement with a search engine. User engagement with web sites and services was studied in various aspects, but there appear to be no studies of its regularity and periodicity. In this paper, we studied periodicity of the user engagement with a popular search engine through applying spectrum analysis to temporal sequences of different engagement metrics. We found periodicity patterns of user engagement and revealed classes of users whose periodicity patterns do not change over a long period of time. In addition, we used the spectrum series as metrics to evaluate search quality.

References

[1]

E. Adar, J. Teevan, and S. T. Dumais. Large scale analysis of web revisitation patterns. In CHI'2008, pages 1197--1206, 2008.

Digital Library

[2]

S. Chakraborty, F. Radlinski, M. Shokouhi, and P. Baecke. On correlation of absence time and search effectiveness. In SIGIR'2014, pages 1163--1166, 2014.

Digital Library

[3]

A. Deng, Y. Xu, R. Kohavi, and T. Walker. Improving the sensitivity of online controlled experiments by utilizing pre-experiment data. In WSDM'2013, pages 123--132, 2013.

Digital Library

[4]

A. Diriye, R. White, G. Buscher, and S. Dumais. Leaving so soon?: understanding and predicting web search abandonment rationales. In CIKM'2012, pages 1025--1034, 2012.

Digital Library

[5]

G. Dupret and M. Lalmas. Absence time and user engagement: evaluating ranking functions. In WSDM'2013, pages 173--182, 2013.

Digital Library

[6]

H. A. Feild, J. Allan, and R. Jones. Predicting searcher frustration. In SIGIR'2010, pages 34--41, 2010.

Digital Library

[7]

Q. Guo, R. W. White, Y. Zhang, B. Anderson, and S. T. Dumais. Why searchers switch: understanding and predicting engine switching rationales. In SIGIR'2011, pages 335--344, 2011.

Digital Library

[8]

A. Hassan, X. Shi, N. Craswell, and B. Ramsey. Beyond clicks: query reformulation as a predictor of search satisfaction. In CIKM'2013, pages 2019--2028, 2013.

Digital Library

[9]

A. Hassan, Y. Song, and L.-w. He. A task level metric for measuring web search satisfaction and its application on improving relevance estimation. In CIKM'2011, pages 125--134, 2011.

Digital Library

[10]

A. Hassan and R. W. White. Personalized models of search satisfaction. In CIKM'2013, pages 2009--2018, 2013.

Digital Library

[11]

V. Hu, M. Stone, J. Pedersen, and R. W. White. Effects of search success on search engine re-use. In CIKM'2011, pages 1841--1846. ACM, 2011.

Digital Library

[12]

B. J. Jansen, A. Spink, and V. Kathuria. How to define searching sessions on web search engines. In Advances in Web Mining and Web Usage Analysis, pages 92--109. Springer, 2007.

Digital Library

[13]

R. Kohavi, A. Deng, R. Longbotham, and Y. Xu. Seven rules of thumb for web site experimenters. In KDD'2014, 2014.

Digital Library

[14]

T. Kramar and M. Bielikova. Context of seasonality in web search. In Advances in Information Retrieval, pages 644--649. Springer, 2014.

[15]

J. Lehmann, M. Lalmas, G. Dupret, and R. Baeza-Yates. Online multitasking and user engagement. In CIKM'2013, pages 519--528, 2013.

Digital Library

[16]

J. Lehmann, M. Lalmas, E. Yom-Tov, and G. Dupret. Models of user engagement. In User Modeling, Adaptation, and Personalization, pages 164--175. Springer, 2012.

Digital Library

[17]

K. Radinsky, K. Svore, S. Dumais, J. Teevan, A. Bocharov, and E. Horvitz. Modeling and predicting behavioral dynamics on the web. In WWW, 2012.

Digital Library

[18]

K. Rodden, H. Hutchinson, and X. Fu. Measuring the user experience on a large scale: user-centered metrics for web applications. In SIGCHI'2010, pages 2395--2398, 2010.

Digital Library

[19]

D. Savenkov, D. Lagun, and Q. Liu. Search engine switching detection based on user personal preferences and behavior patterns. In SIGIR'2013, pages 33--42, 2013.

Digital Library

[20]

M. Shokouhi. Detecting seasonal queries by time-series analysis. In SIGIR'2011, pages 1171--1172, 2011.

Digital Library

[21]

Y. Song, X. Shi, and X. Fu. Evaluating and predicting user engagement change with degraded search relevance. In WWW'2013, pages 1213--1224, 2013.

Digital Library

[22]

M. Vlachos, C. Meek, Z. Vagena, and D. Gunopulos. Identifying similarities, periodicities and bursts for online search queries. In SIGMOD'2004, pages 131--142, 2004.

Digital Library

[23]

W. W.-S. Wei. Time series analysis. Addison-Wesley Redwood City, California, 1994.

[24]

R. West, R. W. White, and E. Horvitz. From cookies to cooks: Insights on dietary patterns via analysis of web usage logs. In WWW'2013, pages 1399--1410, 2013.

Digital Library

[25]

R. W. White and S. T. Dumais. Characterizing and predicting search engine switching behavior. In CIKM'2009, pages 87--96, 2009.

Digital Library

[26]

R. W. White, A. Kapoor, and S. T. Dumais. Modeling long-term search engine usage. In User Modeling, Adaptation, and Personalization, pages 28--39. Springer, 2010.

Digital Library

[27]

Y. Zhang, B. J. Jansen, and A. Spink. Time series analysis of a web search engine transaction log. Information Processing & Management, 45(2):230--245, 2009.

Digital Library

Cited By

Zhuang MConcannon DManley E(2022)A Framework for Evaluating Dashboards in HealthcareIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.314715428:4(1715-1731)Online publication date: 1-Apr-2022
https://doi.org/10.1109/TVCG.2022.3147154
Drutsa AGusev GKharitonov EKulemyakin DSerdyukov PYashkov IPiwowarski BChevalier MGaussier EMaarek YNie JScholer F(2019)Effective Online Evaluation for Web SearchProceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3331184.3331378(1399-1400)Online publication date: 18-Jul-2019
https://dl.acm.org/doi/10.1145/3331184.3331378
Doherty KDoherty G(2018)Engagement in HCIACM Computing Surveys10.1145/323414951:5(1-39)Online publication date: 19-Nov-2018
https://dl.acm.org/doi/10.1145/3234149
Show More Cited By

Index Terms

Engagement Periodicity in Search Engine Usage: Analysis and its Application to Search Quality Evaluation
1. Computing methodologies
  1. Artificial intelligence
    1. Philosophical/theoretical foundations of artificial intelligence
      1. Cognitive science
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods

Recommendations

Periodicity in User Engagement with a Search Engine and Its Application to Online Controlled Experiments

Nowadays, billions of people use the Web in connection with their daily needs. A significant part of these needs are constituted by search tasks that are usually addressed by search engines. Thus, daily search needs result in regular user engagement ...
Evaluating and predicting user engagement change with degraded search relevance
WWW '13: Proceedings of the 22nd international conference on World Wide Web

User engagement in search refers to the frequency for users (re-)using the search engine to accomplish their tasks. Among factors that affected users' visit frequency, relevance of search results is believed to play a pivotal role. While multiple work ...
Google Search Engine: Seo Tools You Need to Explode Your Website Traffic - Google Seo, Google Ranking

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WSDM '15: Proceedings of the Eighth ACM International Conference on Web Search and Data Mining

February 2015

482 pages

ISBN:9781450333177

DOI:10.1145/2684822

General Chairs:
Xueqi Cheng
ICT, Chinese Academy of Sciences, China
,
Hang Li
Huawei Technologies, China
,
Program Chairs:
Evgeniy Gabrilovich
Google, USA
,
Jie Tang
Tsinghua University, China

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 February 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

WSDM 2015

Sponsor:

WSDM 2015: Eighth ACM International Conference on Web Search and Data Mining

February 2 - 6, 2015

Shanghai, China

Acceptance Rates

WSDM '15 Paper Acceptance Rate 39 of 238 submissions, 16%;

Overall Acceptance Rate 498 of 2,863 submissions, 17%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

17
Total Citations
View Citations
389
Total Downloads

Downloads (Last 12 months)7
Downloads (Last 6 weeks)0

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhuang MConcannon DManley E(2022)A Framework for Evaluating Dashboards in HealthcareIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.314715428:4(1715-1731)Online publication date: 1-Apr-2022
https://doi.org/10.1109/TVCG.2022.3147154
Drutsa AGusev GKharitonov EKulemyakin DSerdyukov PYashkov IPiwowarski BChevalier MGaussier EMaarek YNie JScholer F(2019)Effective Online Evaluation for Web SearchProceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3331184.3331378(1399-1400)Online publication date: 18-Jul-2019
https://dl.acm.org/doi/10.1145/3331184.3331378
Doherty KDoherty G(2018)Engagement in HCIACM Computing Surveys10.1145/323414951:5(1-39)Online publication date: 19-Nov-2018
https://dl.acm.org/doi/10.1145/3234149
He RYang SYang JCao J(2018)Automated Mining of Approximate Periodicity on Numeric DataProceedings of the 2nd International Conference on Compute and Data Analysis10.1145/3193077.3194509(20-27)Online publication date: 23-Mar-2018
https://dl.acm.org/doi/10.1145/3193077.3194509
Budylin RDrutsa AKatsev ITsoy VChang YZhai CLiu YMaarek Y(2018)Consistent Transformation of Ratio Metrics for Efficient Online Controlled ExperimentsProceedings of the Eleventh ACM International Conference on Web Search and Data Mining10.1145/3159652.3159699(55-63)Online publication date: 2-Feb-2018
https://dl.acm.org/doi/10.1145/3159652.3159699
Machmouchi WAwadallah AZitouni IBuscher GLim EWinslett MSanderson MFu ASun JCulpepper SLo EHo JDonato DAgrawal RZheng YCastillo CSun ATseng VLi C(2017)Beyond Success RateProceedings of the 2017 ACM on Conference on Information and Knowledge Management10.1145/3132847.3132850(757-765)Online publication date: 6-Nov-2017
https://dl.acm.org/doi/10.1145/3132847.3132850
Li ZKiseleva Jde Rijke MGrotov AKamps JKanoulas Ede Rijke MFang HYilmaz E(2017)Towards Learning Reward Functions from User InteractionsProceedings of the ACM SIGIR International Conference on Theory of Information Retrieval10.1145/3121050.3121098(289-292)Online publication date: 1-Oct-2017
https://dl.acm.org/doi/10.1145/3121050.3121098
Drutsa AGusev GSerdyukov PBarrett RCummings RAgichtein EGabrilovich E(2017)Using the Delay in a Treatment Effect to Improve Sensitivity and Preserve Directionality of Engagement Metrics in A/B ExperimentsProceedings of the 26th International Conference on World Wide Web10.1145/3038912.3052664(1301-1310)Online publication date: 3-Apr-2017
https://dl.acm.org/doi/10.1145/3038912.3052664
Zamani HBendersky MWang XZhang MBarrett RCummings RAgichtein EGabrilovich E(2017)Situational Context for Ranking in Personal SearchProceedings of the 26th International Conference on World Wide Web10.1145/3038912.3052648(1531-1540)Online publication date: 3-Apr-2017
https://dl.acm.org/doi/10.1145/3038912.3052648
Drutsa AGusev GSerdyukov P(2017)Periodicity in User Engagement with a Search Engine and Its Application to Online Controlled ExperimentsACM Transactions on the Web10.1145/285682211:2(1-35)Online publication date: 14-Apr-2017
https://dl.acm.org/doi/10.1145/2856822
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten