research-article

Lessons from the journey: a query log analysis of within-session learning

Authors:

Carsten Eickhoff,

Susan DumaisAuthors Info & Claims

WSDM '14: Proceedings of the 7th ACM international conference on Web search and data mining

Pages 223 - 232

https://doi.org/10.1145/2556195.2556217

Published: 24 February 2014 Publication History

Abstract

The Internet is the largest source of information in the world. Search engines help people navigate the huge space of available data in order to acquire new skills and knowledge. In this paper, we present an in-depth analysis of sessions in which people explicitly search for new knowledge on the Web based on the log files of a popular search engine. We investigate within-session and cross-session developments of expertise, focusing on how the language and search behavior of a user on a topic evolves over time. In this way, we identify those sessions and page visits that appear to significantly boost the learning process. Our experiments demonstrate a strong connection between clicks and several metrics related to expertise. Based on models of the user and their specific context, we present a method capable of automatically predicting, with good accuracy, which clicks will lead to enhanced learning. Our findings provide insight into how search engines might better help users learn as they search.

References

[1]

John R Anderson. Language, Memory, and Thought. Earlbaum, 1976.

[2]

Azin Ashkan, Charles LA Clarke, Eugene Agichtein, and Qi Guo. Classifying and characterizing query intent. In ECIR 2009.

Digital Library

[3]

Ricardo Baeza-Yates, Liliana Calderón-Benavides, and Cristina González-Caro. The intention behind web queries. In SPIRE 2006.

Digital Library

[4]

Nicholas J Belkin, Robert N Oddy, and Helen M Brooks. Ask for information retrieval: Part i. background and theory. Journal of documentation, 1982.

[5]

Paul N Bennett, Ryen W White, Wei Chu, Susan T Dumais, Peter Bailey, Fedor Borisyuk, and Xiaoyuan Cui. Modeling the impact of short-and long-term behavior on search personalization. In SIGIR 2012.

Digital Library

[6]

Michael S Bernstein, Jaime Teevan, Susan Dumais, Daniel Liebling, and Eric Horvitz. Direct answers for search queries in the long tail. In SIGCHI 2012.

Digital Library

[7]

Andrei Broder. A taxonomy of web search. In ACM SIGIR Forum 2002.

Digital Library

[8]

Jamie Callan and Maxine Eskenazi. Combining lexical and grammatical features to improve readability measures for first and second language texts. In NAACL HLT 2007.

[9]

Kevyn Collins-Thompson, Paul N Bennett, Ryen W White, Sebastian de la Chica, and David Sontag. Personalizing web search results by reading level. In CIKM 2011.

Digital Library

[10]

Maurice de Kunder. Daily estimated size of the world wide web. http://www.worldwidewebsize.com/, 2013.

[11]

Carsten Eickhoff, Kevyn Collins-Thompson, Paul N. Bennett, and Susan T. Dumais. Personalizing atypical web search sessions. In WSDM 2013.

Digital Library

[12]

Carsten Eickhoff, Pieter Dekker, and Arjen P de Vries. Supporting children's web search in school environments. In IIiX 2012.

Digital Library

[13]

Gene Golovchinsky, John Adcock, Jeremy Pickens, Pernilla Qvarfordt, and Maribeth Back. Cerchiamo: a collaborative exploratory search tool. Proceedings of Computer Supported Cooperative Work (CSCW), 2008.

[14]

Thorsten Joachims, Laura Granka, Bing Pan, Helene Hembrooke, and Geri Gay. Accurately interpreting clickthrough data as implicit feedback. In SIGIR 2005.

Digital Library

[15]

Rosie Jones and Kristina Lisa Klinkner. Beyond the session timeout: automatic hierarchical segmentation of search topics in query logs. In CIKM 2008.

Digital Library

[16]

Jin Young Kim, Kevyn Collins-Thompson, Paul N Bennett, and Susan T Dumais. Characterizing web content, user interests, and search behavior by reading level and topic. In WSDM 2012.

Digital Library

[17]

Aris Kosmopoulos, Eric Gaussier, Georgios Paliouras, and Sujeevan Aseervatham. The ecir 2010 large scale hierarchical classification workshop. In ACM SIGIR Forum, 2010.

Digital Library

[18]

Bill Kules, Robert Capra, Matthew Banta, and Tito Sierra. What do exploratory searchers look at in a faceted search interface? In 2009 ACM/IEEE-CS joint conference on Digital libraries.

Digital Library

[19]

Victor Kuperman, Hans Stadthagen-Gonzalez, and Marc Brysbaert. Age-of-acquisition ratings for 30,000 english words. Behavior Research Methods, 2012.

[20]

Lin Li, Zhenglu Yang, Botao Wang, and Masaru Kitsuregawa. Dynamic adaptation strategies for long-term and short-term user profile to personalize search. In Advances in Data and Web Management. 2007.

Digital Library

[21]

Jingjing Liu, Nicholas J Belkin, Xiangmin Zhang, and Xiaojun Yuan. Examining users' knowledge change in the task completion process. Information Processing & Management, 2012.

Digital Library

[22]

Gary Marchionini. Exploratory search: from finding to understanding. Communications of the ACM, 2006.

Digital Library

[23]

G Harry McLaughlin. Smog grading: A new readability formula. Journal of Reading, 1969.

[24]

Thomas R Niesler and PC Woodland. A variable-length category-based n-gram language model. In ICASSP 1996.

Digital Library

[25]

Brendan O'Connor, Michel Krieger, and David Ahn. Tweetmotif: Exploratory search and topic summarization for twitter. In ICWSM, 2010.

[26]

James Pitkow, Hinrich Schütze, Todd Cass, Rob Cooley, Don Turnbull, Andy Edmonds, Eytan Adar, and Thomas Breuel. Personalized search. Communications of the ACM, 2002.

Digital Library

[27]

Daniel E Rose and Danny Levinson. Understanding user goals in web search. In WWW 2004.

Digital Library

[28]

Dou Shen, Jian-Tao Sun, Qiang Yang, and Zheng Chen. A comparison of implicit and explicit links for web page classification. In WWW 2006.

Digital Library

[29]

Mirco Speretta and Susan Gauch. Personalized search based on user search histories. In 2005 IEEE/WIC/ACM International Conference on Web Intelligence.

Digital Library

[30]

Sofia Stamou and Lefteris Kozanidis. Impact of search results on user queries. In International Workshop on Web Information and Data Management 2009.

Digital Library

[31]

Shanu Sushmita, Benjamin Piwowarski, and Mounia Lalmas. Dynamics of genre and domain intents. In Information Retrieval Technology. 2010.

[32]

Chenhao Tan, Evgeniy Gabrilovich, and Bo Pang. To each his own: personalized content selection based on text comprehensibility. In WSDM 2012.

Digital Library

[33]

Jaime Teevan, Susan T Dumais, and Eric Horvitz. Personalizing search via automated analysis of interests and activities. In SIGIR 2005.

Digital Library

[34]

Jaime Teevan, Susan T Dumais, and Eric Horvitz. Potential for personalization. ACM TOCHI 2010.

Digital Library

[35]

Takashi Tomokiyo and Matthew Hurst. A language model approach to keyphrase extraction. In Proceedings of the ACL 2003 workshop on Multiword expressions: analysis, acquisition and treatment-Volume 18.

Digital Library

[36]

Ryen W. White, Susan T. Dumais, and Jaime Teevan. Characterizing the inuence of domain expertise on web search behavior. In WSDM 2009.

Digital Library

[37]

Ryen W White and Jeff Huang. Assessing the scenic route: measuring the value of search trails in web logs. In SIGIR 2010.

Digital Library

[38]

Barbara M Wildemuth. The effects of domain knowledge on search tactic formulation. JASIST 2004.

Digital Library

[39]

Zahide Yildirim, M Yasar Ozden, and Meral Aksu. Comparison of hypermedia learning and traditional instruction on knowledge acquisition and retention. The Journal of Educational Research, 2001.

[40]

Xiangmin Zhang, Michael Cole, and Nicholas Belkin. Predicting users' domain knowledge from search behaviors. In SIGIR 2011.

Digital Library

Cited By

Schumacher AKammerer YScharinger CGottschling SHübner NTibus MKasneci EAppel TGerjets PBardach L(2025)How do intellectually curious and interested people learn and attain knowledge? A focus on behavioral traces of information seekingEuropean Journal of Personality10.1177/08902070241309124Online publication date: 19-Jan-2025
https://doi.org/10.1177/08902070241309124
Lafia SMillion AHemphill L(2024)Exploratory and directed search strategies at a social science data archiveIASSIST Quarterly10.29173/iq108748:1Online publication date: 28-Mar-2024
https://doi.org/10.29173/iq1087
Urgo KArguello J(2024)The Effects of Goal-setting on Learning Outcomes and Self-Regulated Learning ProcessesProceedings of the 2024 Conference on Human Information Interaction and Retrieval10.1145/3627508.3638348(278-290)Online publication date: 10-Mar-2024
https://dl.acm.org/doi/10.1145/3627508.3638348
Show More Cited By

Index Terms

Lessons from the journey: a query log analysis of within-session learning
1. Computing methodologies
  1. Machine learning
    1. Learning settings
2. Information systems
  1. Information retrieval
    1. Information retrieval query processing

Recommendations

Actively predicting diverse search intent from user browsing behaviors
WWW '10: Proceedings of the 19th international conference on World wide web

This paper is concerned with actively predicting search intent from user browsing behavior data. In recent years, great attention has been paid to predicting user search intent. However, the prediction was mostly passive because it was performed only ...
Search Engine Optimization to detect user's intent
BDCA'17: Proceedings of the 2nd international Conference on Big Data, Cloud and Applications

The major evolution of the search engine is to understand the user's query and user's intent. The other size change and you all know, is the evolution of the mobile query. Indeed, research on search engines is trying today to bring results of research ...
Effective rank aggregation for metasearching

Nowadays, mashup services and especially metasearch engines play an increasingly important role on the Web. Most of users use them directly or indirectly to access and aggregate information from more than one data sources. Similarly to the rest of the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WSDM '14: Proceedings of the 7th ACM international conference on Web search and data mining

February 2014

712 pages

ISBN:9781450323512

DOI:10.1145/2556195

General Chairs:
Ben Carterette
University of Delaware, USA
,
Fernando Diaz
Microsoft Research, USA
,
Program Chairs:
Carlos Castillo
Qatar Computing Research Institute, Qatar
,
Donald Metzler
Google, USA

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 February 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

WSDM 2014

Sponsor:

WSDM 2014: Seventh ACM International Conference on Web Search and Data Mining

February 24 - 28, 2014

New York, New York, USA

Acceptance Rates

WSDM '14 Paper Acceptance Rate 64 of 355 submissions, 18%;

Overall Acceptance Rate 498 of 2,863 submissions, 17%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

104
Total Citations
View Citations
776
Total Downloads

Downloads (Last 12 months)61
Downloads (Last 6 weeks)13

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Schumacher AKammerer YScharinger CGottschling SHübner NTibus MKasneci EAppel TGerjets PBardach L(2025)How do intellectually curious and interested people learn and attain knowledge? A focus on behavioral traces of information seekingEuropean Journal of Personality10.1177/08902070241309124Online publication date: 19-Jan-2025
https://doi.org/10.1177/08902070241309124
Lafia SMillion AHemphill L(2024)Exploratory and directed search strategies at a social science data archiveIASSIST Quarterly10.29173/iq108748:1Online publication date: 28-Mar-2024
https://doi.org/10.29173/iq1087
Urgo KArguello J(2024)The Effects of Goal-setting on Learning Outcomes and Self-Regulated Learning ProcessesProceedings of the 2024 Conference on Human Information Interaction and Retrieval10.1145/3627508.3638348(278-290)Online publication date: 10-Mar-2024
https://dl.acm.org/doi/10.1145/3627508.3638348
Sekulic IBalog KCrestani F(2024)Towards Self-Contained Answers: Entity-Based Answer Rewriting in Conversational SearchProceedings of the 2024 Conference on Human Information Interaction and Retrieval10.1145/3627508.3638300(209-218)Online publication date: 10-Mar-2024
https://dl.acm.org/doi/10.1145/3627508.3638300
Juneja PZhang WSmith-Renner ALamba HTetreault JJaimes A(2024)Dissecting users' needs for search result explanationsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642059(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642059
Burchart MHaake J(2024)Supporting Collaborative Writing Tasks in Large-Scale Distance EducationIEEE Transactions on Learning Technologies10.1109/TLT.2024.335579117(1051-1068)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TLT.2024.3355791
Kameni JBatchakui BNkambou R(2024)Optimization of Information Retrieval Systems for Learning ContextsInternational Journal of Artificial Intelligence in Education10.1007/s40593-024-00415-z35:1(65-95)Online publication date: 8-Jul-2024
https://doi.org/10.1007/s40593-024-00415-z
Gritz WHoppe AEwerth R(2024)On the Influence of Reading Sequences on Knowledge Gain During Web SearchAdvances in Information Retrieval10.1007/978-3-031-56063-7_28(364-373)Online publication date: 24-Mar-2024
https://dl.acm.org/doi/10.1007/978-3-031-56063-7_28
Vahedparast HGanjoo MGhasemi FDehghani L(2023)Information-seeking process and clinical scenario solving: introduction of a new tool in nursing educationBMC Medical Education10.1186/s12909-023-04943-523:1Online publication date: 12-Dec-2023
https://doi.org/10.1186/s12909-023-04943-5
Choi BCasteel SArguello JCapra R(2023)Better Understanding Procedural Search Tasks: Perceptions, Behaviors, and ChallengesACM Transactions on Information Systems10.1145/363000442:3(1-32)Online publication date: 29-Dec-2023
https://dl.acm.org/doi/10.1145/3630004
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten