Abstract
Search task difficulty refers to a user’s assessment about the amount of effort required to complete a search task. Our goal in this work is to learn predictive models of search task difficulty. We evaluate features derived from the user’s interaction with the search engine as well as features derived from the user’s level of interest in the task and level of prior knowledge in the task domain. In addition to user-interaction features used in prior work, we evaluate features generated from scroll and mouse-movement events on the SERP. In some situations, we may prefer a system that can predict search task difficulty early in the search session. To this end, we evaluate features in terms of whole-session evidence and first-round evidence, which excludes all interactions starting with the second query. Our results found that the most predictive features were different for whole-session vs. first-round prediction, that mouseover features were effective for first-round prediction, and that level of interest and prior knowledge features did not improve performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Anderson, L.W., Krathwohl, D.R.: A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives (2001)
Aula, A., Khan, R.M., Guan, Z.: How does search behavior change as search becomes more difficult? In: CHI, pp. 35–44 (2010)
Bell, D.J., Ruthven, I.: Searchers’ assessments of task complexity for web searching. In: McDonald, S., Tait, J.I. (eds.) ECIR 2004. LNCS, vol. 2997, pp. 57–71. Springer, Heidelberg (2004)
Byström, K., Järvelin, K.: Task complexity affects information seeking and use. Inf. Process. Manage. 31(2), 191–213 (1995)
Campbell, D.J.: Task complexity: A review and analysis. The Academy of Management Review 13(1), 40–52 (1988)
Fan, R.-E., Chang, K.-W., Hsieh, C.-J., Wang, X.-R., Lin, C.-J.: Liblinear: A library for large linear classification. JMLR 9, 1871–1874 (2008)
Feild, H.A., Allan, J., Jones, R.: Predicting searcher frustration. In: SIGIR, pp. 34–41 (2010)
Jansen, B.J., Booth, D., Smith, B.: Using the taxonomy of cognitive learning to model online searching. Inf. Process. Manage. 45(6), 643–663 (2009)
Kim, J.: Task difficulty as a predictor and indicator of web searching interaction. In: CHI, pp. 959–964 (2006)
Li, Y., Belkin, N.J.: A faceted approach to conceptualizing tasks in information seeking. Inf. Process. Manage. 44(6), 1822–1837 (2008)
Liu, J., Cole, M.J., Liu, C., Bierig, R., Gwizdka, J., Belkin, N.J., Zhang, J., Zhang, X.: Search behaviors in different task types. In: JCDL, pp. 69–78 (2010)
Liu, J., Gwizdka, J., Liu, C., Belkin, N.J.: Predicting task difficulty for different task types. ASIS&T, 16:1–16:10 (2010)
Liu, J., Liu, C., Cole, M., Belkin, N.J., Zhang, X.: Exploring and predicting search task difficulty. In: CIKM, pp. 1313–1322 (2012)
Liu, J., Liu, C., Gwizdka, J., Belkin, N.J.: Can search systems detect users‘ task difficulty?: some behavioral signals. In: SIGIR, pp. 845–846 (2010)
Smucker, M.D., Allan, J., Carterette, B.: A comparison of statistical significance tests for information retrieval evaluation. In: CIKM, pp. 623–632 (2007)
White, R.W., Dumais, S.T.: Characterizing and predicting search engine switching behavior. In: CIKM, pp. 87–96 (2009)
Wood, R.E.: Task complexity: Definition of the construct. Organizational Behavior and Human Decision Processes 37(1), 60–82 (1986)
Wu, W.-C., Kelly, D., Edwards, A., Arguello, J.: Grannies, tanning beds, tattoos and nascar: evaluation of search tasks with varying levels of cognitive complexity. In: IIIX, pp. 254–257 (2012)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Arguello, J. (2014). Predicting Search Task Difficulty. In: de Rijke, M., et al. Advances in Information Retrieval. ECIR 2014. Lecture Notes in Computer Science, vol 8416. Springer, Cham. https://doi.org/10.1007/978-3-319-06028-6_8
Download citation
DOI: https://doi.org/10.1007/978-3-319-06028-6_8
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-06027-9
Online ISBN: 978-3-319-06028-6
eBook Packages: Computer ScienceComputer Science (R0)