David Maxwell
ABSTRACT
Simulation has been used within the field of Information Retrieval (IR) for many years to evaluate retrieval models and other aspects of the wider IR process. In recent years, there has been a renewed interest towards using simulation for Interactive Information Retrieval (IIR), an area which focuses on the study of human interactions with IR systems. A variety of different interaction models (e.g. click models) associated with behavioural aspects of searchers have over time been developed and evaluated using simulation in order for us to better understand the complex processes involved. Despite these advances, such models are still relatively naïve, and further work is required to make simulations of searchers more realistic. To this end, this project seeks to build more realistic simulations, using a more Complex Searcher Model (CSM). Within the CSM, each component and decision point can be varied and customised as required. The CSM can then be instantiated using components that are grounded from empirical evidence based upon actual real-world searcher behaviour and interaction data.
- L. Azzopardi. The economics in interactive information retrieval. In Proc. 34th ACM SIGIR, pages 15--24, 2011. Google Scholar
Digital Library
- L. Azzopardi. Modelling interaction with economic models of search. In Proc. 37th ACM SIGIR, pages 3--12, 2014. Google ScholarDigital Library
- L. Azzopardi, K. Järvelin, J. Kamps, and M.D. Smucker. Report on the sigir 2010 workshop on the simulation of interaction. SIGIR Forum, 44 (2): 35--47, 2011. Google ScholarDigital Library
- F. Baskaya, H. Keskustalo, and K. Järvelin. Modeling behavioral factors in interactive information retrieval. In Proc. 22nd ACM CIKM, pages 2297--2302, 2013. Google ScholarDigital Library
- G.J. Browne, M.G. Pitts, and J.C. Wetherbe. Stopping rule use during web-based search. In Proc. HICSS-38, page 271b, 2005. Google ScholarDigital Library
- W.S. Cooper. On selecting a measure of retrieval effectiveness part ii. implementation of the philosophy. J. of the American Soc. for Info. Sci., 24 (6): 413--424, 1973.Google ScholarCross Ref
- W.S. Cooper. The paradoxical role of unexamined documents in the evaluation of retrieval effectiveness. Info. Processing and Management, 12 (6): 367--375, 1976.Google ScholarCross Ref
- D. Harman. Is the cranfield paradigm outdated? In Proceedings of SIGIR 2010, pages 1--1, 2010. Google ScholarDigital Library
- H. Keskustalo, K. Järvelin, and A. Pirkola. Evaluating the effectiveness of relevance feedback based on a user simulation model: Effects of a user scenario on cumulated gain value. Information Retrieval, 11 (3): 209--228, 2008. Google ScholarDigital Library
- D.H. Kraft and T. Lee. Stopping rules and their effect on expected search length. IPM, 15 (1): 47--58, 1979.Google Scholar
- D. Maxwell and L. Azzopardi. Stuck in traffic: How temporal delays affect search behaviour. In Proc. 5th IIiX, pages 155--164, 2014. Google ScholarDigital Library
- D. Maxwell, L. Azzopardi, K. Järvelin, and H. Keskustalo. An initial investigation into fixed and adaptive stopping strategies. In Proc. 38th ACM SIGIR, pages 903--906, 2015. Google ScholarDigital Library
- D. Maxwell, L. Azzopardi, K. Järvelin, and H. Keskustalo. Searching and stopping: An analysis of stopping rules and strategies. In Proc. 24th ACM CIKM, pages 313--322, 2015. Google ScholarDigital Library
- A. Moffat, P. Thomas, and F. Scholer. Users versus models: What observation tells us about effectiveness metrics. In Proc. 22nd ACM CIKM, pages 659--668, 2013. Google ScholarDigital Library
- M.D. Smucker. An analysis of user strategies for examining and processing ranked lists of documents. In Proc. of 5th HCIR, 2011.Google Scholar
- P. Thomas, A. Moffat, P. Bailey, and F. Scholer. Modeling decision points in user search behavior. In Proc. 5th IIiX, pages 239--242, 2014. Google ScholarDigital Library
Index Terms
- Building Realistic Simulations for Interactive Information Retrieval
Recommendations
Agents, Simulated Users and Humans: An Analysis of Performance and Behaviour
CIKM '16: Proceedings of the 25th ACM International on Conference on Information and Knowledge ManagementMost of the current models that are used to simulate users in Interactive Information Retrieval (IIR) lack realism and agency. Such models generally make decisions in a stochastic manner, without recourse to the actual information encountered or the ...
Simulating Interactive Information Retrieval: SimIIR: A Framework for the Simulation of Interaction
SIGIR '16: Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information RetrievalSimulation provides a powerful and cost-effective approach to explore and evaluate how interactions between a searcher and system influence search behaviour and performance. With a growing interest in simulation and an increasing number of papers using ...
On Obtaining Effort Based Judgements for Information Retrieval
WSDM '16: Proceedings of the Ninth ACM International Conference on Web Search and Data MiningDocument relevance has been the primary focus in the design, optimization and evaluation of retrieval systems. Traditional testcollections are constructed by asking judges the relevance grade for a document with respect to an input query. Recent work of ...
Comments