Skip to main content
SpringerLink
Log in

Reproduction and Simulation of Interactive Retrieval Experiments

  • Conference paper
  • First Online:
Advances in Information Retrieval (ECIR 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14612))

Included in the following conference series:

  • 320 Accesses

Abstract

The reproducibility crisis, spanning across various scientific fields, substantially affects information retrieval research [1].

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Baker, M.: 1,500 Scientists lift the lid on reproducibility. Nature 533,(7604) (2016)

    Google Scholar 

  2. Balog, K., Maxwell, D., Thomas, P., Zhang, S.: Report on the 1st simulation for information retrieval workshop (Sim4IR 2021) at SIGIR2021. In: ACM SIGIR Forum vol. 55, no. 2 (2022)

    Google Scholar 

  3. Balog, K., Zhai, C.: User Simulation for Evaluating Information Access Systems. arXiv preprint arXiv:2306.08550 (2023)

  4. Breuer, T., Fuhr, N., Schaer, P.: Validating simulations of user query variants. In: Hagen, M., Verberne, S., Macdonald, C., Seifert, C., Balog, K., Nørvåg, K., Setty, V. (eds.) ECIR 2022. LNCS, vol. 13185, pp. 80–94. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-99736-6_6

    Chapter  Google Scholar 

  5. Breuer, T.: Reproducible Information Retrieval Research: From Principled System-Oriented Evaluations Towards User-Oriented Experimentation, University of Duisburg-Essen (2023)

    Google Scholar 

  6. Ferro, N., Fuhr, N., Jarvelin, K., Kando, N., Lippold, M., Zobel, J.: Increasing reproducibility in IR: findings from the dagstuhl seminar on” reproducibility of data-oriented experiments in e-science. In: ACM SIGIR Forum. vol. 50, no. 1, pp. 68–82. ACM, New York, NY, USA (2016)

    Google Scholar 

  7. Fuhr, N.: A probability ranking principle for interactive information retrieval. Inf. Retrieval 11(3), 251–265 (2008)

    Google Scholar 

  8. Kelly, D.: Methods for evaluating interactive information retrieval systems with users. Found. Trends in Inf. Retrieval 3(1–2), 1–224 (2009)

    Google Scholar 

  9. Liu, J., Shah, C.: Interactive IR user study design, evaluation, and reporting. Morgan & Claypool Publishers (2019)

    Google Scholar 

  10. Maistro, M., Breuer, T., Schaer, P., Ferro, N.: An in-depth investigation on the behavior of measures to quantify reproducibility. Inf. Process. & Manage 60(3), 103332 (2023)

    Google Scholar 

  11. Maxwell, D., Azzopardi, L.: Simulating interactive information retrieval: SimIIR: A framework for the simulation of interaction. In: Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval, SIGIR 2016, pp. 1141–1144. ACM, Pisa, Italy (2016)

    Google Scholar 

  12. Petras, V., Bogers, T., Gade, M.: Elements of IIR studies: a review of the 2006-2018 IIiX and CHIIR conferences. In: CEUR Workshop Proceedings, vol. 2337, pp. 37–41 (2019)

    Google Scholar 

  13. Tran, V.: Modellierung von Benutzerverhalten im interaktiven Retrieval mit Markov-Ketten, University of Duisburg-Essen (2020)

    Google Scholar 

  14. Yang, G.H., Dong, X., Luo, J., Zhang, S.: Session search modeling by partially observable Markov decision process. Inform. Retrieval J. 21(1), 56–80 (2017). https://doi.org/10.1007/s10791-017-9316-8

    Article  Google Scholar 

  15. Zerhoudi, S., et al.: The SimIIR 2.0 framework: user types, markov model-based interaction simulation, and advanced query generation. In: Proceedings of the 31st ACM International Conference on Information and Knowledge Management, pp. 4661–4666 (2022)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Duisburg-Essen, Duisburg, Germany

    Jana Isabelle Friese

Authors
  1. Jana Isabelle Friese
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jana Isabelle Friese .

Editor information

Editors and Affiliations

  1. Georgetown University, Washington, WA, USA

    Nazli Goharian

  2. University of Pisa, PISA, Pisa, Italy

    Nicola Tonellotto

  3. King's College London, London, UK

    Yulan He

  4. University College London, London, UK

    Aldo Lipani

  5. University of Glasgow, Glasgow, UK

    Graham McDonald

  6. University of Glasgow, Glasgow, UK

    Craig Macdonald

  7. University of Glasgow, Glasgow, UK

    Iadh Ounis

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Friese, J.I. (2024). Reproduction and Simulation of Interactive Retrieval Experiments. In: Goharian, N., et al. Advances in Information Retrieval. ECIR 2024. Lecture Notes in Computer Science, vol 14612. Springer, Cham. https://doi.org/10.1007/978-3-031-56069-9_40

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-56069-9_40

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-56068-2

  • Online ISBN: 978-3-031-56069-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation