skip to main content
10.1145/3539618.3591794acmconferencesArticle/Chapter ViewAbstractPublication PagesirConference Proceedingsconference-collections
abstract

Defining and Measuring Cost, Effort, and Load in Information Retrieval

Published: 18 July 2023 Publication History

Abstract

The demands imposed on the user during the Information Retrieval (IR) process are well-recognised, with numerous studies highlighting the influence and importance of cost, effort, and load in explaining user search behaviour and experience. Despite this understanding, there exists no universal definitions of these constructs or standardised methods of measuring them within the field of IR. As a result, an open research problem has emerged in relation to how these constructs have been defined, interpreted, and subsequently measured. To address this research problem, the aim of this research is two-fold. Firstly, to establish working definitions and a conceptual framework for defining cost, effort, and load; and secondly, to examine and evaluate existing measures of effort and load within an IR context.

Supplemental Material

MP4 File
This presentation outlines my doctoral research which focuses on tackling the key challenges facing cost, effort, and load definition and measurement within IR.

References

[1]
Leif Azzopardi, Diane Kelly, and Kathy Brennan. 2013. How Query Cost Affects Search Behavior Categories and Subject Descriptors. Sigir (2013), 23--32. https://doi.org/10.1145/2484028.2484049
[2]
Jacek Gwizdka. 2010. Distribution of cognitive load in Web search. Journal of the American Society for Information Science and Technology, Vol. 61, 11 (2010), 2167--2187. https://doi.org/10.1002/asi.21385
[3]
Martin Halvey and Robert Villa. 2014. Evaluating the effort involved in relevance assessments for images. (2014), 887--890. https://doi.org/10.1145/2600428.2609466
[4]
Molly Mcgregor, Leif Azzopardi, and Martin Halvey. 2023. A Systematic Review of Cost, Efort, and Load Research in Information Search and Retrieval, 1972--2020. (2023). https://doi.org/10.1145/3583069
[5]
Molly Mcgregor, Leif A. Azzopardi, and Martin Halvey. 2021. Untangling Cost, Effort, and Load in Information Seeking and Retrieval. In Proceedings of the 2021 Conference on Human Information Interaction and Retrieval. 151--161.
[6]
Emine Yilmaz, Manisha Verma, Nick Craswell, Filip Radlinski, and Peter Bailey. 2014. Relevance and effort: An analysis of document utility. CIKM 2014 - Proceedings of the 2014 ACM International Conference on Information and Knowledge Management (2014), 91--100. https://doi.org/10.1145/2661829.2661953 io

Index Terms

  1. Defining and Measuring Cost, Effort, and Load in Information Retrieval

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    SIGIR '23: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval
    July 2023
    3567 pages
    ISBN:9781450394086
    DOI:10.1145/3539618
    Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 18 July 2023

    Check for updates

    Author Tags

    1. cognitive load
    2. cost
    3. effort

    Qualifiers

    • Abstract

    Funding Sources

    • European Unionðs Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie Grant Agreement

    Conference

    SIGIR '23
    Sponsor:

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 66
      Total Downloads
    • Downloads (Last 12 months)23
    • Downloads (Last 6 weeks)3
    Reflects downloads up to 05 Mar 2025

    Other Metrics

    Citations

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media