research-article

A Visual Analytics Approach for What-If Analysis of Information Retrieval Systems

Authors:

Marco Angelini,

Nicola Ferro,

Giuseppe Santucci,

Gianmaria SilvelloAuthors Info & Claims

SIGIR '16: Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval

Pages 1081 - 1084

https://doi.org/10.1145/2911451.2911462

Published: 07 July 2016 Publication History

Get Access

Abstract

We present the innovative visual analytics approach of the VATE system, which eases and makes more effective the experimental evaluation process by introducing the what-if analysis. The what-if analysis is aimed at estimating the possible effects of a modification to an IR system to select the most promising fixes before implementing them, thus saving a considerable amount of effort. VATE builds on an analytical framework which models the behavior of the systems in order to make estimations, and integrates this analytical framework into a visual part which, via proper interaction and animations, receives input and provides feedback to the user.

References

[1]

M. Angelini, N. Ferro, G. Santucci, and G. Silvello. A Visual Interactive Environment for Making Sense of Experimental Data. In Proc. 36th European Conference on IR Research (ECIR 2014), pages 767--770. LNCS 8416, Springer, 2014.

Crossref

Google Scholar

[2]

M. Angelini, N. Ferro, G. Santucci, and G. Silvello. VIRTUE: A visual tool for information retrieval performance evaluation and failure analysis. Journal of Vis. Lang. & Comp. (JVLC), 25(4):394--413, 2014.

Digital Library

Google Scholar

[3]

N. Ferro and G. Silvello. What-If Analysis: A Visual Analytics Approach to Information Retrieval Evaluation. In Proc. 7th Italian Information Retrieval Workshop (IIR 2016). CEUR Workshop Proc. (CEUR-WS.org), 2016.

Google Scholar

[4]

K. Jarvelin and J. Kekalainen. Cumulated Gain-Based Evaluation of IR Techniques. ACM Trans. on Inf. Sys. (TOIS), 20(4):422--446, October 2002.

Digital Library

Google Scholar

[5]

C. J. van Rijsbergen. Information Retrieval. Butterworths, London, England, 2nd edition, 1979.

Digital Library

Google Scholar

Cited By

View all

Rocco GSilvello G(2020)An Information Visualization Tool for the Interactive Component-Based Evaluation of Search EnginesDigital Libraries: The Era of Big Data and Data Science10.1007/978-3-030-39905-4_3(15-25)Online publication date: 22-Jan-2020
https://doi.org/10.1007/978-3-030-39905-4_3
Fuhr N(2019)Reproducibility and Validity in CLEFInformation Retrieval Evaluation in a Changing World10.1007/978-3-030-22948-1_23(555-564)Online publication date: 14-Aug-2019
https://doi.org/10.1007/978-3-030-22948-1_23
Silvello GBordea GFerro NBuitelaar PBogers T(2017)Semantic representation and enrichment of information retrieval experimental dataInternational Journal on Digital Libraries10.1007/s00799-016-0172-818:2(145-172)Online publication date: 1-Jun-2017
https://dl.acm.org/doi/10.1007/s00799-016-0172-8
Show More Cited By

Index Terms

A Visual Analytics Approach for What-If Analysis of Information Retrieval Systems
1. Information systems
  1. Information retrieval
    1. Evaluation of retrieval results
      1. Retrieval effectiveness

Recommendations

Effort-based information retrieval evaluation with varied evaluation depth and topic sizes
ICBIM '19: Proceedings of the 3rd International Conference on Business and Information Management

The information retrieval accessed globally is a vital productivity boost for most organization. However, the outcome of information retrieval system evaluation does not agree with the real user's satisfaction. Information retrieval systems retrieving ...
Pooling-based continuous evaluation of information retrieval systems
Abstract
The dominant approach to evaluate the effectiveness of information retrieval (IR) systems is by means of reusable test collections built following the Cranfield paradigm. In this paper, we propose a new IR evaluation methodology based on pooled ...
Visual information retrieval environments

Comments

Information & Contributors

Information

Published In

SIGIR '16: Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval

July 2016

1296 pages

ISBN:9781450340694

DOI:10.1145/2911451

General Chairs:
Raffaele Perego
ISTI-CNR, Italy
,
Fabrizio Sebastiani
Qatar Computing Research Institute, HBKU, Qatar
,
Program Chairs:
Javed Aslam
Northeastern University, US
,
Ian Ruthven
University of Strathclyde, UK
,
Justin Zobel
University of Melbourne, Australia

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 July 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SIGIR '16

Sponsor:

SIGIR

SIGIR '16: The 39th International ACM SIGIR conference on research and development in Information Retrieval

July 17 - 21, 2016

Pisa, Italy

Acceptance Rates

SIGIR '16 Paper Acceptance Rate 62 of 341 submissions, 18%;

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
191
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 01 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Rocco GSilvello G(2020)An Information Visualization Tool for the Interactive Component-Based Evaluation of Search EnginesDigital Libraries: The Era of Big Data and Data Science10.1007/978-3-030-39905-4_3(15-25)Online publication date: 22-Jan-2020
https://doi.org/10.1007/978-3-030-39905-4_3
Fuhr N(2019)Reproducibility and Validity in CLEFInformation Retrieval Evaluation in a Changing World10.1007/978-3-030-22948-1_23(555-564)Online publication date: 14-Aug-2019
https://doi.org/10.1007/978-3-030-22948-1_23
Silvello GBordea GFerro NBuitelaar PBogers T(2017)Semantic representation and enrichment of information retrieval experimental dataInternational Journal on Digital Libraries10.1007/s00799-016-0172-818:2(145-172)Online publication date: 1-Jun-2017
https://dl.acm.org/doi/10.1007/s00799-016-0172-8
Carvalho MBelo O(2017)Conceiving Hybrid What-If Scenarios Based on Usage PreferencesDecision Support Systems VII. Data, Information and Knowledge Visualization in Decision Support Systems10.1007/978-3-319-57487-5_9(119-132)Online publication date: 21-Apr-2017
https://doi.org/10.1007/978-3-319-57487-5_9

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Effort-based information retrieval evaluation with varied evaluation depth and topic sizes

Pooling-based continuous evaluation of information retrieval systems

Visual information retrieval environments

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations