column

Report on the SIGIR 2015 Workshop on Reproducibility, Inexplicability, and Generalizability of Results (RIGOR)

Authors:

Andrew TrotmanAuthors Info & Claims

ACM SIGIR Forum, Volume 49, Issue 2

Pages 107 - 116

https://doi.org/10.1145/2888422.2888439

Published: 29 January 2016 Publication History

Get Access

Abstract

The SIGIR 2015 Workshop on Reproducibility, Inexplicability, and Generalizability of Results (RIGOR) took place on Thursday, August 13, 2015 in Santiago, Chile. The goal of the workshop was two fold. The first to provide a venue for the publication and presentation of negative results. The second was to provide a venue through which the authors of open source search engines could compare performance of indexing and searching on the same collections and on the same machines - encouraging the sharing of ideas and discoveries in a like-to-like environment. In total three papers were presented and seven systems participated.

References

[1]

T. Armstrong, A. Moffat, W. Webber, and J. Zobel. Improvements that don't add up: Ad-hoc retrieval results since 1998. In CIKM 2009, pages 601--610, 2009.

Digital Library

Google Scholar

[2]

E. Di Buccio, G.M. Di Nunzio, N. Ferro, D. Harman, M. Maistro, and G. Silvello. Unfolding off-the-shelf IR systems for reproducibility. In RIGOR 2015, 2015.

Google Scholar

[3]

B. Carterette and K. Sabhnani. Using simulation to analyze the potential for reproducibility. In RIGOR 2015, 2015.

Google Scholar

[4]

X. Lu, A. Moffat, and S. Culpepper. Observed volatility in effectiveness metrics. In RIGOR 2015, 2015.

Google Scholar

[5]

H. Mhleisen, T. Samar, J. Lin, and A. de Vries. Old dogs are great at new tricks: Column stores for ir prototyping. In SIGIR 2014, pages 863--866, 2014.

Digital Library

Google Scholar

[6]

A. Trotman, A. Puurula, and B. Burgess. Improvements to BM25 and language models examined. In ADCS 2014, pages 58--65, 2014.

Digital Library

Google Scholar

Cited By

View all

Farzana SFröbe MGranitzer MHendriksen GHiemstra DPotthast Mde Vries AZerhoudi S(2024)Report on the 1st International Workshop on Open Web Search (WOWS 2024) at ECIR 2024ACM SIGIR Forum10.1145/3687273.368729058:1(1-13)Online publication date: 7-Aug-2024
https://dl.acm.org/doi/10.1145/3687273.3687290
Farzana SFröbe MGranitzer MHendriksen GHiemstra DPotthast MZerhoudi S(2024)The First International Workshop on Open Web Search (WOWS)Advances in Information Retrieval10.1007/978-3-031-56069-9_58(426-431)Online publication date: 23-Mar-2024
https://doi.org/10.1007/978-3-031-56069-9_58
Göksel GArslan ADinçer B(2023)A selective approach to stemming for minimizing the risk of failure in information retrieval systemsPeerJ Computer Science10.7717/peerj-cs.11759(e1175)Online publication date: 10-Jan-2023
https://doi.org/10.7717/peerj-cs.1175
Show More Cited By

Recommendations

Report on the SIGIR 2017 Workshop on eCommerce (ECOM17)

The SIGIR 2017 Workshop on eCommerce (ECOM17), was a full day workshop that took place on Friday, August 11, 2017 in Tokyo, Japan. The purpose of the workshop was to serve as a platform for publication and discussion of Information Retrieval and NLP ...
IWGS 2015 workshop report: the 6th ACM SIGSPATIAL International Workshop on GeoStreaming (Seattle, WA, USA - November 3, 2015)

The ACM SIGSPATIAL International Workshop on Geostreaming (IWGS) was held for the sixth time in conjunction with the 23rd ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems (ACMGIS 2015). The workshop has been a ...
The SIGIR 2005 workshop program

The SIGIR 2005 workshop program included seven workshops that were competitively selected from among eight proposals. Organizers of SIGIR 2004 workshops had reported that tight timelines for publicizing workshops and reviewing submissions had resulted ...

Comments

Information & Contributors

Information

Published In

ACM SIGIR Forum Volume 49, Issue 2

December 2015

141 pages

ISSN:0163-5840

DOI:10.1145/2888422

Editors:
Ben Carterette
University of Delaware, Newark, DE, USA
,
Craig Macdonald
University of Glasgow, Glasgow, UK

Issue’s Table of Contents

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 January 2016

Published in SIGIR Volume 49, Issue 2

Check for updates

Qualifiers

Column

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

54
Total Citations
View Citations
178
Total Downloads

Downloads (Last 12 months)13
Downloads (Last 6 weeks)0

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Farzana SFröbe MGranitzer MHendriksen GHiemstra DPotthast Mde Vries AZerhoudi S(2024)Report on the 1st International Workshop on Open Web Search (WOWS 2024) at ECIR 2024ACM SIGIR Forum10.1145/3687273.368729058:1(1-13)Online publication date: 7-Aug-2024
https://dl.acm.org/doi/10.1145/3687273.3687290
Farzana SFröbe MGranitzer MHendriksen GHiemstra DPotthast MZerhoudi S(2024)The First International Workshop on Open Web Search (WOWS)Advances in Information Retrieval10.1007/978-3-031-56069-9_58(426-431)Online publication date: 23-Mar-2024
https://doi.org/10.1007/978-3-031-56069-9_58
Göksel GArslan ADinçer B(2023)A selective approach to stemming for minimizing the risk of failure in information retrieval systemsPeerJ Computer Science10.7717/peerj-cs.11759(e1175)Online publication date: 10-Jan-2023
https://doi.org/10.7717/peerj-cs.1175
Baskin ANixon BChrysanthis PLaoudias CCosta C(2023)RETSINA: Reproducibility and Experimentation Testbed for Signal-Strength Indoor Near Analysis2023 13th International Conference on Indoor Positioning and Indoor Navigation (IPIN)10.1109/IPIN57070.2023.10332500(1-6)Online publication date: 25-Sep-2023
https://doi.org/10.1109/IPIN57070.2023.10332500
Maistro MBreuer TSchaer PFerro N(2023)An in-depth investigation on the behavior of measures to quantify reproducibilityInformation Processing & Management10.1016/j.ipm.2023.10333260:3(103332)Online publication date: May-2023
https://doi.org/10.1016/j.ipm.2023.103332
Yılmazel İArslan A(2021)An intrinsic evaluation of the Waterloo spam rankings of the ClueWeb09 and ClueWeb12 datasetsJournal of Information Science10.1177/016555151986655147:1(41-57)Online publication date: 25-Feb-2021
https://dl.acm.org/doi/10.1177/0165551519866551
Lin JMa XLin SYang JPradeep RNogueira RDiaz FShah CSuel TCastells PJones RSakai T(2021)Pyserini: A Python Toolkit for Reproducible Information Retrieval Research with Sparse and Dense RepresentationsProceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3404835.3463238(2356-2362)Online publication date: 11-Jul-2021
https://dl.acm.org/doi/10.1145/3404835.3463238
Bellogín ASaid A(2021)Improving accountability in recommender systems research through reproducibilityUser Modeling and User-Adapted Interaction10.1007/s11257-021-09302-x31:5(941-977)Online publication date: 1-Nov-2021
https://dl.acm.org/doi/10.1007/s11257-021-09302-x
Sequiera RTan LZhang YLin JO'Brien HFreund LArapakis IHoeber OLopatovska I(2020)Update Delivery Mechanisms for Prospective Information NeedsProceedings of the 2020 Conference on Human Information Interaction and Retrieval10.1145/3343413.3377988(308-312)Online publication date: 14-Mar-2020
https://dl.acm.org/doi/10.1145/3343413.3377988
Marchesin SPurpura ASilvello G(2020)Focal elements of neural information retrieval models. An outlook through a reproducibility studyInformation Processing & Management10.1016/j.ipm.2019.10210957:6(102109)Online publication date: Nov-2020
https://doi.org/10.1016/j.ipm.2019.102109
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Recommendations

Report on the SIGIR 2017 Workshop on eCommerce (ECOM17)

IWGS 2015 workshop report: the 6th ACM SIGSPATIAL International Workshop on GeoStreaming (Seattle, WA, USA - November 3, 2015)

The SIGIR 2005 workshop program

Comments

Information

Published In

Publisher

Publication History

Check for updates

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations