column

Report on the SIGIR 2017 Workshop on Axiomatic Thinking for Information Retrieval and Related Tasks (ATIR)

Authors:

Enrique Amigó,

Hui Fang,

Stefano Mizzaro,

ChengXiang ZhaiAuthors Info & Claims

ACM SIGIR Forum, Volume 51, Issue 3

Pages 99 - 106

https://doi.org/10.1145/3190580.3190596

Published: 22 February 2018 Publication History

Get Access

Abstract

The SIGIR 2017 workshop on Axiomatic Thinking for Information Retrieval and Related Tasks took place on August 11, 2017 in Tokyo, Japan. The workshop aimed to help foster collaboration of researchers working on different perspectives of axiomatic thinking and encourage discussion and research on general methodological issues related to applying axiomatic thinking to information retrieval and related tasks. The program consisted of one keynote talk, four research presentations and a final panel discussion. This report outlines the events of the workshop and summarizes the major outcomes. More information about the workshop is available at https://www.eecis.udel.edu/~hfang/ATIR.html.

References

[1]

E. Amigó, J. Gonzalo, J. Artiles, and F. Verdejo. A comparison of extrinsic clustering evaluation metrics based on formal constraints. Information Retrieval, 12(4):461--486, 2009.

Digital Library

Google Scholar

[2]

E. Amigó, J. Gonzalo, and F. Verdejo. A general evaluation measure for document organization tasks. In Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '13, pages 643--652, New York, NY, USA, 2013. ACM.

Digital Library

Google Scholar

[3]

P. D. Bruza and T. W. C. Huibers. Investigating aboutness axioms using information fields. In Proceedings of the 17th ACM SIGIR, pages 112--121, New York, NY, USA, 1994. Springer-Verlag New York, Inc. ISBN 0-387-19889-X. URL http://dl.acm.org/ citation.cfm?id=188490.188521.

Digital Library

Google Scholar

[4]

L. Busin and S. Mizzaro. Axiometrics: An axiomatic approach to information retrieval effectiveness metrics. In Proceedings of ICTIR 2013: 4th International Conference on the Theory of Information Retrieval, pages 22--29, New York -- USA, Oct. 2013. ACM.

Digital Library

Google Scholar

[5]

H. Fang and C. Zhai. An exploration of axiomatic approaches to information retrieval. In SIGIR '05, pages 480--487, 2005. ISBN 1-59593-034-5.

Digital Library

Google Scholar

[6]

H. Fang, T. Tao, and C. Zhai. Diagnostic evaluation of information retrieval models. ACM Trans. Inf. Syst., 29(2):7:1--7:42, Apr. 2011. ISSN 1046-8188.

Digital Library

Google Scholar

[7]

M. Ferrante, N. Ferro, and M. Maistro. Towards a formal framework for utility-oriented measurements of retrieval effectiveness. In Proceedings of ICTIR 2015, pages 21--30, New York, NY, USA, 2015. ACM. ISBN 978-1-4503-3833-2.

Digital Library

Google Scholar

[8]

C. Ferri, J. Hernández-Orallo, and R. Modroiu. An experimental comparison of performance measures for classification. Pattern Recognition Letters, 30(1):27--38, 2009.

Digital Library

Google Scholar

[9]

Y. Lv and C. Zhai. Lower-bounding term frequency normalization. In Proceedings of the 20th ACM International Conference on Information and Knowledge Management, CIKM '11, 2011.

Digital Library

Google Scholar

[10]

E.Maddalena and S.Mizzaro. Axiometrics: Axioms of information retrieval effectiveness metrics. In Proceedings of the Sixth EVIA Workshop, pages 17--24, Tokyo, Japan, 2014. National Institute of Informatics. ISBN: 978-4-86049-066-9.

Google Scholar

[11]

M. Meila. Comparing clusterings. In Proceedings of COLT 03, 2003. URL http://www.stat.washington.edu/mmp/www.stat.washington.edu/mmp/Papers/compare-colt.pdf.

Google Scholar

[12]

A. Moffat. Seven numeric properties of effectiveness metrics. In AIRS'13, pages 1--12, 2013.

Crossref

Google Scholar

[13]

F. Sebastiani. An axiomatically derived measure for the evaluation of classification algorithms. In Proceedings of ICTIR 2015, pages 11--20. ACM, 2015. ISBN 978--1--4503- 3833--2.

Digital Library

Google Scholar

[14]

M. Sokolova. Assessing invariance properties of evaluation measures. In Proceedings of NIPS'06 Workshop on Testing Deployable Learning and Decision Systems, 2006.

Google Scholar

Cited By

View all

Ferrante MFerro NLosiouk E(2020)How do interval scales help us with better understanding IR evaluation measures?Information Retrieval10.1007/s10791-019-09362-z23:3(289-317)Online publication date: 1-Jun-2020
https://dl.acm.org/doi/10.1007/s10791-019-09362-z
van Dijk DFerrante MFerro NKanoulas E(2019)A Markovian Approach to Evaluate Session-Based IR SystemsAdvances in Information Retrieval10.1007/978-3-030-15712-8_40(621-635)Online publication date: 14-Apr-2019
https://dl.acm.org/doi/10.1007/978-3-030-15712-8_40
Gudivada VRao DGudivada A(2018)Information Retrieval: Concepts, Models, and SystemsComputational Analysis and Understanding of Natural Languages: Principles, Methods and Applications10.1016/bs.host.2018.07.009(331-401)Online publication date: 2018
https://doi.org/10.1016/bs.host.2018.07.009

Recommendations

Report on the SIGIR 2017 Workshop on eCommerce (ECOM17)

The SIGIR 2017 Workshop on eCommerce (ECOM17), was a full day workshop that took place on Friday, August 11, 2017 in Tokyo, Japan. The purpose of the workshop was to serve as a platform for publication and discussion of Information Retrieval and NLP ...
Report on the SIGIR 2016 Workshop on Neural Information Retrieval (Neu-IR)

The SIGIR 2016 workshop on Neural Information Retrieval (Neu-IR) took place on 21 July, 2016 in Pisa. The goal of the Neu-IR (pronounced "New IR") workshop was to serve as a forum for academic and industrial researchers, working at the intersection of ...
Report on the Second SIGIR Workshop on Neural Information Retrieval (Neu-IR'17)

The second SIGIR workshop on neural information retrieval (Neu-IR?17) took place on August 11, 2017, in Tokyo, Japan. Following the successful 2016 edition, the workshop continued to serve as a forum for academic and industrial researchers to present new ...

Comments

Information & Contributors

Information

Published In

ACM SIGIR Forum Volume 51, Issue 3

December 2017

157 pages

ISSN:0163-5840

DOI:10.1145/3190580

Editors:
Claudia Hauff
Delft University of Technology, The Netherlands
,
Craig Macdonald
University of Glasgow, Glasgow, United Kingdom

Issue’s Table of Contents

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 February 2018

Published in SIGIR Volume 51, Issue 3

Check for updates

Qualifiers

Column

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
53
Total Downloads

Downloads (Last 12 months)1
Downloads (Last 6 weeks)1

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Ferrante MFerro NLosiouk E(2020)How do interval scales help us with better understanding IR evaluation measures?Information Retrieval10.1007/s10791-019-09362-z23:3(289-317)Online publication date: 1-Jun-2020
https://dl.acm.org/doi/10.1007/s10791-019-09362-z
van Dijk DFerrante MFerro NKanoulas E(2019)A Markovian Approach to Evaluate Session-Based IR SystemsAdvances in Information Retrieval10.1007/978-3-030-15712-8_40(621-635)Online publication date: 14-Apr-2019
https://dl.acm.org/doi/10.1007/978-3-030-15712-8_40
Gudivada VRao DGudivada A(2018)Information Retrieval: Concepts, Models, and SystemsComputational Analysis and Understanding of Natural Languages: Principles, Methods and Applications10.1016/bs.host.2018.07.009(331-401)Online publication date: 2018
https://doi.org/10.1016/bs.host.2018.07.009

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Recommendations

Report on the SIGIR 2017 Workshop on eCommerce (ECOM17)

Report on the SIGIR 2016 Workshop on Neural Information Retrieval (Neu-IR)

Report on the Second SIGIR Workshop on Neural Information Retrieval (Neu-IR'17)

Comments

Information

Published In

Publisher

Publication History

Check for updates

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations