skip to main content
research-article

Report on CLEF 2019

Published: 23 March 2021 Publication History

Abstract

This is a report on the tenth edition of the Conference and Labs of the Evaluation Forum (CLEF 2019), held from September 9--12, 2019, in Lugano, Switzerland.
CLEF was a four day event combining a Conference and an Evaluation Forum. The Conference featured keynotes by Bruce Croft, Yair Neuman, and Miguel Martínez, and presentation of peer reviewed research papers covering a wide range of topics in addition to many posters. The Evaluation Forum consisted to nine Labs: CENTRE, CheckThat, eHealth, eRisk, ImageCLEF, LifeCLEF, PAN, PIR-CLEF, and ProtestNews, addressing a wide range of tasks, media, languages, and ways to go beyond standard test collections.
CLEF 2019 marked the 20th anniversary of CLEF, which was celebrated with a dedicated session and a book on the lessons learnt in twenty years of evaluation activities and the future perspectives for CLEF.
CLEF 2019 also introduced the Industry Days to further extend the reach and impact of CLEF.

References

[1]
L. Cappellato, N. Ferro, D. E. Losada, and H. Müller, editors. CLEF 2019 Working Notes. CEUR Workshop Proceedings (CEUR-WS.org), ISSN 1613-0073, http://ceur-ws.org/Vol-2380/, 2019.
[2]
F. Crestani, M. Braschler, J. Savoy, A. Rauber, H. Müller, D. E. Losada, G. Heinatz Bürki, L. Cappellato, and N. Ferro, editors. Experimental IR Meets Multilinguality, Multimodality, and Interaction. Proceedings of the Tenth International Conference of the CLEF Association (CLEF 2019). Lecture Notes in Computer Science (LNCS) 11696, Springer, Heidelberg, Germany, 2019.
[3]
W. Daelemans, M. Kestemont, E. Manjavacas, M. Potthast, F. M. Rangel Pardo, P. Rosso, G. Specht, E. Stamatatos, B. Stein, M. Tschuggnall, M. Wiegmann, and E. Zangerle. Overview of PAN 2019: Bots and Gender Profiling, Celebrity Profiling, Cross-domain Authorship Attribution and Style Change Detection. In Crestani et al. [2], pages 402--416.
[4]
T. Elsayed, P. Nakov, A. Barrón-Cedeño, M. Hasanain, R. Suwaileh, G. Da San Martino, and P. Atanasova. Overview of the CLEF-2019 CheckThat!: Automatic Identification and Verification of Claims. In Crestani et al. [2], pages 301--321.
[5]
N. Ferro. What Happened in CLEF … For a While? In Crestani et al. [2], pages 3--45.
[6]
N. Ferro, N. Fuhr, M. Maistro, T. Sakai, and I. Soboroff. Overview of CENTRE@CLEF 2019: Sequel in the Systematic Reproducibility Realm. In Crestani et al. [2], pages 287--300.
[7]
N. Ferro and C. Peters, editors. Information Retrieval Evaluation in a Changing World - Lessons Learned from 20 Years of CLEF, volume 41 of The Information Retrieval Series. Springer International Publishing, Germany, 2019.
[8]
N. Ferro and G. Silvello. 3.5K runs, 5K topics, 3M assessments and 70M measures: What trends in 10 years of Adhoc-ish CLEF? Information Processing & Management, 53(1):175--202, January 2017.
[9]
A. Hürriyetoğlu, E. Yörük, D. Yuret, Ç. Yoltar, B. Gürel, F. Duruşan, O. Mutlu, and A. Akdemir. Overview of CLEF 2019 Lab ProtestNews: Extracting Protests from News in a Cross-context Setting. In Crestani et al. [2], pages 425--432.
[10]
B. Ionescu, H. Müller, R. Péteri, Y. Dicente Cid, V. Liauchuk, V. Kovalev, D. Klimuk, A. Tarasau, A. Ben Abacha, S. A. Hasan, V. Datla, J. Liu, D. Demner-Fushman, D.-T. Dang-Nguyen, L. Piras, M. Riegler, M.-T. Tran, M. Lux, C. Gurrin, O. Pelka, C. M. Friedrich, A. Garcia Seco de Herrera, N. Garcia, E. Kavallieratou, C. R. del Blanco, C. Cuevas Rodríguez, N. Vasillopoulos, K. Karampidis, J. Chamberlain, A. Clark, and A. Campello. ImageCLEF 2019: Multimedia Retrieval in Medicine, Lifelogging, Security and Nature. In Crestani et al. [2], pages 358--386.
[11]
A. Joly, H. Goëau, C. Botella, S. Kahl, M. Servajean, H. Glotin, P. Bonnet, W.-P. Vellinga, R. Planqué, F.-R. Stöter, and H. Müller. Overview of LifeCLEF 2019: Identification of Amazonian Plants, South & North American Birds, and Niche Prediction. In Crestani et al. [2], pages 387--401.
[12]
L. Kelly, H. Suominen, L. Goeuriot, M. Neves, E. Kanoulas, D. Li, L. Azzopardi, R. Spijker, G. Zuccon, H. Scells, and J. Palotti. Overview of the CLEF eHealth Evaluation Lab 2019. In Crestani et al. [2], pages 322--339.
[13]
B. Larsen. The Scholarly Impact of CLEF 2010-2017. In Ferro and Peters [7], pages 547--554.
[14]
D. E. Losada, F. Crestani, and J. Parapar. Overview of eRisk 2019: Early Risk Prediction on the Internet. In Crestani et al. [2], pages 340--357.
[15]
G. Pasi, G. J. F. Jones, L. Goeuriot, L. Kelly, S. Marrara, and C. Sanvitto. Overview of the CLEF 2019 Personalised Information Retrieval Lab (PIR-CLEF 2019). In Crestani et al. [2], pages 417--424.
[16]
T. Tsikrika, A. Garcia Seco de Herrera, and H. Müller. Assessing the Scholarly Impact of ImageCLEF. In P. Forner, J. Gonzalo, J. Kekäläinen, M. Lalmas, and M. de Rijke, editors, Multilingual and Multimodal Information Access Evaluation. Proceedings of the Second International Conference of the Cross-Language Evaluation Forum (CLEF 2011), pages 95--106. Lecture Notes in Computer Science (LNCS) 6941, Springer, Heidelberg, Germany, 2011.
[17]
T. Tsikrika, B. Larsen, H. Müller, S. Endrullis, and E. Rahm. The Scholarly Impact of CLEF (2000--2009). In P. Forner, H. Müller, R. Paredes, P. Rosso, and B. Stein, editors, Information Access Evaluation meets Multilinguality, Multimodality, and Visualization. Proceedings of the Fourth International Conference of the CLEF Initiative (CLEF 2013), pages 1--12. Lecture Notes in Computer Science (LNCS) 8138, Springer, Heidelberg, Germany, 2013.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM SIGIR Forum
ACM SIGIR Forum  Volume 53, Issue 2
December 2019
125 pages
ISSN:0163-5840
DOI:10.1145/3458553
Issue’s Table of Contents
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 March 2021
Published in SIGIR Volume 53, Issue 2

Check for updates

Qualifiers

  • Research-article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 28
    Total Downloads
  • Downloads (Last 12 months)5
  • Downloads (Last 6 weeks)0
Reflects downloads up to 15 Feb 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media