K. Selçuk Candan
Nicola Ferro
Bogdan Ionescu
Birger Larsen
Skip Abstract Section
Abstract
This is a report on the eleventh edition of the Conference and Labs of the Evaluation Forum (CLEF 2021), (virtually) held on September 21--24, 2021, in Bucharest, Romania. CLEF was a four day event combining a Conference and an Evaluation Forum. The Conference featured keynotes by Naila Murray and Mark Sanderson, and presentation of peer reviewed research papers covering a wide range of topics in addition to many posters. The Evaluation Forum consisted to thirteen Labs: ARQMath, BioASQ, CheckThat!, ChEMU, CLEF eHealth, eRisk, HIPE, ImageCLEF, LifeCLEF, LiLAS, PAN, SimpleText, and Touché, addressing a wide range of tasks, media, languages, and ways to go beyond standard test collections.
Date: 21--24 September, 2021.
Website: http://clef2021.clef-initiative.eu/.
- J. Bevendorff, B. Chulvi, G. L. De La Penã Sarracén, M. Kestemont, E. Manjavacas, I. Markov, M. Mayerl, M. Potthast, F. Rangel, P. Rosso, E. Stamatatos, B. Stein, M. Wiegmann, M. Wolska, and E. Zangerle. Overview of PAN 2021: Authorship Verification, Profiling Hate Speech Spreaders on Twitter, and Style Change Detection. In Candan et al. [2021], pages 419--431.Google Scholar
- A. Bondarenko, L. Gienapp, M. Fröbe, M. Beloucif, Y. Ajjour, A. Panchenko, C. Biemann, B. Stein, J. Wachsmuth, M. Potthast, and M. Hagen. Overview of Touché 2021: Argument Retrieval. In Candan et al. [2021], pages 450--467.Google Scholar
- K. S. Candan, B. Ionescu, L. Goeuriot, B. Larsen, H. Müller, A. Joly, M. Maistro, F. Piroi, G. Faggioli, and N. Ferro, editors. Experimental IR Meets Multilinguality, Multimodality, and Interaction. Proceedings of the Twelfth International Conference of the CLEF Association (CLEF 2021), 2021. Lecture Notes in Computer Science (LNCS) 12880, Springer, Heidelberg, Germany.Google Scholar
- L. Ermakova, P. Bellot, P. Braslavski, J. Kamps, J. Mothe, D. Nurbakova, I. Ovchinnikova, and E. SanJuan. Overview of SimpleText 2021 - CLEF Workshop on Text Simplification for Scientific Information Access. In Candan et al. [2021], pages 432--449.Google Scholar
- G. Faggioli, N. Ferro, A. Joly, M. Maistro, and F. Piroi, editors. CLEF 2021 Working Notes, 2021. CEUR Workshop Proceedings (CEUR-WS.org), ISSN 1613-0073, http://ceur-ws.org/Vol-2936/.Google Scholar
- N. Ferro. What Happened in CLEF... For a While? In F. Crestani, M. Braschler, J. Savoy, A. Rauber, H. Müller, D. E. Losada, G. Heinatz Bürki, L. Cappellato, and N. Ferro, editors, Experimental IR Meets Multilinguality, Multimodality, and Interaction. Proceedings of the Tenth International Conference of the CLEF Association (CLEF 2019), pages 3--45. Lecture Notes in Computer Science (LNCS) 11696, Springer, Heidelberg, Germany, 2019.Google Scholar
- N. Ferro and C. Peters, editors. Information Retrieval Evaluation in a Changing World - Lessons Learned from 20 Years of CLEF, volume 41 of The Information Retrieval Series, 2019. Springer International Publishing, Germany.Google ScholarCross Ref
- N. Ferro and G. Silvello. 3.5K runs, 5K topics, 3M assessments and 70M measures: What trends in 10 years of Adhoc-ish CLEF? Information Processing & Management, 53(1):175--202, January 2017.Google ScholarCross Ref
- B. Ionescu, H. Müller, R. Péteri, A. B. Abacha, M. Sarrouti, D. Demner-Fushman, S. A. Hasan, S. Kozlovski, V. Liauchuk, Y. D. Cid, V. Kovalev, O. Pelka, A. García Seco de Herrera, J. Jacutprakart, C. M. Friedrich, R. Berari, A. Tauteanu, D. Fichou, P. Brie, M. Dogariu, L. D. Ştefan, M. G. Constantin, J. Chamberlain, A. Campello, A. Clark, T. A. Oliver, H. Moustahfid, A. Popescu, and J. Deshayes-Chossart. Overview of the ImageCLEF 2021: Multimedia Retrieval in Medical, Nature, Internet and Social Media Applications. In Candan et al. [2021], pages 345--370.Google Scholar
- A. Joly, H. Goëau, S. Kahl, L. Picek, T. Lorieul, E. Cole, B. Deneu, M. Servajean, A. Durso, I. Bolon, H. Glotin, R. Planqué, R. L. Ruiz de Castañeda, W.-P. Vellinga, H. Klinck, T. Denton, I. Eggel, P. Bonnet, and H. Müller. Overview of LifeCLEF 2021: An Evaluation of Machine-Learning Based Species Identification and Species Distribution Prediction. In Candan et al. [2021], pages 371--393.Google Scholar
- B. Larsen. The Scholarly Impact of CLEF 2010--2017. In Ferro and Peters [2019], pages 547--554.Google ScholarCross Ref
- Y. Li, B. Fang, J. He, H. Yoshikawa, S. A. Akhondi, C. Druckenbrodt, C. Thorne, Z. Afzal, Z. Zhai, T. Baldwin, and K. Verspoor. Overview of ChEMU 2021: Reaction Reference Resolution and Anaphora Resolution in Chemical Patents. In Candan et al. [2021], pages 292--307.Google Scholar
- B. Mansouri, R. Zanibbi, D. W. Oard, and A. Agarwal. Overview of ARQMath-2 (2021): Second CLEF Lab on Answer Retrieval for Questions on Math. In Candan et al. [2021], pages 215--238.Google Scholar
- P. Nakov, G. Da San Martino, T. Elsayed, A. Barrón-Cedeño, R. Míguez, S. Shaar, F. Alam, F. Haouari, M. Hasanain, W. Mansour, B. Hamdan, Z. S. Ali, N. Babulkov, A. Nikolov, G. K. Shahi, J. M. Struß, T. Mandl, M. Kutlu, and Y. S. Kartal. Overview of the CLEF-2021 CheckThat! Lab on Detecting Check-Worthy Claims, Previously Fact-Checked Claims, and Fake News. In Candan et al. [2021], pages 264--291.Google Scholar
- A. Nentidis, G. Katsimpras, E. Vandorou, A. Krithara, L. Gasco, M. Krallinger, and G. Paliouras. Overview of BioASQ 2021: The Ninth BioASQ Challenge on Large-Scale Biomedical Semantic Indexing and Question Answering. In Candan et al. [2021], pages 239--263.Google Scholar
- J. Parapar, P. Martín-Rodilla, D. E. Losada, and F. Crestani. Overview of eRisk 2021: Early Risk Prediction on the Internet. In Candan et al. [2021], pages 324--344.Google Scholar
- P. Schaer, T. Breuer, L. J. Castro, B. Wolff, J. Schaible, and N. Tavakolpoursaleh. Overview of LiLAS 2021 - Living Labs for Academic Search. In Candan et al. [2021], pages 394--418.Google Scholar
- H. Suominen, L. Goeuriot, L. Kelly, L. Alonso Alemany, E. Bassani, N. Brew-Sam, V. Cotik, D. Filippo, G. González-Sáez, F. Luque, P. Mulhem, G. Pasi, R. Roller, S. Seneviratne, R. Upadhyay, J. Vivaldi, M. Viviani, and C. Xu. Overview of the CLEF eHealth Evaluation Lab 2021. In Candan et al. [2021], pages 308--323.Google Scholar
- T. Tsikrika, A. Garcia Seco de Herrera, and H. Müller. Assessing the Scholarly Impact of Image-CLEF. In P. Forner, J. Gonzalo, J. Kekäläinen, M. Lalmas, and M. de Rijke, editors, Multilingual and Multimodal Information Access Evaluation. Proceedings of the Second International Conference of the Cross-Language Evaluation Forum (CLEF 2011), pages 95--106. Lecture Notes in Computer Science (LNCS) 6941, Springer, Heidelberg, Germany, 2011.Google Scholar
- T. Tsikrika, B. Larsen, H. Müller, S. Endrullis, and E. Rahm. The Scholarly Impact of CLEF (2000--2009). In P. Forner, H. Müller, R. Paredes, P. Rosso, and B. Stein, editors, Information Access Evaluation meets Multilinguality, Multimodality, and Visualization. Proceedings of the Fourth International Conference of the CLEF Initiative (CLEF 2013), pages 1--12. Lecture Notes in Computer Science (LNCS) 8138, Springer, Heidelberg, Germany, 2013.Google Scholar
Index Terms
- Report on the 12th conference and labs of the evaluation forum (CLEF 2021): experimental IR meets multilinguality, multimodality, and interaction
Index terms have been assigned to the content through auto-classification.
Recommendations
ACM SIGSPATIAL 2016 conference report: the 24th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems (ACM SIGSPATIAL 2016): San Francisco Bay Area, California, USA October 31--November 36, 2016
This is the conference report on the 24th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems (ACM SIGSPATIAL 2016), held in the San Francisco Bay Area, California, USA, October 31 through November 3, 2016. The ...
Comments