Skip to main content
Springer Nature Link
Log in

CLEF 2004: Ad Hoc Track Overview and Results Analysis

  • Conference paper
Multilingual Information Access for Text, Speech and Images (CLEF 2004)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 3491))

Included in the following conference series:

  • 647 Accesses

Abstract

We describe the objectives and organization of the CLEF 2004 ad hoc track and discuss the main characteristics of the experiments. The results are analyzed and commented and their statistical significance is investigated. The paper concludes with some observations on the impact of the CLEF campaign on the state-of-the-art in cross-language information retrieval.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Peters, C.: What happened in CLEF 2004? In: Peters, C., Clough, P., Gonzalo, J., Jones, G.J.F., Kluck, M., Magnini, B. (eds.) CLEF 2004. LNCS, vol. 3491, pp. 1–9. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  2. Cleverdon, C.: The Cranfield Tests on Index Language Devices. In: Sparck-Jones, K., Willett, P. (eds.) Readings in Information Retrieval, pp. 47–59. Morgan Kaufmann, San Francisco (1997)

    Google Scholar 

  3. Harman, D.: The TREC Conferences. In: Kuhlen, R., Rittberger, M. (eds.) Hypertext - Information Retrieval - Multimedia: Synergieeffekte Elektronischer Informationssysteme, Proceedings of HIM 1995, Universitätsverlag Konstanz, pp. 9–28 (1995)

    Google Scholar 

  4. CLEF 2004 Working Notes, http://clef.isti.cnr.it/2004/working_notes/CLEF2004WN

  5. Braschler, M.: CLEF 2003 – Overview of results. In: Peters, C., Gonzalo, J., Braschler, M., Kluck, M. (eds.) CLEF 2003. LNCS, vol. 3237, pp. 44–63. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  6. Braschler, M., Peters, C.: CLEF 2003 Methodology and Metrics. In: Peters, C., Gonzalo, J., Braschler, M., Kluck, M. (eds.) CLEF 2003. LNCS, vol. 3237, pp. 7–20. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  7. Besançon, R., Ferre, O., Fluhr, C.: LIC2M Experiments at CLEF 2004. In: Peters, C., Clough, P., Gonzalo, J., Jones, G.J.F., Kluck, M., Magnini, B. (eds.) CLEF 2004. LNCS, vol. 3491. Springer, Heidelberg (2005)

    Google Scholar 

  8. Martínez-Santiago, F., García-Cumbreras, M.A., Díaz-Galiano, M.C., Ureña, L.A.: SINAI at CLEF 2004: Using Machine Translation Resources with a Mixed 2-Step RSV Merging Algorithm. In: Peters, C., Clough, P., Gonzalo, J., Jones, G.J.F., Kluck, M., Magnini, B. (eds.) CLEF 2004. LNCS, vol. 3491, pp. 156–164. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  9. Levow, G.-A., Matveeva, I.: University of Chicago at CLEF2004: Cross-language Text and Spoken Document Retrieval. In: Peters, C., Clough, P., Gonzalo, J., Jones, G.J.F., Kluck, M., Magnini, B. (eds.) CLEF 2004. LNCS, vol. 3491, pp. 170–179. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  10. Tomlinson, S.: Portuguese and Russian Retrieval with Hummingbird SearchServerTM at CLEF 2004. In: Peters, C., Clough, P., Gonzalo, J., Jones, G.J.F., Kluck, M., Magnini, B. (eds.) CLEF 2004. LNCS, vol. 3491, pp. 221–232. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  11. Di Nunzio, G.M., Ferro, N., Orio, N.: Experiments on Statistical Approaches to Compensate for Limited Linguistic Resources. In: Peters, C., Clough, P., Gonzalo, J., Jones, G.J.F., Kluck, M., Magnini, B. (eds.) CLEF 2004. LNCS, vol. 3491, pp. 60–72. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  12. Savoy, J.: Data Fusion for Effective European Monolingual Information Retrieval. In: Peters, C., Clough, P., Gonzalo, J., Jones, G.J.F., Kluck, M., Magnini, B. (eds.) CLEF 2004. LNCS, vol. 3491, pp. 233–244. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  13. Kamps, J., Adafre, S.F., de Rijke, M.: Effective Translation, Tokenization and Combination for Cross-Lingual Retrieval. In: Peters, C., Clough, P., Gonzalo, J., Jones, G.J.F., Kluck, M., Magnini, B. (eds.) CLEF 2004. LNCS, vol. 3491, pp. 123–134. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  14. Singhal, A., Buckley, C., Mitra, M.: Pivoted Document Length Normalization. In: Frei, H.P., Harman, D., Schäuble, P., Wilkinson, R. (eds.) Proceedings of the 19th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 1996), Zurich, Switzerland, pp. 21–29. ACM, New York (1996)

    Chapter  Google Scholar 

  15. Robertson, S.E., Walker, S., Jones, S., Hancock-Beaulieu, M.M., Gatford, M.: Okapi at TREC-3. In: Overview of the Third Text Retrieval Conference (TREC-3), pp. 109–126. NIST Special Publication 500-225 (1995)

    Google Scholar 

  16. Lioma, C., He, B., Plachouras, V., Ounis, I.: The University of Glasgow at CLEF 2004: French Monolingual Information Retrieval with Terrier. In: Peters, C., Clough, P., Gonzalo, J., Jones, G.J.F., Kluck, M., Magnini, B. (eds.) CLEF 2004. LNCS, vol. 3491, pp. 253–259. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  17. Serasset, G., Chevallet, J.-P.: Using Surface-Syntactic Parser and Derivation from Randomness: X-IOTA IR System used for CLIPS Mono & Bilingual Experiments for CLEF 2004. In: Peters, C., Clough, P., Gonzalo, J., Jones, G.J.F., Kluck, M., Magnini, B. (eds.) CLEF 2004. LNCS, vol. 3491, pp. 38–49. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  18. Ruiz, M.E., Srikanth, M.: UB at CLEF2004: Cross-language Information Retrieval using Statistical Language Models. In: Peters, C., Clough, P., Gonzalo, J., Jones, G.J.F., Kluck, M., Magnini, B. (eds.) CLEF 2004. LNCS, vol. 3491, pp. 180–187. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  19. Chen, A.: Cross-Language Retrieval Experiments at CLEF 2002. In: Peters, C., Braschler, M., Gonzalo, J. (eds.) CLEF 2002. LNCS, vol. 2785, pp. 5–20. Springer, Heidelberg (2003)

    Chapter  Google Scholar 

  20. Braschler, M.: Combination Approaches for Multilingual Text Retrieval. In: Information Retrieval, vol. 7(1-2), pp. 183–204. Kluwer Academic Publishers, Dordrecht

    Google Scholar 

  21. Savoy, J., Berger, P.-Y.: Selection and Merging Strategies for Multilingual Information Retrieval. In: Peters, C., Clough, P., Gonzalo, J., Jones, G.J.F., Kluck, M., Magnini, B. (eds.) CLEF 2004. LNCS, vol. 3491, pp. 27–37. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  22. Goñi-Menoyo, J.M., González, J.C., Martínez-Fernández, J.L., Villena-Román, J.: MIRACLE’s Hybrid Approach to Bilingual and Monolingual Information Retrieval. In: Peters, C., Clough, P., Gonzalo, J., Jones, G.J.F., Kluck, M., Magnini, B. (eds.) CLEF 2004. LNCS, vol. 3491, pp. 188–199. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  23. Nadeau, D., Jarmasz, M., Barrière, C., Foster, G., St-Jacques, C.: Using COTS Search Engines and Custom Query Strategies at CLEF. In: Peters, C., Clough, P., Gonzalo, J., Jones, G.J.F., Kluck, M., Magnini, B. (eds.) CLEF 2004. LNCS, vol. 3491, pp. 100–109. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  24. Hackl, R., Mandl, T., Womser-Hacker, C.: Mono- and Crosslingual Retrieval Experiments at the University of Hildesheim. In: Peters, C., Clough, P., Gonzalo, J., Jones, G.J.F., Kluck, M., Magnini, B. (eds.) CLEF 2004. LNCS, vol. 3491, pp. 165–169. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  25. Braschler, M., Peters, C.: Cross-Language Evaluation Forum: Objectives, Results, Achievements. Information Retrieval 7(1-2), 5–29

    Google Scholar 

  26. Gey, F.C.: Searching a Russian Document Collection using English, Chinese and Japanese Queries. In: Peters, C., Clough, P., Gonzalo, J., Jones, G.J.F., Kluck, M., Magnini, B. (eds.) CLEF 2004. LNCS, vol. 3491, pp. 200–206. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  27. Jones, G.J.F., Burke, M., Judge, J., Khasin, A., Lam-Adesina, A., Wagner, J.: Dublin City University at CLEF 2004: Experiments in Monolingual, Bilingual and Multilingual Retrieval. In: Peters, C., Clough, P., Gonzalo, J., Jones, G.J.F., Kluck, M., Magnini, B. (eds.) CLEF 2004. LNCS, vol. 3491, pp. 207–220. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  28. Hull, D.: Using Statistical Testing in the Evaluation of Retrieval Experiments. In: Korfhage, R., Rasmussen, E., Willett, P. (eds.) Proc. 16th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 1993), pp. 329–338. ACM Press, New York (1993)

    Chapter  Google Scholar 

  29. Blustein, J.: IR STAT PAK, http://www.csd.uwo.ca/~jamie/IRSP-overview.html

  30. Conover, W.J.: Practical Nonparametric Statistics, 1st edn. John Wiley and Sons, New York (1971)

    Google Scholar 

  31. Judge, G.G., Hill, R.C., Griffiths, W.E., Lütkepohl, H., Lee, T.C.: Introduction to the Theory and Practice of Econometrics, 2nd edn. John Wiley and Sons, New York (1988)

    MATH  Google Scholar 

  32. Tague-Sutcliffe, J.: The Pragmatics of Information Retrieval Experimentation, Revisited. In: Reading in Information Retrieval, pp. 205–216. Morgan Kaufmann Publishers, San Francisco (1997)

    Google Scholar 

  33. Voorhees, E., Buckley, C.: The Effect of Topic Set Size on Retrieval Experiment Error. In: Proc. of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 307–314 (1998)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Eurospider Information Technology AG, 8006, Zurich, Switzerland

    Martin Braschler

  2. Department of Information Engineering, University of Padua, Italy

    Giorgio M. Di Nunzio & Nicola Ferro

  3. ISTI-CNR, Area di Ricerca, 56124, Pisa, Italy

    Carol Peters

Authors
  1. Martin Braschler
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Giorgio M. Di Nunzio
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nicola Ferro
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Carol Peters
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. ISTI-CNR, Area di Ricerca, Pisa, Italy

    Carol Peters

  2. Sheffield University, Sheffield, United Kingdom

    Paul Clough

  3. No Affiliations,  

    Julio Gonzalo

  4. Centre for Digital Video Processing & School of Computing, Dublin City University, Dublin 9, Ireland

    Gareth J. F. Jones

  5. German Institute for International and Security Affairs, Stiftung Wissenschaft und Politik (SWP), Ludwigkirchplatz 3-4, P.O. Box, 10719, Berlin, Germany

    Michael Kluck

  6. ITC-IRST, Trento, Italy

    Bernardo Magnini

Rights and permissions

Reprints and permissions

Copyright information

© 2005 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Braschler, M., Di Nunzio, G.M., Ferro, N., Peters, C. (2005). CLEF 2004: Ad Hoc Track Overview and Results Analysis. In: Peters, C., Clough, P., Gonzalo, J., Jones, G.J.F., Kluck, M., Magnini, B. (eds) Multilingual Information Access for Text, Speech and Images. CLEF 2004. Lecture Notes in Computer Science, vol 3491. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11519645_2

Download citation

  • DOI: https://doi.org/10.1007/11519645_2

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-27420-9

  • Online ISBN: 978-3-540-32051-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics