Abstract
Hummingbird submitted ranked result sets for all Monolingual Information Retrieval tasks of the Cross-Language Evaluation Forum (CLEF) 2002. Enabling stemming in SearchServer increased average precision by 16 points in Finnish, 9 points in German, 4 points in Spanish, 3 points in Dutch, 2 points in French and Italian, and 1 point in Swedish and English. Accent-indexing increased average precision by 3 points in Finnish and 2 points in German, but decreased it by 2 points in French and 1 point in Italian and Swedish. Treating apostrophes as word separators increased average precision by 3 points in French and 1 point in Italian. Confidence intervals produced using the bootstrap percentile method were found to be very similar to those produced using the standard method; both were of similar width to rank-based intervals for differences in average precision, but substantially narrower for differences in Precision@10.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
M. Braschler and C. Peters. CLEF2002: Methodology and Metrics. This volume 243, 246
Michael R. Chernick. Bootstrap Methods: A Practitioner’s Guide. 1999. John Wiley & Sons 254
Cross-Language Evaluation Forum web site. http://www.clef-campaign.org/ 243
Andrew Hodgson. Converting the Fulcrum Search Engine to Unicode. In Sixteenth International Unicode Conference, Amsterdam, The Netherlands, March 2000 243
Myles Hollander and Douglas A. Wolfe. Nonparametric Statistical Methods. Second Edition, 1999. John Wiley & Sons 254
NTCIR (NII-NACSIS Test Collection for IR Systems) Home Page. http://www.research.nii.ac.jp/~ntcadm/index-en.html 243
S.E. Robertson, S. Walker, S. Jones, M. M. Hancock-Beaulieu, M. Gatford. (City University.) Okapi at TREC-3. In D. K. Harman, editor, Overview of the Third Text REtrieval Conference (TREC-3). NIST Special Publication 500-226. http://trec.nist.gov/pubs/trec3/t3proceedings.html 246
Jacques Savoy. (Université de Neuchâtel.) CLEF and Multilingual information retrieval resource page. http://www.unine.ch/info/clef/ 251
Amit Singhal, John Choi, Donald Hindle, David Lewis and Fernando Pereira. AT&T at TREC-7. In E.M. Voorhees and D.K. Harman, editors, Proceedings of the Seventh Text REtrieval Conference (TREC-7). NIST Special Publication 500-242. http://trec.nist.gov/pubs/trec7/t7proceedings.html 246
K. Sparck Jones, S. Walker and S.E. Robertson. (City University.) A probabilistic model of information retrieval: development and status. August 1998. Page 15 247
Text REtrieval Conference (TREC) Home Page. http://trec.nist.gov/ 243
Stephen Tomlinson. Stemming Evaluated in 6 Languages by Hummingbird SearchServer™ at CLEF 2001. In C. Peters, M. Braschler, J. Gonzalo, M. Kluck, editors, Evaluation of Cross-Language Information Retrieval Systems: Second Workshop of the Cross-Language Evaluation Forum, CLEF 2001, Darmstadt, Germany, September 3–4, 2001. Revised Papers. Springer LNCS 2406. http://link.springer.de/link/service/series/0558/tocs/t2406.htm 248
Stephen Tomlinson and Tom Blackwell. Hummingbird’s Fulcrum SearchServer at TREC-9. In E. M. Voorhees and D. K. Harman, editors, Proceedings of the Ninth Text REtrieval Conference (TREC-9). NIST Special Publication 500-249. http://trec.nist.gov/pubs/trec9/t9proceedings.html 243
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Tomlinson, S. (2003). Experiments in 8 European Languages with Hummingbird SearchServer™ at CLEF 2002. In: Peters, C., Braschler, M., Gonzalo, J., Kluck, M. (eds) Advances in Cross-Language Information Retrieval. CLEF 2002. Lecture Notes in Computer Science, vol 2785. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-45237-9_20
Download citation
DOI: https://doi.org/10.1007/978-3-540-45237-9_20
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-40830-7
Online ISBN: 978-3-540-45237-9
eBook Packages: Springer Book Archive