skip to main content
10.1145/1244002.1244193acmconferencesArticle/Chapter ViewAbstractPublication PagessacConference Proceedingsconference-collections
Article

Why do successful search systems fail for some topics

Published: 11 March 2007 Publication History

Abstract

This paper describes and evaluates the vector-space and probabilistic IR models used to retrieve news articles from a corpus written in the French language. Based on three CLEF test-collections and 151 queries, we classify the poor retrieval results of difficult topics under 6 categories. The explanations we obtain from this analysis differ from those suggested a priori by our students. We use the Web to manually or automatically find related search terms to the original query. We evaluate these two query expansion strategies in order to improve mean average precision (MAP) and to reduce the number of topics for which no pertinent responses are listed among the top ten references returned.

References

[1]
Robertson, S. E., Walker, S., and Beaulieu, M. Experimentation as a way of life: Okapi at TREC. IP&M, 36(1), 2000, 95--108.
[2]
Hiemstra, D. Using language models for information retrieval. CTIT Ph.D. Thesis, 2000.
[3]
Zhai, C., and Lafferty, J. A study of smoothing methods for language models applied to information retrieval. ACM-TOIS, 22(2), 2004, 179--214.
[4]
Amati, G., and van Rijsbergen, C. J. Probabilistic models of information retrieval based on measuring the divergence from randomness. ACM-TOIS, 20(4), 2002, 357--389.
[5]
Peters, C., Braschler, M., Gonzalo, J., and Kluck, M. (Eds). Evaluation of cross-language information retrieval. LNCS #2406, Springer, Berlin, 2002.
[6]
Peters, C., Braschler, M., Gonzalo, J., and Kluck, M. (Eds). Advances in cross-language information retrieval. LNCS #2785, Springer, Berlin, 2003.
[7]
Peters, C., Braschler, M., Gonzalo, J., and Kluck, M. (Eds). Comparative evaluation of multilingual information access systems. LNCS #3237, Springer, Berlin, 2004.
[8]
Salton, G., and Buckley, C. Term-weighting approaches in automatic text retrieval. IP&M, 24(5), 19888, 513--523.
[9]
Buckley, C., Singhal, A., Mitra, M., and Salton, G. New retrieval approaches using SMART. In Proceedings of TREC-4. Gaithersburg, MA, 1996, 25--48.
[10]
Singhal, A., Choi, J., Hindle, D., Lewis, D. D. and Pereira, F. (1999). AT&T at TREC-7. In Proceedings TREC-7, Gaithersburg, MA, 1999, 239--251.
[11]
Moulinier, I. Thomson Legal and Regulatory at NTCIR-4: Monolingual and pivot-language retrieval experiments. In Proceedings NTCIR-4, Tokyo, 2004, 158--165.
[12]
Krovetz, R. Viewing morphology as an inference process. In Proceedings of the ACM-SIGIR. Pittsburgh, PA, 1993, 191--202.
[13]
Buckely C. Why current IR engines fail. In Proceedings ACM-SIGIR, Sheffield, 2004, 584--585.
[14]
Voorhees, E. M. Overview of the TREC 2004 robust retrieval track. In Proceedings TREC-2004. Gaithersburg, MA, 2004, 70--79.
[15]
Voorhees, E. M. The TREC 2005 robust track. ACM-SIGIR Forum, 40(1), 2006, 41--48.
[16]
Kwok, K. L., Grunfeld, L., Sun, H. L., and Deng, P. TREC2004 robust track experiments using PIRCS. In Proceedings TREC-2004. Gaithersburg, MA, 2004.
[17]
Kwok, K. L., Grunfeld L., Dinstl, N., and Chan, M. TREC-9 cross-language, web and question-answering track experiments using PIRCS. In Proceedings TREC-9, Gaithersburg, MA, 2001, 417--426

Cited By

View all
  • (2021)Use of Faceted Search: The Effect on ResearchersAdvances in Visual Informatics10.1007/978-3-030-90235-3_24(276-286)Online publication date: 16-Nov-2021
  • (2017)Negative Relevance Feedback for Exploratory Search with Visual Interactive Intent ModelingProceedings of the 22nd International Conference on Intelligent User Interfaces10.1145/3025171.3025222(149-159)Online publication date: 7-Mar-2017
  • (2017)Optimierung der Onsite-Suche am Beispiel von otto.deDatenbank-Spektrum10.1007/s13222-016-0240-717:1(15-20)Online publication date: 17-Jan-2017
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SAC '07: Proceedings of the 2007 ACM symposium on Applied computing
March 2007
1688 pages
ISBN:1595934804
DOI:10.1145/1244002
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 March 2007

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. failure analysis
  2. robust evaluation

Qualifiers

  • Article

Conference

SAC07
Sponsor:

Acceptance Rates

Overall Acceptance Rate 1,650 of 6,669 submissions, 25%

Upcoming Conference

SAC '25
The 40th ACM/SIGAPP Symposium on Applied Computing
March 31 - April 4, 2025
Catania , Italy

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)0
  • Downloads (Last 6 weeks)0
Reflects downloads up to 15 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2021)Use of Faceted Search: The Effect on ResearchersAdvances in Visual Informatics10.1007/978-3-030-90235-3_24(276-286)Online publication date: 16-Nov-2021
  • (2017)Negative Relevance Feedback for Exploratory Search with Visual Interactive Intent ModelingProceedings of the 22nd International Conference on Intelligent User Interfaces10.1145/3025171.3025222(149-159)Online publication date: 7-Mar-2017
  • (2017)Optimierung der Onsite-Suche am Beispiel von otto.deDatenbank-Spektrum10.1007/s13222-016-0240-717:1(15-20)Online publication date: 17-Jan-2017
  • (2016)The twist measure for IR evaluationJournal of the Association for Information Science and Technology10.1002/asi.2341667:3(620-648)Online publication date: 1-Mar-2016
  • (2014)A Visual Interactive Environment for Making Sense of Experimental DataProceedings of the 36th European Conference on IR Research on Advances in Information Retrieval - Volume 841610.5555/2964060.2964135(767-770)Online publication date: 13-Apr-2014
  • (2014)Information RetrievalEncyclopedia of Information Science and Technology, Third Edition10.4018/978-1-4666-5888-2.ch386(3923-3931)Online publication date: 31-Jul-2014
  • (2014)VIRTUEJournal of Visual Languages and Computing10.1016/j.jvlc.2013.12.00325:4(394-413)Online publication date: 1-Aug-2014
  • (2014)A Visual Interactive Environment for Making Sense of Experimental DataAdvances in Information Retrieval10.1007/978-3-319-06028-6_92(767-770)Online publication date: 2014
  • (2012)Translation techniques in cross-language information retrievalACM Computing Surveys10.1145/2379776.237977745:1(1-44)Online publication date: 7-Dec-2012
  • (2011)Word sense disambiguation in information retrieval using query expansionProceedings of the 2011 International Conference on Electrical Engineering and Informatics10.1109/ICEEI.2011.6021532(1-6)Online publication date: Jul-2011
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media