Tahia Infantes-Morris
ABSTRACT
In this paper we report the results of an independent experimental evaluation of an information retrieval (IR) system developed at the Illinois Institute of Technology (IIT). The system, which is called the Advanced Information Retrieval Engine (AIRE), consists of a set of tools and utilities providing indexing, extraction, searching and visualization. We evaluated AIRE on three data sets from the Text REtrieval Conference (TREC) - TREC 8, 9 and 10. Overall, our results indicate that AIRE is a highly accurate IR system. Compared with results published by IIT, in our experiments AIRE consistently scored higher in recall. AIRE also scored higher in precision, but only for automatic tasks. In manual tasks, AIRE scored lower in precision in our experiments, but we attributed that to factors external to AIRE. Our final conclusion is that AIRE is a highly accurate IR system.
- Ricardo Baeza-Yates, Berthier Ribeiro-Neto, and Berthier Ribiero-Neto, "Modern Information Retrieval," Pearson Education, May 1999. Google Scholar
Digital Library
- Gerald Kowalski, Information Retrieval Systems: Theory and Implementation, Kluwer Academic Publishers, Boston, 1997. Google ScholarDigital Library
- Grossman, David A. and Frieder, Ophir, Information Retrieval: Algorithms and Heuristics, Kluwer Academic Publishers, 1998. Google ScholarDigital Library
- Advaned Information Retrieval Engine (AIRE), http://www.ir.iit.edu/projects/AIRE.html.Google Scholar
- AIRE (Advanced Information Retrieval Engine) http://ir.iit.edu/abdur/research/aire/AIRE.html.Google Scholar
- McCabe, M. C., Chowdhury, A., Holmes, D. O., Grossman, D. A., Alford, K. L., and Frieder, O., "IIT at TREC-8: Improving Baseline Precision," NIST Special Publication 500--246: The Eighth Text Retrieval Conference (TReC 8).Google Scholar
- Chowdhury, A., Beitzel, S., Jensen, E., Sai-lee, M., Grossman, D. A., Frieder, O., McCabe, M. C., and Holmes, D. O., "IIT TREC-9 - Entity Based Feedback with Fusion," NIST Special Publication 500--249: The Ninth Text REtrieval Conference (TREC-9).Google Scholar
- Aljlayl, M., Beitzel, S., Jensen, E., Chowdhury, A., Holmes, D., Lee, M., Grossman, D., and Frieder, O., "IIT at TREC-10," NIST Special Publication 500--250: The Tenth Text REtrieval Conference (TREC 2001).Google Scholar
- Hawking, D., Voorhees, E., Craswell, N., and Bailey, P., "Overview of the TREC-8 Web Track," NIST Special Publication 500-246: The Eighth Text Retrieval Conference (TReC 8).Google Scholar
- Hawking, D., "Overview of the TREC-9 Web Track," NIST Special Publication 500--249: The Ninth Text REtrieval Conference (TREC-9).Google Scholar
- Voorhees, E. M., and Harman, D., "Overview of TREC 2001," NIST Special Publication 500--250: The Tenth Text REtrieval Conference (TREC 2001).Google Scholar
- Hawking, D., and Craswell, N., "Overview of the TREC-2001 Web Track," NIST Special Publication 500--250: The Tenth Text REtrieval Conference (TREC 2001).Google Scholar
Index Terms
- Industrial evaluation of a highly-accurate academic IR system
Recommendations
Query clustering and IR system detection: experiments on TREC data
RIAO '07: Large Scale Semantic Access to Content (Text, Image, Video, and Sound)This paper investigates two aspects in this experiment. Linguistic techniques are used to categorize queries in a first step. This classification is then used to analyze systems performances in a TREC context. More precisely, we cluster TREC topics with ...
IR evaluation methods for retrieving highly relevant documents
SIGIR Test-of-Time Awardees 1978-2001This paper proposes evaluation methods based on the use of non-dichotomous relevance judgements in IR experiments. It is argued that evaluation methods should credit IR methods for their ability to retrieve highly relevant documents. This is desirable ...
Relevance dimensions in preference-based IR evaluation
SIGIR '13: Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrievalEvaluation of information retrieval (IR) systems has recently been exploring the use of preference judgments over two search result lists. Unlike the traditional method of collecting relevance labels per single result, this method allows to consider the ...
Comments