Diane Kelly
ABSTRACT
Term relevance feedback has had a long history in information retrieval. However, research on interactive term relevance feedback has yielded mixed results. In this paper, we investigate several aspects related to the elicitation of term relevance feedback: the display of document surrogates, the technique for identifying or selecting terms, and sources of expansion terms. We conduct a between subjects experiment (n=61) of three term relevance feedback interfaces using the 2005 TREC HARD collection, and evaluate each interface with respect to query length and retrieval performance. Results demonstrate that queries created with each experimental interface significantly outperformed corresponding baseline queries, even though there were no differences in performance between interface conditions. Results also demonstrate that pseudo-relevance feedback runs outperformed both baseline and experimental runs as assessed by recall-oriented measures, but that user-generated terms improved precision.
- Allan, J. (2006). HARD Track overview in TREC 2005 high accuracy retrieval from documents. In E. M. Voorhees & L. P. Buckland (Eds.), TREC-2005, Proceedings of the Fourteenth Text Retrieval Conference. Washington, D.C.: Government Printing Office.Google Scholar
- Anick, P. (2003). Using terminological feedback for web search refinement: A log based study. In Proceedings of the 26th Annual ACM International Conference on Research and Development in Information Retrieval (SIGIR '03), Toronto, CA, 88--95. Google ScholarDigital Library
- Beaulieu, M. (1997). Experiments on interfaces to support query expansion. Journal of Documentation, 53(1), 8--19.Google ScholarCross Ref
- Belkin, N. J. (1993). Interaction with texts: Information retrieval as information-seeking behavior. In Information Retrieval '93, Germany, 55--66.Google Scholar
- Belkin, N. J., Cool, C., Kelly, D., Lin, S. J., Park, S. Y., Perez-Carballo, J., & Sikora, C. (2001). Iterative exploration, design and evaluation of support for query reformulation in interactive information retrieval. Information Processing & Management, 37(3), 404--434. Google ScholarDigital Library
- Belkin, N. J., Cool, C., Kelly, D., Lee, H.-J., Muresan, G., Tang, M.-C., & Yuan, X.-J. (2003). Query length in interactive information retrieval. In Proceedings of the 26th Annual ACM International Conference on Research and Development in Information Retrieval (SIGIR '03), Canada, 205--212. Google ScholarDigital Library
- Croft, W. B. & Das, R. (1990). Experiments with query acquisition and use in document retrieval systems. In Proceedings of the 13th Annual ACM International Conference on Research and Development in Information Retrieval (SIGIR '90), Brussels, 349--368. Google ScholarDigital Library
- Efthimiadis, E. N. (2000). Interactive query expansion: A user-based evaluation in a relevance feedback environment. Journal of the American Society for Information Science & Technology, 51(11), 989--1003. Google ScholarDigital Library
- Efthimiadis, E. N. (1996). Query expansion. Annual Review of Information Science & Technology, 31.Google Scholar
- Harman, D. (1988). Towards interactive query expansion. In Proceedings of the 11th Annual ACM International Conference on Research and Development in Information Retrieval (SIGIR '88), Grenoble, 321--333. Google ScholarDigital Library
- Joho, H., Coverson, C., Sanderson, M., & Beaulieu, M. (2002). Hierarchical presentation of expansion terms. In Proceedings of the 17th Annual ACM Symposium on Applied Computing (SAC '02), Madrid, Spain, 645--649. Google ScholarDigital Library
- Kelly, D., Dollu, V. D. & Fu, X. (2005). The loquacious user: A document-independent source of terms for query expansion. In Proceedings of the 28th Annual ACM International Conference on Research and Development in Information Retrieval (SIGIR '05), Brazil, 457--464. Google ScholarDigital Library
- Kelly, D. & Fu, X. (2006). University of North Carolina's HARD Track Experiments at TREC 2005. In E. M. Voorhees & L. P. Buckland (Eds.), TREC-2006, Proceedings of the Fourteenth Text Retrieval Conference. Washington, D.C.: Government Printing Office.Google Scholar
- Koenemann, J., & Belkin, N. J. (1996). A case for interaction: A study of interactive information retrieval behavior and effectiveness. In Proceedings of the SIGCHI Conference (CHI'96), Canada, 205--212. Google ScholarDigital Library
- Larson, R. R. (2001). TREC interactive with Cheshire II. Information Processing & Management, 37(3), 485--505. Google ScholarDigital Library
- Magennis, M. & van Rijsbergen, C. J. (1997). The potential and actual effectiveness of interactive query expansion. In Proceedings of the 20th Annual ACM International Conference on Research and Development in Information Retrieval (SIGIR '97), Philadelphia PA, USA, 324--332. Google ScholarDigital Library
- Nameth, Y., Shapira, B., & Taeib-Maimon, M. (2004). Evaluation of the real and perceived value of automatic and interactive query expansion. In Proceedings of the 27th Annual ACM International Conference on Research and Development in Information Retrieval (SIGIR '04), Sheffield, UK, 526--527. Google ScholarDigital Library
- Pennanen, M., & Vakkari, P. (2003). Students' conceptual structure, search process, and outcome while preparing a research proposal: A longitudinal case study. Journal of the American Society for Information Science & Technology, 54(8), 759--770. Google ScholarDigital Library
- Robertson, S. E., Walker, S., Jones, S., Hancock-Beaulieu, M. M., Gatford, M. (1995). Okapi at TREC-3. In D. Harman (Ed.), TREC-3, Proceedings of the Third Text Retrieval Conference. Washington, D.C.: Government Printing Office.Google Scholar
- Ruthven, I. (2003). Re-examining the potential effectiveness of interactive query expansion. In Proceedings of the 26th Annual ACM International Conference on Research and Development in Information Retrieval (SIGIR '03), Toronto, CA, 213--220. Google ScholarDigital Library
- Spink, A. (1994). Term relevance feedback and query expansion: Relation to design. In Proceedings of the 17th Annual ACM International Conference on Research and Development in Information Retrieval (SIGIR '94), Dublin, Ireland, 81--90. Google ScholarDigital Library
- Spink, A., & Jansen, B. J. (2004). Web search: Public searching of the Web. Kluwer Academic Publishers. Google ScholarDigital Library
- Voorhees, E. M. (2006). Overview of TREC 2006. In E. M. Voorhees & L. P. Buckland (Eds.), TREC-2006, Proceedings of the Fourteenth Text Retrieval Conference. Washington, D.C.: Government Printing Office.Google Scholar
- Wu, M., Fuller, M., & Wilkinson, R. (2001). Using clustering and classification approaches in interactive retrieval. Information Processing & Management, 37(3), 459--484. Google ScholarDigital Library
- Yang, K., Maglaughlin, K. L., & Newby, G. B. (2001). Passage feedback with IRIS. Information Processing & Management, 37(3), 521--541. Google ScholarDigital Library
Index Terms
- Elicitation of term relevance feedback: an investigation of term source and context
Recommendations
Document-based and term-based linear methods for pseudo-relevance feedback
Query expansion is a successful approach for improving Information Retrieval effectiveness. This work focuses on pseudo-relevance feedback (PRF) which provides an automatic method for expanding queries without explicit user feedback. These techniques ...
Query dependent pseudo-relevance feedback based on wikipedia
SIGIR '09: Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrievalPseudo-relevance feedback (PRF) via query-expansion has been proven to be e®ective in many information retrieval (IR) tasks. In most existing work, the top-ranked documents from an initial search are assumed to be relevant and used for PRF. One problem ...
Relevance Feedback Fusion via Query Expansion
WI-IAT '12: Proceedings of the The 2012 IEEE/WIC/ACM International Joint Conferences on Web Intelligence and Intelligent Agent Technology - Volume 03Relevance Feedback (RF) is an important technique to improve information retrieval and has emerged as one of the hottest topics for both the industry and academic researchers. The performance of RF depends on feedback information. As the volume of ...
Comments