Effects of highly agreed documents in relevancy prediction
Pages 883 - 884
Abstract
Finding significant contextual features is a challenging task in the development of interactive information retrieval (IR) systems. This paper investigated a simple method to facilitate such a task by looking at aggregated relevance judgements of retrieved documents. Our study suggested that the agreement on relevance judgements can indicate the effectiveness of retrieved documents as the source of significant features. The effect of highly agreed documents gives us practical implication for the design of adaptive search models in interactive IR systems.
References
[1]
E. Agichtein, et al. Learning user interaction models for predicting web search result preferences. In Proceedings of the 29th SIGIR Conference, 3--10, 2006.
[2]
R. O. Duda and P. E. Hart. Pattern Classification and Scene Analysis. John Wiley Sons, New York, 1973.
[3]
S. Fox, K. Karnawat, M. Mydland, S. Dumais, and T. White. Evaluating implicit measures to improve web search. ACM TOIS, 23(2):147--168, 2005.
[4]
L. J. H. Zhang and J. Su. Hidden naive bayes. In Proceedings of AAAI-05. AAAI Press, 919--924, 2005.
[5]
P. Ingwersen and K. Järvelin. Information retrieval in context: IRiX. SIGIR Forum, 39(2):31--39, 2005.
[6]
H. Joho and J. M. Jose. Slicing and dicing the information space using local contexts. In Proceedings of the First IIiX Symposium, 111--126, 2006.
[7]
J. Pearl. Probabilistic Reasoning with Intelligent Systems. Morgan & Kaufman, San Mateo, 1988.
[8]
E. Sormunen. Liberal relevance criteria of TREC : counting on negligible documents? In Proceedings of the 25th SIGIR Conference, 324--330, 2002.
[9]
E. M. Voorhees. Variations in relevance judgments and the measurement of retrieval effectiveness. In Proceedings of the 21st SIGIR conference, 315--323, 1998.
[10]
G. I. Webb et al. Not so naive bayes: aggregating one-dependence estimators. Mach. Learn., 58(1):5--24, 2005.
Index Terms
- Effects of highly agreed documents in relevancy prediction
Recommendations
Exploring Peripheral Physiology as a Predictor of Perceived Relevance in Information Retrieval
IUI '15: Proceedings of the 20th International Conference on Intelligent User InterfacesPeripheral physiological signals, as obtained using electrodermal activity and facial electromyography over the corrugator supercilii muscle, are explored as indicators of perceived relevance in information retrieval tasks. An experiment with 40 ...
IR evaluation methods for retrieving highly relevant documents
SIGIR Test-of-Time Awardees 1978-2001This paper proposes evaluation methods based on the use of non-dichotomous relevance judgements in IR experiments. It is argued that evaluation methods should credit IR methods for their ability to retrieve highly relevant documents. This is desirable ...
Relevance prediction in similarity-search systems using extreme value theory
Highlights- It is possible to predict the relevance of top-ranked results returned by a retrieval system by considering only their scores and employing the Extreme Value ...
AbstractAmong the challenges present in the design of retrieval systems, how to accurately assess their performance is perhaps one of the most important. Many applications such as rank aggregation or relevance feedback can be significantly ...
Comments
Information & Contributors
Information
Published In
July 2007
946 pages
ISBN:9781595935977
DOI:10.1145/1277741
- General Chairs:
- Wessel Kraaij,
- Arjen P. de Vries,
- Program Chairs:
- Charles L. A. Clarke,
- Norbert Fuhr,
- Noriko Kando
Copyright © 2007 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 23 July 2007
Check for updates
Author Tags
Qualifiers
- Article
Conference
Acceptance Rates
Overall Acceptance Rate 792 of 3,983 submissions, 20%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 262Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 28 Feb 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in