research-article

Adaptive relevance feedback in information retrieval

Authors:

ChengXiang ZhaiAuthors Info & Claims

CIKM '09: Proceedings of the 18th ACM conference on Information and knowledge management

Pages 255 - 264

https://doi.org/10.1145/1645953.1645988

Published: 02 November 2009 Publication History

Abstract

Relevance Feedback has proven very effective for improving retrieval accuracy. A difficult yet important problem in all relevance feedback methods is how to optimally balance the original query and feedback information. In the current feedback methods, the balance parameter is usually set to a fixed value across all the queries and collections. However, due to the difference in queries and feedback documents, this balance parameter should be optimized for each query and each set of feedback documents.

In this paper, we present a learning approach to adaptively predict the optimal balance coefficient for each query and each collection. We propose three heuristics to characterize the balance between query and feedback information. Taking these three heuristics as a road map, we explore a number of features and combine them using a regression approach to predict the balance coefficient. Our experiments show that the proposed adaptive relevance feedback is more robust and effective than the regular fixed-coefficient feedback.

References

[1]

Giambattista Amati, Claudio Carpineto, and Giovanni Romano. Query difficulty, robustness, and selective application of query expansion. In Proceedings of ECIR '04, pages 127--137, 2004.

[2]

Ann Aula. Query formulation in web information search. In Proceedings of ICWI '03, pages 403--410, 2003.

[3]

Chris Buckley and Stephen E. Robertson. Relevance feedback track overview: Trec 2008. In TREC '08, 2008.

[4]

Chris Buckley, Gerard Salton, James Allan, and Amit Singhal. Automatic query expansion using smart: Trec 3. In Proceedings of TREC '94, pages 69--80, 1994.

[5]

Chris Burges, Tal Shaked, Erin Renshaw, Ari Lazier, Matt Deeds, Nicole Hamilton, and Greg Hullender. Learning to rank using gradient descent. In ICML '05: Proceedings of the 22nd international conference on Machine learning, pages 89--96, 2005.

Digital Library

[6]

Zhe Cao, Tao Qin, Tie-Yan Liu, Ming-Feng Tsai, and Hang Li. Learning to rank: from pairwise approach to listwise approach. In Proceedings of ICML, pages 129--136, 2007.

Digital Library

[7]

David Carmel, Elad Yom-Tov, Adam Darlow, and Dan Pelleg. What makes a query difficult? In Proceedings of SIGIR '06, pages 390--397, 2006.

Digital Library

[8]

Ben Carterette and Desislava Petkova. Learning a ranking from pairwise preferences. In Proceedings of SIGIR '06, pages 629--630, 2006.

Digital Library

[9]

Charles Clarke, Nick Craswell, and Ian Soboroff. Overview of the trec 2004 terabyte track. In Proceedings of TREC '04, pages 500--261, 2004.

[10]

William S. Cooper, Fredric C. Gey, and Daniel P. Dabney. Probabilistic retrieval based on staged logistic regression. In Proceedings of SIGIR '92, pages 198--210, 1992.

Digital Library

[11]

Stephen Cronen-Townsend, Yun Zhou, and W. Bruce Croft. Predicting query performance. In Proceedings of SIGIR '02, pages 299--306, 2002.

Digital Library

[12]

Stephen Cronen-Townsend, Yun Zhou, and W. Bruce Croft. A framework for selective query expansion. In Proceedings of CIKM '04, pages 236--237, 2004.

Digital Library

[13]

Norbert Fuhr and Chris Buckley. A probabilistic learning approach for document indexing. ACM Transactions on Information Systems (TOIS), 9(3):223--248, 1991.

Digital Library

[14]

Fredric C. Gey. Inferring probability of relevance using the method of logistic regression. In Proceedings of SIGIR '94, pages 222--231, 1994.

Digital Library

[15]

Trevor Hastie, Robert Tibshirani, and Jerome Friedman. The Elements of Statistical Learning. Springer Series in Statistics. Springer New York Inc., New York, NY, USA, 2001.

[16]

Claudia Hauff, Djoerd Hiemstra, and Franciska de Jong. A survey of pre-retrieval query performance predictors. In Proceedings of CIKM '08, pages 1419--1420, 2008.

Digital Library

[17]

Ben He and Iadh Ounis. Combining fields for query expansion and adaptive query expansion. Inf. Process. Manage., 43(5):1294--1307, 2007.

Digital Library

[18]

T. Joachims. Optimizing search engines using clickthrough data. In Proceedings of the ACM KDD 2002, pages 133--142, 2002.

Digital Library

[19]

Diane Kelly and Jaime Teevan. Implicit feedback for inferring user preference: a bibliography. SIGIR Forum, 37(2):18--28, 2003.

Digital Library

[20]

John D. Lafferty and Chengxiang Zhai. Document language models, query models, and risk minimization for information retrieval. In Proceedings of SIGIR '01, pages 111--119, 2001.

Digital Library

[21]

Victor Lavrenko and W. Bruce Croft. Relevance-based language models. In Proceedings of SIGIR '01, pages 120--127, 2001.

Digital Library

[22]

Jianhua Lin. Divergence measures based on the shannon entropy. IEEE Trans. Infor. Theory, 37:145--151, 1991.

Digital Library

[23]

Yuanhua Lv and ChengXiang Zhai. A comparative study of methods for estimating query language models with pseudo feedback. In Proceedings of CIKM '09, 2009.

Digital Library

[24]

Jay M. Ponte and W. Bruce Croft. A language modeling approach to information retrieval. In Proceedings of SIGIR '98, pages 275--281, 1998.

Digital Library

[25]

Stephen E. Robertson and Karen Sparck Jones. Relevance weighting of search terms. Journal of the American Society of Information Science, 27(3):129--146, 1976.

[26]

Stephen E. Robertson, Steve Walker, Susan Jones, Micheline Hancock-Beaulieu, and Mike Gatford. Okapi at trec-3. In TREC '94, pages 109--126, 1994.

[27]

J. J. Rocchio. Relevance feedback in information retrieval. In The SMART Retrieval System: Experiments in Automatic Document Processing, pages 313--323. Prentice-Hall Inc., 1971.

Digital Library

[28]

Gerard Salton and Chris Buckley. Improving retrieval performance by relevance feedback. Journal of the American Society of Information Science, 41(4):288--297, 1990.

[29]

Tao Tao and ChengXiang Zhai. Regularized estimation of mixture models for robust pseudo-relevance feedback. In Proceedings of SIGIR '06, pages 162--169, 2006.

Digital Library

[30]

Zuobing Xu and Ram Akella. A bayesian logistic regression model for active relevance feedback. In Proceedings of SIGIR '08, pages 227--234, 2008.

Digital Library

[31]

Elad Yom-Tov, Shai Fine, David Carmel, and Adam Darlow. Learning to estimate query difficulty: including applications to missing content detection and distributed information retrieval. In Proceedings of SIGIR '05, pages 512--519, 2005.

Digital Library

[32]

ChengXiang Zhai and John D. Lafferty. Model-based feedback in the language modeling approach to information retrieval. In Proceedings of CIKM '01, pages 403--410, 2001.

Digital Library

[33]

ChengXiang Zhai and John D. Lafferty. A study of smoothing methods for language models applied to ad hoc information retrieval. In Proceedings of SIGIR '01, pages 334--342, 2001.

Digital Library

[34]

Zhaohui Zheng, Keke Chen, Gordon Sun, and Hongyuan Zha. A regression framework for learning ranking functions using relative relevance judgments. In Proceedings of SIGIR '07, pages 287--294, 2007.

Digital Library

Cited By

Datta SGanguly DMacAvaney SGreene D(2024)A Deep Learning Approach for Selective Relevance FeedbackAdvances in Information Retrieval10.1007/978-3-031-56060-6_13(189-204)Online publication date: 16-Mar-2024
https://doi.org/10.1007/978-3-031-56060-6_13
Ye ZXie XAi QLiu YWang ZSu WZhang M(2023)Relevance Feedback with Brain SignalsACM Transactions on Information Systems10.1145/363787442:4(1-37)Online publication date: 18-Dec-2023
https://dl.acm.org/doi/10.1145/3637874
Capatina AMicu AMicu ARibeiro‐Navarrete S(2023)Knowledge maps for large‐scale group decision making in social media content analysisExpert Systems10.1111/exsy.1350942:1Online publication date: 23-Nov-2023
https://doi.org/10.1111/exsy.13509
Show More Cited By

Index Terms

Adaptive relevance feedback in information retrieval
1. Information systems
  1. Information retrieval
    1. Evaluation of retrieval results
      1. Relevance assessment
    2. Retrieval models and ranking

Recommendations

The Study of Methods for Language Model Based Positive and Negative Relevance Feedback in Information Retrieval
ISISE '12: Proceedings of the 2012 Fourth International Symposium on Information Science and Engineering

Relevance feedback techniques are important to Information retrieval (IR), which can effectively improve the performance of IR. The feedback includes positive and negative relevance one. The most of the previous work using feedback have focused on ...
Image retrieval based on indexing and relevance feedback

In content based image retrieval (CBIR) system, search engine retrieves the images similar to the query image according to a similarity measure. It should be fast enough and must have a high precision of retrieval. Indexing scheme is used to achieve a ...
Interactive content-based image retrieval using relevance feedback

Database search engines are generally used in a one-shot fashion in which a user provides query information to the system and, in return, the system provides a number of database instances to the user. A relevance feedback system allows the user to ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '09: Proceedings of the 18th ACM conference on Information and knowledge management

November 2009

2162 pages

ISBN:9781605585123

DOI:10.1145/1645953

General Chairs:
David Cheung
University of Hong Kong, Hong Kong
,
Il-Yeol Song
Drexel University, USA
,
Program Chairs:
Wesley Chu
UCLA, USA
,
Xiaohua Hu
Drexel University, USA
,
Jimmy Lin
University of Maryland, USA

Copyright © 2009 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 November 2009

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CIKM '09

Sponsor:

CIKM '09: Conference on Information and Knowledge Management

November 2 - 6, 2009

Hong Kong, China

Acceptance Rates

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Sponsor:
sigir
sigir

The 34th ACM International Conference on Information and Knowledge Management

November 10 - 14, 2025

Seoul , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

67
Total Citations
View Citations
1,051
Total Downloads

Downloads (Last 12 months)27
Downloads (Last 6 weeks)4

Reflects downloads up to 01 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Datta SGanguly DMacAvaney SGreene D(2024)A Deep Learning Approach for Selective Relevance FeedbackAdvances in Information Retrieval10.1007/978-3-031-56060-6_13(189-204)Online publication date: 16-Mar-2024
https://doi.org/10.1007/978-3-031-56060-6_13
Ye ZXie XAi QLiu YWang ZSu WZhang M(2023)Relevance Feedback with Brain SignalsACM Transactions on Information Systems10.1145/363787442:4(1-37)Online publication date: 18-Dec-2023
https://dl.acm.org/doi/10.1145/3637874
Capatina AMicu AMicu ARibeiro‐Navarrete S(2023)Knowledge maps for large‐scale group decision making in social media content analysisExpert Systems10.1111/exsy.1350942:1Online publication date: 23-Nov-2023
https://doi.org/10.1111/exsy.13509
Bhagawati RSubramanian T(2023)An approach of a quantum-inspired document ranking algorithm by using feature selection methodologyInternational Journal of Information Technology10.1007/s41870-023-01543-w15:8(4041-4053)Online publication date: 8-Oct-2023
https://doi.org/10.1007/s41870-023-01543-w
Zhu YPang LLan YShen HCheng XAmigo ECastells PGonzalo JCarterette BCulpepper JKazai G(2022)LoLProceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3477495.3532017(825-836)Online publication date: 6-Jul-2022
https://dl.acm.org/doi/10.1145/3477495.3532017
Shibata KKato M(2022)Active Learning for Efficient Partial Improvement of Learning to RankFrom Born-Physical to Born-Virtual: Augmenting Intelligence in Digital Libraries10.1007/978-3-031-21756-2_11(135-143)Online publication date: 7-Dec-2022
https://doi.org/10.1007/978-3-031-21756-2_11
Soliudeen M(2021)The Relevance of Feedback Mechanisms to Library Databases in Academic LibrariesDigital Solutions and the Case for Africa’s Sustainable Development10.4018/978-1-7998-2967-6.ch008(116-130)Online publication date: 2021
https://doi.org/10.4018/978-1-7998-2967-6.ch008
Arampatzis APeikos GSymeonidis S(2021)Pseudo relevance feedback optimizationInformation Retrieval Journal10.1007/s10791-021-09393-5Online publication date: 25-May-2021
https://doi.org/10.1007/s10791-021-09393-5
Zou JKanoulas E(2020)Towards Question-based High-recall Information RetrievalACM Transactions on Information Systems10.1145/338864038:3(1-35)Online publication date: 18-May-2020
https://dl.acm.org/doi/10.1145/3388640
Tang MChen JChen HXu ZWang YXie MLin J(2020)An ontology-improved vector space model for semantic retrievalThe Electronic Library10.1108/EL-04-2020-0081ahead-of-print:ahead-of-printOnline publication date: 30-Nov-2020
https://doi.org/10.1108/EL-04-2020-0081
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten