poster

Improve retrieval accuracy for difficult queries using negative feedback

Authors:

Xuanhui Wang,

Hui Fang,

ChengXiang ZhaiAuthors Info & Claims

CIKM '07: Proceedings of the sixteenth ACM conference on Conference on information and knowledge management

Pages 991 - 994

https://doi.org/10.1145/1321440.1321593

Published: 06 November 2007 Publication History

Get Access

Abstract

How to improve search accuracy for difficult topics is an under-addressed, yet important research question. In this paper, we consider a scenario when the search results are so poor that none of the top-ranked documents is relevant to a user's query, and propose to exploit negative feedback to improve retrieval accuracy for such difficult queries. Specifically, we propose to learn from a certain number of top-ranked non-relevant documents to rerank the rest unseen documents. We propose several approaches to penalizing the documents that are similar to the known non-relevant documents in the language modeling framework. To evaluate the proposed methods, we adapt standard TREC collections to construct a test collection containing only difficult queries. Experiment results show that the proposed approaches are effective for improving retrieval accuracy of difficult queries.

References

[1]

M. Dunlop. The effect of accessing non-matching documents on relevance feedback. ACM TOIS, 15(2), 1997.

Digital Library

Google Scholar

[2]

T. Hofmann. Probabilistic latent semantic indexing. In SIGIR, pages 50--57, 1999.

Digital Library

Google Scholar

[3]

J. D. Lafferty and C. Zhai. Document language models, query models, and risk minimization for information retrieval. In SIGIR, pages 111--119, 2001.

Digital Library

Google Scholar

[4]

J. M. Ponte and W. B. Croft. A language modeling approach to information retrieval. In SIGIR, pages 275--281, 1998.

Digital Library

Google Scholar

[5]

J. J. Rocchio. Relevance feedback in information retrieval. In The SMART Retrieval System - Experiments in Automatic Document Processing, pages 313--323, 1971.

Google Scholar

[6]

A. Singhal, M. Mitra, and C. Buckley. Learning rounting queries in a query zone. In SIGIR, pages 25--32, 1997.

Digital Library

Google Scholar

[7]

E. M. Voorhees. Draft: Overview of the trec 2005 robust retrieval track. In Notebook of the Thirteenth Text REtrieval Conference (TREC 2005), 2005.

Crossref

Google Scholar

[8]

C. Zhai and J. Lafferty. Model-based feedback in the KL-divergence retrieval model. In Tenth International Conference on Information and Knowledge Management (CIKM 2001), pages 403--410, 2001.

Digital Library

Google Scholar

[9]

C. Zhai and J. Lafferty. A study of smoothing methods for language models applied to ad hoc information retrieval. In Proceedings of SIGIR'01, pages 334--342, 2001.

Digital Library

Google Scholar

[10]

C. Zhai, A. Velivelli, and B. Yu. A cross-collection mixture model for comparative text minning. In Proceeding of the 10th ACM SIGKDD international conference on Knowledge discovery in data mining, pages 743--748, 2004.

Digital Library

Google Scholar

Cited By

View all

Bi KAi QCroft WHasibi FFang YAizawa A(2021)Asking Clarifying Questions Based on Negative Feedback in Conversational SearchProceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval10.1145/3471158.3472232(157-166)Online publication date: 11-Jul-2021
https://dl.acm.org/doi/10.1145/3471158.3472232
Elvina Mandala R(2020)Improving Effectiveness Information Retrieval System Using Pseudo Irrelevance Feedback2020 IEEE International Conference on Sustainable Engineering and Creative Computing (ICSECC)10.1109/ICSECC51444.2020.9557550(463-468)Online publication date: 16-Dec-2020
https://doi.org/10.1109/ICSECC51444.2020.9557550
Bi KAi QZhang YCroft WZhu WTao DCheng XCui PRundensteiner ECarmel DHe QXu Yu J(2019)Conversational Product Search Based on Negative FeedbackProceedings of the 28th ACM International Conference on Information and Knowledge Management10.1145/3357384.3357939(359-368)Online publication date: 3-Nov-2019
https://dl.acm.org/doi/10.1145/3357384.3357939
Show More Cited By

Index Terms

Improve retrieval accuracy for difficult queries using negative feedback
1. Information systems
  1. Information retrieval
    1. Retrieval models and ranking

Recommendations

Negative Relevance Feedback for Exploratory Search with Visual Interactive Intent Modeling
IUI '17: Proceedings of the 22nd International Conference on Intelligent User Interfaces

In difficult information seeking tasks, the majority of top-ranked documents for an initial query may be non-relevant, and negative relevance feedback may then help find relevant documents. Traditional negative relevance feedback has been studied on ...
A study of methods for negative relevance feedback
SIGIR '08: Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval

Negative relevance feedback is a special case of relevance feedback where we do not have any positive example; this often happens when the topic is difficult and the search results are poor. Although in principle any standard relevance feedback ...
Improving retrieval accuracy of difficult queries through generalizing negative document language models
CIKM '11: Proceedings of the 20th ACM international conference on Information and knowledge management

When a query topic is difficult and the search results are very poor, negative feedback is a very useful method to improve the retrieval accuracy and user experience. One challenge in negative feedback is that negative documents tend to be distracting ...

Comments

Information & Contributors

Information

Published In

CIKM '07: Proceedings of the sixteenth ACM conference on Conference on information and knowledge management

November 2007

1048 pages

ISBN:9781595938039

DOI:10.1145/1321440

Co-chair:
Alberto H. F. Laender,
Conference Chairs:
André O. Falcão
Universidade de Lisboa, Portugal
,
Øystein Haug Olsen,
General Chair:
Mário J. Silva
(Universidade de Lisboa, Portugal)
,
Program Chairs:
Ricardo Baeza-Yates,
Deborah L. McGuinness,
Bjorn Olstad

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 November 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Poster

Conference

CIKM07

Sponsor:

CIKM07: Conference on Information and Knowledge Management

November 6 - 10, 2007

Lisbon, Portugal

Acceptance Rates

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Sponsor:
sigir
sigir

The 34th ACM International Conference on Information and Knowledge Management

November 10 - 14, 2025

Seoul , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

29
Total Citations
View Citations
464
Total Downloads

Downloads (Last 12 months)7
Downloads (Last 6 weeks)0

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Bi KAi QCroft WHasibi FFang YAizawa A(2021)Asking Clarifying Questions Based on Negative Feedback in Conversational SearchProceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval10.1145/3471158.3472232(157-166)Online publication date: 11-Jul-2021
https://dl.acm.org/doi/10.1145/3471158.3472232
Elvina Mandala R(2020)Improving Effectiveness Information Retrieval System Using Pseudo Irrelevance Feedback2020 IEEE International Conference on Sustainable Engineering and Creative Computing (ICSECC)10.1109/ICSECC51444.2020.9557550(463-468)Online publication date: 16-Dec-2020
https://doi.org/10.1109/ICSECC51444.2020.9557550
Bi KAi QZhang YCroft WZhu WTao DCheng XCui PRundensteiner ECarmel DHe QXu Yu J(2019)Conversational Product Search Based on Negative FeedbackProceedings of the 28th ACM International Conference on Information and Knowledge Management10.1145/3357384.3357939(359-368)Online publication date: 3-Nov-2019
https://dl.acm.org/doi/10.1145/3357384.3357939
Wu HLuk RWong KNie J(2019)Binary Independence Language Model in a Relevance Feedback EnvironmentInternational Journal of Software Engineering and Knowledge Engineering10.1142/S021819401950030X29:06(873-895)Online publication date: 25-Jun-2019
https://doi.org/10.1142/S021819401950030X
Peltonen JStrahl JFloréen PPapadopoulos GKuflik TChen FDuarte CFu W(2017)Negative Relevance Feedback for Exploratory Search with Visual Interactive Intent ModelingProceedings of the 22nd International Conference on Intelligent User Interfaces10.1145/3025171.3025222(149-159)Online publication date: 7-Mar-2017
https://dl.acm.org/doi/10.1145/3025171.3025222
Zagheli HAriannezhad MShakery A(2017)Negative Feedback in the Language Modeling Framework for Text RecommendationAdvances in Information Retrieval10.1007/978-3-319-56608-5_63(662-668)Online publication date: 8-Apr-2017
https://doi.org/10.1007/978-3-319-56608-5_63
Dehghani MAbnar SKamps JMukhopadhyay SZhai CBertino ECrestani FMostafa JTang JSi LZhou XChang YLi YSondhi P(2016)The Healing Power of PoisonProceedings of the 25th ACM International on Conference on Information and Knowledge Management10.1145/2983323.2983910(2065-2068)Online publication date: 24-Oct-2016
https://dl.acm.org/doi/10.1145/2983323.2983910
Zhang AGoyal AKong WDeng HDong AChang YGunter CHan JBaeza-Yates RLalmas MMoffat ARibeiro-Neto B(2015)adaQACProceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/2766462.2767697(143-152)Online publication date: 9-Aug-2015
https://dl.acm.org/doi/10.1145/2766462.2767697
Ma YLin H(2014)A Multiple Relevance Feedback Strategy with Positive and Negative ModelsPLoS ONE10.1371/journal.pone.01047079:8(e104707)Online publication date: 19-Aug-2014
https://doi.org/10.1371/journal.pone.0104707
Anne AStephen SDavid D(2012)Extracting Critical Information from Free Text Data for Systems Health ManagementMachine Learning and Knowledge Discovery for Engineering Systems Health Management10.1201/b11580-17(423-450)Online publication date: 9-Mar-2012
https://doi.org/10.1201/b11580-17
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Negative Relevance Feedback for Exploratory Search with Visual Interactive Intent Modeling

A study of methods for negative relevance feedback

Improving retrieval accuracy of difficult queries through generalizing negative document language models

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Upcoming Conference

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations