research-article

Joint relevance and freshness learning from clickthroughs for news search

Authors:

Evgeniy GabrilovichAuthors Info & Claims

WWW '12: Proceedings of the 21st international conference on World Wide Web

Pages 579 - 588

https://doi.org/10.1145/2187836.2187915

Published: 16 April 2012 Publication History

Abstract

In contrast to traditional Web search, where topical relevance is often the main selection criterion, news search is characterized by the increased importance of freshness. However, the estimation of relevance and freshness, and especially the relative importance of these two aspects, are highly specific to the query and the time when the query was issued. In this work, we propose a unified framework for modeling the topical relevance and freshness, as well as their relative importance, based on click logs. We use click statistics and content analysis techniques to define a set of temporal features, which predict the right mix of freshness and relevance for a given query. Experimental results on both historical click data and editorial judgments demonstrate the effectiveness of the proposed approach.

References

[1]

D. Agarwal, B.-C. Chen, P. Elango, and X. Wang. Click shaping to optimize multiple objectives. In KDD, 2011.

Digital Library

[2]

E. Agichtein, E. Brill, S. Dumais, and R. Ragno. Learning user interaction models for predicting web search result preferences. In SIGIR, 2006.

Digital Library

[3]

R. Baeza-Yates, B. Ribeiro-Neto, et al. Modern information retrieval, volume 463. ACM press New York, 1999.

Digital Library

[4]

S. Brin and L. Page. The anatomy of a large-scale hypertextual web search engine. Computer networks and ISDN systems, 30(1--7):107--117, 1998.

Digital Library

[5]

Z. Cao, T. Qin, T. Liu, M. Tsai, and H. Li. Learning to rank: from pairwise approach to listwise approach. In ICML, pages 129--136. ACM, 2007.

Digital Library

[6]

H. Chernoff and E. Lehmann. The use of maximum likelihood estimates in $\chi$2 tests for goodness of fit. The Annals of Mathematical Statistics, pages 579--586, 1954.

[7]

N. Dai, M. Shokouhi, and B. D. Davison. Learning to rank for freshness and relevance. In SIGIR, pages 95--104, 2011.

Digital Library

[8]

A. Dong, Y. Chang, Z. Zheng, G. Mishne, J. Bai, R. Zhang, K. Buchner, C. Liao, and F. Diaz. Towards recency ranking in web search. In WSDM, pages 11--20, 2010.

Digital Library

[9]

A. Dong, R. Zhang, P. Kolari, J. Bai, F. Diaz, Y. Chang, Z. Zheng, and H. Zha. Time is of the essence: improving recency ranking using twitter data. In WWW, 2010.

Digital Library

[10]

M. Efron and G. Golovchinsky. Estimation methods for ranking recent information. In SIGIR, pages 495--504, 2011.

Digital Library

[11]

H. Fang, T. Tao, and C. Zhai. A formal study of information retrieval heuristics. In SIGIR, 2004.

Digital Library

[12]

K. J\"arvelin and J. Kek\"al\"ainen. Cumulated gain-based evaluation of ir techniques. ACM TOIS, 20(4):422--446, 2002.

Digital Library

[13]

T. Joachims. Optimizing search engines using clickthrough data. In KDD, pages 133--142, 2002.

Digital Library

[14]

T. Joachims, L. Granka, B. Pan, H. Hembrooke, and G. Gay. Accurately interpreting clickthrough data as implicit feedback. In SIGIR, pages 154--161, 2005.

Digital Library

[15]

K. Jones, S. Walker, and S. Robertson. A probabilistic model of information retrieval: development and comparative experiments. Information Processing and Management, 36(6):779--808, 2000.

Digital Library

[16]

N. Kanhabua and K. Nørvåg. Determining time of queries for re-ranking search results. Research and Advanced Technology for Digital Libraries, pages 261--272, 2010.

Digital Library

[17]

A. Kulkarni, J. Teevan, K. Svore, and S. Dumais. Understanding temporal query dynamics. In WSDM, 2011.

Digital Library

[18]

L. Li, W. Chu, J. Langford, and X. Wang. Unbiased offline evaluation of contextual-bandit-based news article recommendation algorithms. In Proceedings of the fourth ACM WSDM '11, pages 297--306, 2011.

Digital Library

[19]

X. Li and W. Croft. Time-based language models. In CIKM, pages 469--475, 2003.

Digital Library

[20]

T. Liu. Learning to rank for information retrieval. Foundations and Trends in Information Retrieval, 3(3):225--331, 2009.

Digital Library

[21]

T. Moon, L. Li, W. Chu, C. Liao, Z. Zheng, and Y. Chang. Online learning for recency search ranking using real-time user feedback. In CIKM, pages 1501--1504, 2010.

Digital Library

[22]

G. Salton and M. McGill. Introduction to modern information retrieval. McGraw-Hill, Inc., 1986.

Digital Library

[23]

K. M. Svore, M. N. Volkovs, and C. J. Burges. Learning to rank with multiple objective functions. In WWW, 2011.

Digital Library

[24]

C. Zhai and J. Lafferty. A study of smoothing methods for language models applied to ad hoc information retrieval. In SIGIR, pages 334--342, 2001.

Digital Library

[25]

Z. Zheng, K. Chen, G. Sun, and H. Zha. A regression framework for learning ranking functions using relative relevance judgments. In SIGIR, pages 287--294, 2007.

Digital Library

Cited By

Usta AAltingovde IOzcan RUlusoy O(2021)Learning to Rank for Educational Search EnginesIEEE Transactions on Learning Technologies10.1109/TLT.2021.307519614:2(211-225)Online publication date: 1-Apr-2021
https://doi.org/10.1109/TLT.2021.3075196
Ali FKhusro S(2021)Content and link-structure perspective of ranking webpagesComputer Science Review10.1016/j.cosrev.2021.10039740:COnline publication date: 1-May-2021
https://dl.acm.org/doi/10.1016/j.cosrev.2021.100397
Chen RAi QJayasinghe GCroft WZhu WTao DCheng XCui PRundensteiner ECarmel DHe QXu Yu J(2019)Correcting for Recency Bias in Job RecommendationProceedings of the 28th ACM International Conference on Information and Knowledge Management10.1145/3357384.3358131(2185-2188)Online publication date: 3-Nov-2019
https://dl.acm.org/doi/10.1145/3357384.3358131
Show More Cited By

Index Terms

Joint relevance and freshness learning from clickthroughs for news search
1. Information systems
  1. Information retrieval
    1. Retrieval models and ranking

Recommendations

Learning to rank for freshness and relevance
SIGIR '11: Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval

Freshness of results is important in modern web search. Failing to recognize the temporal aspect of a query can negatively affect the user experience, and make the search engine appear stale. While freshness and relevance can be closely related for some ...
Ranking Relevance in Yahoo Search
KDD '16: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

Search engines play a crucial role in our daily lives. Relevance is the core problem of a commercial search engine. It has attracted thousands of researchers from both academia and industry and has been studied for decades. Relevance in a modern search ...
Learning to rank code examples for code search engines

Source code examples are used by developers to implement unfamiliar tasks by learning from existing solutions. To better support developers in finding existing solutions, code search engines are designed to locate and rank code examples relevant to user'...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

WWW '12: Proceedings of the 21st international conference on World Wide Web

April 2012

1078 pages

ISBN:9781450312295

DOI:10.1145/2187836

General Chairs:
Alain Mille
Université de Lyon, France
,
Fabien Gandon
INRIA, France
,
Jacques Misselis
HP, France
,
Program Chairs:
Michael Rabinovich
Case Western Reserve University, USA
,
Steffen Staab
University of Koblenz-Landau, Germany

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Univ. de Lyon: Universite de Lyon

In-Cooperation

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 April 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

WWW 2012

Sponsor:

Univ. de Lyon

WWW 2012: 21st World Wide Web Conference 2012

April 16 - 20, 2012

Lyon, France

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
324
Total Downloads

Downloads (Last 12 months)6
Downloads (Last 6 weeks)0

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Usta AAltingovde IOzcan RUlusoy O(2021)Learning to Rank for Educational Search EnginesIEEE Transactions on Learning Technologies10.1109/TLT.2021.307519614:2(211-225)Online publication date: 1-Apr-2021
https://doi.org/10.1109/TLT.2021.3075196
Ali FKhusro S(2021)Content and link-structure perspective of ranking webpagesComputer Science Review10.1016/j.cosrev.2021.10039740:COnline publication date: 1-May-2021
https://dl.acm.org/doi/10.1016/j.cosrev.2021.100397
Chen RAi QJayasinghe GCroft WZhu WTao DCheng XCui PRundensteiner ECarmel DHe QXu Yu J(2019)Correcting for Recency Bias in Job RecommendationProceedings of the 28th ACM International Conference on Information and Knowledge Management10.1145/3357384.3358131(2185-2188)Online publication date: 3-Nov-2019
https://dl.acm.org/doi/10.1145/3357384.3358131
Sánchez PBellogín A(2018)Time-Aware Novelty Metrics for Recommender SystemsAdvances in Information Retrieval10.1007/978-3-319-76941-7_27(357-370)Online publication date: 1-Mar-2018
https://doi.org/10.1007/978-3-319-76941-7_27
Wang HWu QWang HMukhopadhyay SZhai CBertino ECrestani FMostafa JTang JSi LZhou XChang YLi YSondhi P(2016)Learning Hidden Features for Contextual BanditsProceedings of the 25th ACM International on Conference on Information and Knowledge Management10.1145/2983323.2983847(1633-1642)Online publication date: 24-Oct-2016
https://dl.acm.org/doi/10.1145/2983323.2983847
Maksai AGarcin FFaltings BWerthner HZanker MGolbeck JSemeraro G(2015)Predicting Online Performance of News Recommender Systems Through Richer Evaluation MetricsProceedings of the 9th ACM Conference on Recommender Systems10.1145/2792838.2800184(179-186)Online publication date: 16-Sep-2015
https://dl.acm.org/doi/10.1145/2792838.2800184
Smith VConnor MStanton ICao LZhang CJoachims TWebb GMargineantu DWilliams G(2015)Going In-DepthProceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining10.1145/2783258.2788599(2109-2118)Online publication date: 10-Aug-2015
https://dl.acm.org/doi/10.1145/2783258.2788599
Bambia MFaiz R(2015)FRel: A Freshness Language Model for Optimizing Real-Time Web SearchIntelligent Systems in Cybernetics and Automation Theory10.1007/978-3-319-18503-3_21(207-216)Online publication date: 2015
https://doi.org/10.1007/978-3-319-18503-3_21

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten