skip to main content
research-article

"Learning to rank for information retrieval from user interactions" by K. Hofmann, S. Whiteson, A. Schuth, and M. de Rijke with Martin Vesely as coordinator

Published: 01 April 2014 Publication History

Abstract

In this article we give an overview of our recent work on online learning to rank for information retrieval (IR). This work addresses IR from a reinforcement learning (RL) point of view, with the aim to enable systems that can learn directly from interactions with their users. Learning directly from user interactions is difficult for several reasons. First, user interactions are hard to interpret as feedback for learning because it is usually biased and noisy. Second, the system can only observe feedback on actions (e.g., rankers, documents) actually shown to users, which results in an exploration-exploitation challenge. Third, the amount of feedback and therefore the quality of learning is limited by the number of user interactions, so it is important to use the observed data as effectively as possible. Here, we discuss our work on interpreting user feedback using probabilistic interleaved comparisons, and on learning to rank from noisy, relative feedback.

References

[1]
CHAPELLE, O., JOACHIMS, T., RADLINSKI, F., AND YUE, Y. 2012. Large-scale validation and analysis of interleaved search evaluation. ACM Transactions on Information Systems 30, 1, 1--41.
[2]
HE, J., ZHAI, C., AND LI, X. 2009. Evaluation of methods for relative comparison of retrieval systems based on clickthroughs. In CIKM '09. ACM Press, 2029--2032.
[3]
HOFMANN, K., BEHR, F., AND RADLINSKI, F. 2012. On caption bias in interleaving experiments. In CIKM '12. ACM Press, 115--124.
[4]
HOFMANN, K., SCHUTH, A.,WHITESON, S., AND DE RIJKE, M. 2013. Reusing historical interaction data for faster online learning to rank for ir. In WSDM '13. ACM Press, 183--192.
[5]
HOFMANN, K.,WHITESON, S., AND DE RIJKE, M. 2011a. Balancing exploration and exploitation in learning to rank online. In ECIR '11. Lecture Notes in Computer Science, vol. 6611. Springer, 251--263.
[6]
HOFMANN, K., WHITESON, S., AND DE RIJKE, M. 2011b. A probabilistic method for inferring preferences from clicks. In CIKM '11. ACM Press, 249--258.
[7]
HOFMANN, K.,WHITESON, S., AND DE RIJKE, M. 2012. Estimating interleaved comparison outcomes from historical click data. In CIKM '12. ACM Press, 1779--1783.
[8]
HOFMANN, K.,WHITESON, S., AND DE RIJKE, M. 2013a. Balancing exploration and exploitation in listwise and pairwise online learning to rank for information retrieval. Information Retrieval Journal 16, 1, 63--90.
[9]
HOFMANN, K.,WHITESON, S., AND DE RIJKE, M. 2013b. Fidelity, soundness, and efficiency of interleaved comparison methods. ACM Transactions on Information Systems 31, 4.
[10]
JOACHIMS, T. 2002. Optimizing search engines using clickthrough data. In KDD '02. ACM Press, 133--142.
[11]
JOACHIMS, T. 2003. Evaluating retrieval performance using clickthrough data. Text Mining.
[12]
RADLINSKI, F., KURUP, M., AND JOACHIMS, T. 2008. How does clickthrough data reflect retrieval quality? In
[13]
CIKM '08. ACM Press, 43--52.
[14]
SCHUTH, A., HOFMANN, K., WHITESON, S., AND DE RIJKE, M. 2013. Lerot: An online learning to rank framework. In Living Lab '13: Workshop on Living Labs for Information Retrieval Evaluation. ACM Press.
[15]
YUE, Y. AND JOACHIMS, T. 2009. Interactively optimizing information retrieval systems as a dueling bandits problem. In ICML '09. ACM Press, 1201--1208.

Cited By

View all
  • (2018)Learning to lurker rank: an evaluation of learning-to-rank methods for lurking behavior analysisSocial Network Analysis and Mining10.1007/s13278-018-0516-z8:1Online publication date: 1-Jun-2018

Index Terms

  1. "Learning to rank for information retrieval from user interactions" by K. Hofmann, S. Whiteson, A. Schuth, and M. de Rijke with Martin Vesely as coordinator

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM SIGWEB Newsletter
    ACM SIGWEB Newsletter  Volume 2014, Issue Spring
    Spring 2014
    26 pages
    ISSN:1931-1745
    EISSN:1931-1435
    DOI:10.1145/2591453
    Issue’s Table of Contents
    Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 01 April 2014
    Published in SIGWEB Volume 2014, Issue Spring

    Check for updates

    Qualifiers

    • Research-article

    Funding Sources

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)0
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 16 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2018)Learning to lurker rank: an evaluation of learning-to-rank methods for lurking behavior analysisSocial Network Analysis and Mining10.1007/s13278-018-0516-z8:1Online publication date: 1-Jun-2018

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media