Skip to main content

Improving Exposure Allocation in Rankings by Query Generation

  • Conference paper
  • First Online:
Advances in Information Retrieval (ECIR 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14612))

Included in the following conference series:

  • 1131 Accesses

Abstract

Deploying methods that incorporate generated queries in their retrieval process, such as Doc2Query, has been shown to be effective for retrieving the most relevant documents for a user’s query. However, to the best of our knowledge, there has been no work yet on whether generated queries can also be used in the ranking process to achieve other objectives, such as ensuring a fair distribution of exposure in the ranking. Indeed, the amount of exposure that a document is likely to receive depends on the document’s position in the ranking, with lower-ranked documents having a lower probability of being examined by the user. While the utility to users remains the main objective of an Information Retrieval (IR) system, an unfair exposure allocation can lead to lost opportunities and unfair economic impacts for particular societal groups. Therefore, in this work, we conduct a first investigation into whether generating relevant queries can help to fairly distribute the exposure over groups of documents in a ranking. In our work, we build on the effective Doc2Query methods to selectively generate relevant queries for underrepresented groups of documents and use their predicted relevance to the original query in order to re-rank the underexposed documents. Our experiments on the TREC 2022 Fair Ranking Track collection show that using generated queries consistently leads to a fairer allocation of exposure compared to a standard ranking while still maintaining utility.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://github.com/terrierteam/pyterrier_doc2query.

References

  1. Amati, G., Ambrosi, E., Bianchi, M., Gaibisso, C., Gambosi, G.: FUB, IASI-CNR and university of “Tor Vergata” at TREC 2008 blog track (2008)

    Google Scholar 

  2. Amati, G., Van Rijsbergen, C.J.: Probabilistic models of information retrieval based on measuring the divergence from randomness. ACM Trans. Inf. Syst. (TOIS) 20(4), 357–389 (2002)

    Article  Google Scholar 

  3. Biega, A.J., Gummadi, K.P., Weikum, G.: Equity of attention: amortizing individual fairness in rankings. In: Proceedings of SIGIR, pp. 405–414 (2018)

    Google Scholar 

  4. Clark, K., Luong, M.T., Le, Q.V., Manning, C.D.: ELECTRA: pre-training text encoders as discriminators rather than generators. In: Proceedings of ICLR (2019)

    Google Scholar 

  5. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of NAACL (2019)

    Google Scholar 

  6. Ekstrand, M.D., Burke, R., Diaz, F.: Fairness and discrimination in retrieval and recommendation. In: Proceedings of SIGIR, pp. 1403–1404 (2019)

    Google Scholar 

  7. Ekstrand, M.D., McDonald, G., Raj, A., Johnson, I.: Overview of the TREC 2022 fair ranking track. In: Proceedings of TREC 2022 (2022)

    Google Scholar 

  8. Gospodinov, M., MacAvaney, S., Macdonald, C.: Doc2Query–: when less is more. In: Kamps, J., et al. (eds.) ECIR 2023. LNCS, vol. 13981, pp. 414–422. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-28238-6_31

    Chapter  Google Scholar 

  9. Heuss, M., Sarvi, F., de Rijke, M.: Fairness of exposure in light of incomplete exposure estimation. In: Proceedings of SIGIR, pp. 759–769 (2022)

    Google Scholar 

  10. Jaenich, T., McDonald, G., Ounis, I.: ColBERT-FairPRF: towards fair pseudo-relevance feedback in dense retrieval. In: Kamps, J., et al. (eds.) ECIR 2023. LNCS, vol. 13981, pp. 457–465. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-28238-6_36

    Chapter  Google Scholar 

  11. Järvelin, K., Kekäläinen, J.: Cumulated gain-based evaluation of IR techniques. ACM Trans. Inf. Syst. (TOIS) 20(4), 422–446 (2002)

    Article  Google Scholar 

  12. Krovetz, R., Croft, W.B.: Lexical ambiguity and information retrieval. ACM Trans. Inf. Syst. (TOIS) 10(2), 115–141 (1992)

    Article  Google Scholar 

  13. Lin, J., Ma, X., Mackenzie, J., Mallia, A.: On the separation of logical and physical ranking models for text retrieval applications. In: DESIRES, pp. 176–178 (2021)

    Google Scholar 

  14. MacAvaney, S., Nardini, F.M., Perego, R., Tonellotto, N., Goharian, N., Frieder, O.: Expansion via prediction of importance with contextualization. In: SIGIR, pp. 1573–1576 (2020)

    Google Scholar 

  15. Macdonald, C., Tonellotto, N.: Declarative experimentation in information retrieval using PyTerrier. In: Proceedings of ICTIR (2020)

    Google Scholar 

  16. McDonald, G., Macdonald, C., Ounis, I.: Search results diversification for effective fair ranking in academic search. Inf. Retrieval J. 25(1), 1–26 (2022)

    Article  Google Scholar 

  17. Morik, M., Singh, A., Hong, J., Joachims, T.: Controlling fairness and bias in dynamic learning-to-rank. In: Proceedings of SIGIR, pp. 429–438 (2020)

    Google Scholar 

  18. Nogueira, R., Jiang, Z., Pradeep, R., Lin, J.: Document ranking with a pretrained sequence-to-sequence model. In: Proceedings of EMNLP 2020, pp. 708–718 (2020)

    Google Scholar 

  19. Nogueira, R., Lin, J., Epistemic, A.: From doc2query to docTTTTTquery. Online preprint, vol. 6, 2 (2019)

    Google Scholar 

  20. Nogueira, R., Yang, W., Cho, K., Lin, J.: Multi-stage document ranking with BERT. arXiv preprint arXiv:1910.14424 (2019)

  21. Raffel, C., et al.: Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res. 21(1), 5485–5551 (2020)

    MathSciNet  Google Scholar 

  22. Rekabsaz, N., Kopeinik, S., Schedl, M.: Societal biases in retrieved contents: measurement framework and adversarial mitigation of BERT rankers. In: Proceedings of SIGIR, pp. 306–316 (2021)

    Google Scholar 

  23. Rekabsaz, N., Schedl, M.: Do neural ranking models intensify gender bias? In: Proceedings of SIGIR, pp. 2065–2068 (2020)

    Google Scholar 

  24. Robertson, S.E.: The probability ranking principle in IR. J. Doc. 33(4), 294–304 (1977). https://doi.org/10.1108/eb026647

    Article  MathSciNet  Google Scholar 

  25. Robertson, S.E., et al.: Okapi at TREC-3. NIST Special Publication Sp, vol. 109 (1995)

    Google Scholar 

  26. Rocchio, Jr., J, J.: Relevance feedback in information retrieval. The SMART retrieval system: experiments in automatic document processing (1971)

    Google Scholar 

  27. Sarvi, F., Heuss, M., Aliannejadi, M., Schelter, S., de Rijke, M.: Understanding and mitigating the effect of outliers in fair ranking. In: Proceedings of WSDM, pp. 861–869 (2022)

    Google Scholar 

  28. Singh, A., Joachims, T.: Fairness of exposure in rankings. In: Proceedings of KDD (2018)

    Google Scholar 

  29. Usunier, N., Do, V., Dohmatob, E.: Fast online ranking with fairness of exposure. In: Proceedings of FACCT, pp. 2157–2167 (2022)

    Google Scholar 

  30. Wang, X., Macdonald, C., Tonellotto, N., Ounis, I.: ColBERT-PRF: semantic pseudo-relevance feedback for dense passage and document retrieval. ACM Trans. Web 17(1), 1–39 (2023)

    Google Scholar 

  31. Zehlike, M., Castillo, C.: Reducing disparate exposure in ranking: a learning to rank approach. In: Proceedings of the Web Conference 2020, pp. 2849–2855 (2020)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Thomas Jaenich .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Jaenich, T., McDonald, G., Ounis, I. (2024). Improving Exposure Allocation in Rankings by Query Generation. In: Goharian, N., et al. Advances in Information Retrieval. ECIR 2024. Lecture Notes in Computer Science, vol 14612. Springer, Cham. https://doi.org/10.1007/978-3-031-56069-9_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-56069-9_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-56068-2

  • Online ISBN: 978-3-031-56069-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics