Skip to main content
Springer Nature Link
Log in

Towards Optimizing Ranking in Grid-Layout for Provider-Side Fairness

  • Conference paper
  • First Online:
Advances in Information Retrieval (ECIR 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14612))

Included in the following conference series:

  • 1089 Accesses

Abstract

Information access systems, such as search engines and recommender systems, order and position results based on their estimated relevance. These results are then evaluated for a range of concerns, including provider-side fairness: whether exposure to users is fairly distributed among items and the people who created them. Several fairness-aware ranking and re-ranking techniques have been proposed to ensure fair exposure for providers, but this work focuses almost exclusively on linear layouts in which items are displayed in single ranked list. Many widely-used systems use other layouts, such as the grid views common in streaming platforms, image search, and other applications. Providing fair exposure to providers in such layouts is not well-studied. We seek to fill this gap by providing a grid-aware re-ranking algorithm to optimize layouts for provider-side fairness by adapting existing re-ranking techniques to grid-aware browsing models, and an analysis of the effect of grid-specific factors such as device size on the resulting fairness optimization. Our work provides a starting point and identifies open gaps in ensuring provider-side fairness in grid-based layouts.

This paper reports work supported by the National Science Foundation under Grant 17-51278 and primarily conducted while the authors were at Boise State University.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    Due to limitations of the underlying data set [40], we are only able to consider binary gender. We understand the potential harm of misrepresentation of gender in research [50]; our methods in this paper are extensible to non-binary gender or other attributes when suitable data is available.

References

  1. Benjamini, Y., Hochberg, Y.: Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. Roy. Stat. Soc.: Ser. B (Methodol.) 57, 289–300 (1995)

    MathSciNet  Google Scholar 

  2. Herlocker, J.L., Konstan, J.A., Borchers, A., Riedl, J.: An algorithmic framework for performing collaborative filtering. In: Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 230–237. Association for Computing Machinery, New York (1999). isbn: 1581130961

    Google Scholar 

  3. Friedman, J.H.: Greedy function approximation: a gradient boosting machine. Ann. Stat. 1189–1232 (2001)

    Google Scholar 

  4. Deshpande, M., Karypis, G.: Item-based Top-n recommendation algorithms. ACM Trans. Inf. Syst. (TOIS) 22, 143–177 (2004)

    Article  Google Scholar 

  5. Burges, C., et al.: Learning to rank using gradient descent. In: Proceedings of the 22nd International Conference on Machine Learning, pp. 89–96 (2005)

    Google Scholar 

  6. Cao, Z., Qin, T., Liu, T.-Y., Tsai, M.-F., Li, H.: Learning to rank: from pairwise approach to listwise approach. In: Proceedings of the 24th International Conference on Machine Learning, pp. 129–136 (2007)

    Google Scholar 

  7. Li, P., Wu, Q., Burges, C.: Mcrank: learning to rank using multiple classification and gradient boosting. Adv. Neural Inf. Process. Syst. 20 (2007)

    Google Scholar 

  8. Shrestha, S., Lenz, K.: Eye gaze patterns while searching vs. browsing a website. Usability News 9, 1–9 (2007)

    Google Scholar 

  9. Tatler, B.W.: The central fixation bias in scene viewing: selecting an optimal viewing position independently of motor biases and image feature distributions. J. Vis. 7, 4–4 (2007)

    Article  Google Scholar 

  10. Xu, J., Li, H.: Adarank: a boosting algorithm for information retrieval. In: Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 391–398 (2007)

    Google Scholar 

  11. Craswell, N., Zoeter, O., Taylor, M., Ramsey, B.: An experimental comparison of click position-bias models. In: Proceedings of the 2008 International Conference on Web Search and Data Mining, pp. 87–94 (2008)

    Google Scholar 

  12. Moffat, A., Zobel, J.: Rank-biased precision for measurement of retrieval effectiveness. ACM Trans. Inf. Syst. (TOIS) 27, 1–27 (2008)

    Article  Google Scholar 

  13. Taylor, M., Guiver, J., Robertson, S., Minka, T.: Softrank: optimizing nonsmooth rank metrics. In: Proceedings of the 2008 International Conference on Web Search and Data Mining, pp. 77–86 (2008)

    Google Scholar 

  14. Xia, F., Liu, T.-Y., Wang, J., Zhang, W., Li, H.: Listwise approach to learning to rank: theory and algorithm. In: Proceedings of the 25th International Conference on Machine Learning, pp. 1192–1199 (2008)

    Google Scholar 

  15. Rendle, S., Freudenthaler, C., Gantner, Z., Schmidt-Thieme, L.: BPR: bayesian personalized ranking from implicit feedback. In: Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence, pp. 452–461. AUAI Press, Montreal, Quebec, Canada (2009), isbn: 9780974903958

    Google Scholar 

  16. Burges, C.J.: From ranknet to lambdarank to lambdamart: an overview. Learning 11, 81 (2010)

    Google Scholar 

  17. Djamasbi, S., Siegel, M., Tullis, T.: Visual hierarchy and viewing behavior: an eye tracking study. In: International Conference on Human-Computer Interaction, pp. 331–340 (2011)

    Google Scholar 

  18. Takács, G., Pilászy, I., Tikk, D.: Applications of the conjugate gradient method for implicit feedback collaborative filtering. In: Proceedings of the Fifth ACM Conference on Recommender Systems, pp. 297–300. Association for Computing Machinery, Chicago (2011). isbn: 9781450306836

    Google Scholar 

  19. Dwork, C., Hardt, M., Pitassi, T., Reingold, O., Zemel, R.: Fairness through awareness. In: Proceedings of the 3rd Innovations in Theoretical Computer Science Conference, pp. 214–226 (2012)

    Google Scholar 

  20. Hsu, H., Lachenbruch, P.A.: Paired t test. Wiley StatsRef: statistics reference online (2014)

    Google Scholar 

  21. Lan, Y., Zhu, Y., Guo, J., Niu, S., Cheng, X.: Position-aware ListMLE: a sequential learning process for ranking. In: UAI, pp. 449–458 (2014)

    Google Scholar 

  22. Zhao, Q., Chang, S., Harper, F.M., Konstan, J.A.: Gaze prediction for recommender systems. In: Proceedings of the 10th ACM Conference on Recommender Systems, pp. 131–138 (2016)

    Google Scholar 

  23. Xie, X., et al. Investigating examination behavior of image search users. In Proceedings of the 40th International ACM Sigir Conference on Research and Development in Information Retrieval, pp. 275–284 (2017)

    Google Scholar 

  24. Yang, K., Stoyanovich, J.: Measuring fairness in ranked outputs. In: Proceedings of the 29th International Conference on Scientific and Statistical Database Management, pp. 1–6 (2017)

    Google Scholar 

  25. Zehlike, M., et al.: FA*IR: a fair top-k ranking algorithm. In: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, 1569–1578. Association for Computing Machinery, Singapore (2017). isbn: 9781450349185. https://doi.org/10.1145/3132847.3132938

  26. Biega, A. J., Gummadi, K.P., Weikum, G.: Equity of attention: amortizing individual fairness in rankings. In: Proceedings of the 41st International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 405–414 (2018)

    Google Scholar 

  27. Ekstrand, M.D., Tian, M., Kazi, M.R.I., Mehrpouyan, H., Kluver, D.: Exploring author gender in book rating and recommendation. In: Proceedings of the 12th ACM Conference on Recommender Systems, pp. 242–250 (2018)

    Google Scholar 

  28. Singh, A., Joachims, T.: Fairness of exposure in rankings. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2219–2228. Association for Computing Machinery, London (2018). isbn: 9781450355520. https://doi.org/10.1145/3219819.3220088

  29. Wan, M., McAuley, J.: Item recommendation on monotonic behavior chains. In: Proceedings of the 12th ACM Conference on Recommender Systems, pp. 86–94. Association for Computing Machinery, Vancouver (2018). isbn: 9781450359016. https://doi.org/10.1145/3240323.3240369

  30. Wu, L., Hsieh, C.-J., Sharpnack, J.: SQL-RANK: a listwise approach to collaborative ranking. In: International Conference on Machine Learning, pp. 5315–5324 (2018)

    Google Scholar 

  31. Beutel, A., et al.: Fairness in recommendation ranking through pairwise comparisons. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2212–2220 (2019)

    Google Scholar 

  32. Biega, A.J., Diaz, F., Ekstrand, M.D., Kohlmeier, S.: Overview of the TREC 2019 fair ranking track. In: The Twenty-Eighth Text REtrieval Conference (TREC 2019) Proceedings (2019)

    Google Scholar 

  33. Geyik, S.C., Ambler, S., Kenthapadi, K.: Fairness-aware ranking in search & recommendation systems with application to linkedin talent search. In: Proceedings of the 25th ACM Sigkdd International Conference on Knowledge Discovery & Data Mining, pp. 2221–2231 (2019)

    Google Scholar 

  34. Liu, W., Guo, J., Sonboli, N., Burke, R., Zhang, S.: Personalized fairnessaware re-ranking for microlending. In: Proceedings of the 13th ACM Conference on Recommender Systems, pp. 467–471 (2019)

    Google Scholar 

  35. Pei, C., et al.: Personalized re-ranking for recommendation. In: Proceedings of the 13th ACM Conference on Recommender Systems, pp. 3–11 (2019)

    Google Scholar 

  36. Sapiezynski, P., Zeng, W., E Robertson, R., Mislove, A., Wilson, C.: Quantifying the impact of user attention on fair group representation in ranked lists. In: Companion Proceedings of The 2019 World Wide Web Conference, pp. 553–562. Association for Computing Machinery, San Francisco (2019). isbn: 9781450366755. https://doi.org/10.1145/3308560.3317595

  37. Xie, X., et al.: Grid-based evaluation metrics for web image search. In: The World Wide Web Conference, pp. 2103–2114 (2019)

    Google Scholar 

  38. Diaz, F., Mitra, B., Ekstrand, M.D., Biega, A.J., Carterette, B.: Evaluating stochastic rankings with expected exposure. In: Proceedings of the 29th ACM International Conference on Information & Knowledge Management, pp. 275–284. Association for Computing Machinery, Virtual Event (2020). isbn: 9781450368599. https://doi.org/10.1145/3340531.3411962

  39. Ekstrand, M.D.: LensKit for python: next-generation software for recommender systems experiments. In: Proceedings of the 29th ACM International Conference on Information & Knowledge Management, pp. 2999–3006. Association for Computing Machinery, Virtual Event (2020). isbn: 9781450368599. https://doi.org/10.1145/3340531.3412778

  40. Ekstrand, M.D., Kluver, D.: Exploring author gender in book rating and recommendation. User Model. User-Adapt. Interact. (2020). https://md.ekstrandom.net/pubs/bag-extended

  41. Jiang, H., Nachum, O.: Identifying and correcting label bias in machine learning. In: International Conference on Artificial Intelligence and Statistics, pp. 702–712 (2020)

    Google Scholar 

  42. Narasimhan, H., Cotter, A., Gupta, M.R., Wang, S.: Pairwise fairness for ranking and regression. In: AAA, vol. I, pp. 5248–5255 (2020)

    Google Scholar 

  43. Raj, A., Wood, C., Montoly, A., Ekstrand, M.D.: Comparing fair ranking metrics (2020). arXiv preprint arXiv:2009.01311

  44. Pitoura, E., Stefanidis, K., Koutrika, G.: Fairness in rankings and recommendations: an overview. VLDB J. 1–28 (2021)

    Google Scholar 

  45. Sonoda, R.: A Pre-processing Method for Fairness in Ranking. arXiv preprint arXiv:2110.15503 (2021)

  46. Chen, S., et al.: Reinforcement Re-ranking with 2D Grid-based Recommendation Panels. arXiv preprint arXiv:2204.04954 (2022)

  47. Ekstrand, M.D., Das, A., Burke, R., Diaz, F., et al.: Fairness in information access systems. Found. Trends® Inf. Retr. 16, 1–177 (2022)

    Google Scholar 

  48. Raj, A., Ekstrand, M.D.: Measuring fairness in ranked results: an analytical and empirical comparison. In: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 726–736 (2022)

    Google Scholar 

  49. Ekstrand, M.D., McDonald, G., Raj, A., Johnson, I.: Overview of the TREC 2022 Fair Ranking Track. arXiv preprint arXiv:2302.05558 (2023)

  50. Pinney, C., Raj, A., Hanna, A., Ekstrand, M.D.: Much ado about gender: current practices and future recommendations for appropriate gender-aware information access. arXiv preprint arXiv:2301.04780 (2023)

  51. Raj, A., Ekstrand, M.: Unified browsing models for linear and grid layouts. arXiv preprint arXiv:2310.12524 (2023)

  52. Raj, A., Ekstrand, M.D.: Towards measuring fairness in grid layout in recommender systems. arXiv preprint arXiv:2309.10271 (2023)

Download references

Author information

Authors and Affiliations

  1. Microsoft, Redmond, USA

    Amifa Raj

  2. Department of Information Science, Drexel University, Philadelphia, USA

    Michael D. Ekstrand

Authors
  1. Amifa Raj
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael D. Ekstrand
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Amifa Raj .

Editor information

Editors and Affiliations

  1. Georgetown University, Washington, WA, USA

    Nazli Goharian

  2. University of Pisa, PISA, Pisa, Italy

    Nicola Tonellotto

  3. King's College London, London, UK

    Yulan He

  4. University College London, London, UK

    Aldo Lipani

  5. University of Glasgow, Glasgow, UK

    Graham McDonald

  6. University of Glasgow, Glasgow, UK

    Craig Macdonald

  7. University of Glasgow, Glasgow, UK

    Iadh Ounis

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Raj, A., Ekstrand, M.D. (2024). Towards Optimizing Ranking in Grid-Layout for Provider-Side Fairness. In: Goharian, N., et al. Advances in Information Retrieval. ECIR 2024. Lecture Notes in Computer Science, vol 14612. Springer, Cham. https://doi.org/10.1007/978-3-031-56069-9_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-56069-9_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-56068-2

  • Online ISBN: 978-3-031-56069-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation