tutorial

Gender Fairness in Information Retrieval Systems

Authors:

Negar Arabzadeh,

Shirin SeyedSalehi,

Morteza Zihayat,

Ebrahim BagheriAuthors Info & Claims

SIGIR '22: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval

Pages 3436 - 3439

https://doi.org/10.1145/3477495.3532680

Published: 07 July 2022 Publication History

Abstract

Recent studies have shown that it is possible for stereotypical gender biases to find their way into representational and algorithmic aspects of retrieval methods; hence, exhibit themselves in retrieval outcomes. In this tutorial, we inform the audience of various studies that have systematically reported the presence of stereotypical gender biases in Information Retrieval (IR) systems. We further classify existing work on gender biases in IR systems as being related to (1) relevance judgement datasets, (2) structure of retrieval methods, and (3) representations learnt for queries and documents. We present how each of these components can be impacted by or cause intensified biases during retrieval. Based on these identified issues, we then present a collection of approaches from the literature that have discussed how such biases can be measured, controlled, or mitigated. Additionally, we introduce publicly available datasets that are often used for investigating gender biases in IR systems as well as evaluation methodology adopted for determining the utility of gender bias mitigation strategies.

References

[1]

Leif Azzopardi. 2021. Cognitive biases in search: a review and reflection of cognitive biases in Information Retrieval. In Proceedings of the 2021 conference on human information interaction and retrieval. 27--37.

Digital Library

[2]

Ricardo Baeza-Yates. 2018. Bias on the web. Commun. ACM 61, 6 (2018), 54--61.

Digital Library

[3]

Ricardo Baeza-Yates. 2020. Bias in search and recommender systems. In Fourteenth ACM Conference on Recommender Systems. 2--2.

Digital Library

[4]

Ebrahim Bagheri, Faezeh Ensan, and Feras Al-Obeidat. 2018. Neural word and entity embeddings for ad hoc retrieval. Information Processing & Management 54, 4 (2018), 657--673.

Digital Library

[5]

Christine Basta, Marta R Costa-Jussà, and Noe Casas. 2019. Evaluating the underlying gender bias in contextualized word embeddings. arXiv preprint arXiv:1904.08783 (2019).

[6]

Amin Bigdeli, Negar Arabzadeh, Shirin Seyedsalehi, Morteza Zihayat, and Ebrahim Bagheri. 2022. A Light-weight Strategy for Restraining Gender Biases in Neural Rankers. In European Conference on Information Retrieval (ECIR2022). Springer.

Digital Library

[7]

Amin Bigdeli, Negar Arabzadeh, Shirin Seyersalehi, Morteza Zihayat, and Ebrahim Bagheri. 2021. On the Orthogonality of Bias and Utility in Ad hoc Retrieval. In Proceedings of the 44rd International ACM SIGIR Conference on Research and Development in Information Retrieval.

Digital Library

[8]

Amin Bigdeli, Negar Arabzadeh, Morteza Zihayat, and Ebrahim Bagheri. 2021. Exploring Gender Biases in Information Retrieval Relevance Judgement Datasets. In European Conference on Information Retrieval. Springer, 216--224.

[9]

Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. Advances in neural information processing systems 29 (2016).

[10]

Shikha Bordia and Samuel R Bowman. 2019. Identifying and reducing gender bias in word-level language models. arXiv preprint arXiv:1904.03035 (2019).

[11]

Marc-Etienne Brunet, Colleen Alkalay-Houlihan, Ashton Anderson, and Richard Zemel. 2019. Understanding the origins of bias in word embeddings. In International Conference on Machine Learning. PMLR, 803--811.

[12]

Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science 356, 6334 (2017), 183--186.

[13]

Tim Draws, Nava Tintarev, Ujwal Gadiraju, Alessandro Bozzon, and Benjamin Timmermans. 2021. This Is Not What We Ordered: Exploring Why Biased Search Result Rankings Affect User Attitudes on Debated Topics. Association for Computing Machinery, New York, NY, USA, 295--305. https://doi.org/10.1145/3404835.3462851

Digital Library

[14]

Michael D Ekstrand, Anubrata Das, Robin Burke, and Fernando Diaz. 2021. Fairness in Information Access Systems. arXiv preprint arXiv:2105.05779 (2021).

[15]

Alessandro Fabris, Alberto Purpura, Gianmaria Silvello, and Gian Antonio Susto. 2020. Gender stereotype reinforcement: Measuring the gender bias conveyed by ranking algorithms. Information Processing & Management 57, 6 (2020), 102377.

[16]

Joel Escudé Font and Marta R Costa-Jussa. 2019. Equalizing gender biases in neural machine translation with word embeddings techniques. arXiv preprint arXiv:1901.03116 (2019).

[17]

Emma J Gerritse, Faegheh Hasibi, and Arjen P de Vries. 2020. Bias in Conversational Search: The Double-Edged Sword of the Personalized Knowledge Graph. In Proceedings of the 2020 ACM SIGIR on International Conference on Theory of Information Retrieval. 133--136.

Digital Library

[18]

Anja Klasnja, Negar Arabzadeh, Mahbod Mehrvarz, and Ebrahim Bagheri. 2022. On the Characteristics of Ranking-based Gender Bias Measures. In WebSci'22 (2022-03--30) (The 14th International ACM Conference on Web Science in 2022 (WebSci'22), 26 -- 29, June, 2022, Universitat Pompeu Fabra, Barcelona, Spain).

Digital Library

[19]

Klara Krieg, Emilia Parada-Cabaleiro, Gertraud Medicus, Oleg Lesota, Markus Schedl, and Navid Rekabsaz. 2022. Grep-BiasIR: A Dataset for Investigating Gender Representation-Bias in Information Retrieval Results. arXiv preprint arXiv:2201.07754 (2022).

[20]

Klara Krieg, Emilia Parada-Cabaleiro, Markus Schedl, and Navid Rekabsaz. 2022. Do Perceived Gender Biases in Retrieval Results Affect Relevance Judgements? arXiv preprint arXiv:2203.01731 (2022).

[21]

Juhi Kulshrestha, Motahhare Eslami, Johnnatan Messias, Muhammad Bilal Zafar, Saptarshi Ghosh, Krishna P Gummadi, and Karrie Karahalios. 2017. Quantifying search bias: Investigating sources of bias for political searches in social media. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing. 417--432.

Digital Library

[22]

Haochen Liu, Jamell Dacon, Wenqi Fan, Hui Liu, Zitao Liu, and Jiliang Tang. 2019. Does gender matter? towards fairness in dialogue systems. arXiv preprint arXiv:1910.10486 (2019).

[23]

Haochen Liu, Wentao Wang, Yiqi Wang, Hui Liu, Zitao Liu, and Jiliang Tang. 2020. Mitigating gender bias for neural dialogue generation with adversarial learning. arXiv preprint arXiv:2009.13028 (2020).

[24]

Kaiji Lu, Piotr Mardziel, Fangjing Wu, Preetam Amancharla, and Anupam Datta. 2020. Gender bias in neural natural language processing. In Logic, Language, and Security. Springer, 189--202.

[25]

Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schütze. 2008. Introduction to information retrieval. Cambridge University Press.

[26]

Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. MS MARCO: A human generated machine reading comprehension dataset. In CoCo@ NIPS.

[27]

Rodrigo Nogueira and Kyunghyun Cho. 2019. Passage Re-ranking with BERT. arXiv preprint arXiv:1901.04085 (2019).

[28]

Alexandra Olteanu, Jean Garcia-Gathright, Maarten de Rijke, Michael D Ekstrand, Adam Roegiest, Aldo Lipani, Alex Beutel, Alexandra Olteanu, Ana Lucic, Ana-Andreea Stoica, et al. 2021. FACTS-IR: fairness, accountability, confidentiality, transparency, and safety in information retrieval. In ACM SIGIR Forum, Vol. 53. ACM New York, NY, USA, 20--43.

[29]

Flavien Prost, Nithum Thain, and Tolga Bolukbasi. 2019. Debiasing embeddings for reduced gender bias in text classification. arXiv preprint arXiv:1908.02810 (2019).

[30]

Navid Rekabsaz, Simone Kopeinik, and Markus Schedl. 2021. Societal Biases in Retrieved Contents: Measurement Framework and Adversarial Mitigation for BERT Rankers. arXiv preprint arXiv:2104.13640 (2021).

[31]

Navid Rekabsaz and Markus Schedl. 2020. Do Neural Ranking Models Intensify Gender Bias?. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2065--2068.

Digital Library

[32]

Shirin SeyedSalehi, Amin Bigdeli, Negar Arabzadeh, Bhaskar Mitra, Morteza Zihayat, and Ebrahim Bagheri. 2022. Bias-aware Fair Neural Ranking for Addressing Stereotypical Gender Biases. In Extending Database Technology (EDBT2022). Springer.

[33]

Karolina Stanczak and Isabelle Augenstein. 2021. A Survey on Gender Bias in Natural Language Processing. arXiv preprint arXiv:2112.14168 (2021).

[34]

Tony Sun, Andrew Gaut, Shirlyn Tang, Yuxin Huang, Mai ElSherief, Jieyu Zhao, Diba Mirza, Elizabeth Belding, Kai-Wei Chang, and William Yang Wang. 2019. Mitigating gender bias in natural language processing: Literature review. arXiv preprint arXiv:1906.08976 (2019).

[35]

Jialu Wang, Yang Liu, and Xin Eric Wang. 2021. Are Gender-Neutral Queries Really Gender-Neutral? Mitigating Gender Bias in Image Search. arXiv preprint arXiv:2109.05433 (2021).

[36]

Zekun Yang and Juan Feng. 2020. A causal inference method for reducing gender bias in word embedding relations. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 9434--9441.

[37]

Jieyu Zhao, Subhabrata Mukherjee, Saghar Hosseini, Kai-Wei Chang, and Ahmed Hassan Awadallah. 2020. Gender bias in multilingual embeddings and cross-lingual transfer. arXiv preprint arXiv:2005.00699 (2020).

[38]

Jieyu Zhao, Tianlu Wang, Mark Yatskar, Ryan Cotterell, Vicente Ordonez, and Kai-Wei Chang. 2019. Gender bias in contextualized word embeddings. arXiv preprint arXiv:1904.03310 (2019).

[39]

Jieyu Zhao, Yichao Zhou, Zeyu Li,WeiWang, and Kai-Wei Chang. 2018. Learning gender-neutral word embeddings. arXiv preprint arXiv:1809.01496 (2018).

Cited By

Yang TXu ZAi Q(2023)Vertical Allocation-based Fair Exposure Amortizing in RankingProceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region10.1145/3624918.3625313(234-244)Online publication date: 26-Nov-2023
https://dl.acm.org/doi/10.1145/3624918.3625313
Yang TXu ZWang ZAi QFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)FARA: Future-aware Ranking Algorithm for Fairness OptimizationProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3614877(2906-2916)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3614877
Asyrofi RDewi MLutfhi MWibowo P(2023)Systematic Literature Review Langchain Proposed2023 International Electronics Symposium (IES)10.1109/IES59143.2023.10242497(533-537)Online publication date: 8-Aug-2023
https://doi.org/10.1109/IES59143.2023.10242497
Show More Cited By

Index Terms

Gender Fairness in Information Retrieval Systems
1. Information systems
  1. Information retrieval
    1. Evaluation of retrieval results
      1. Presentation of retrieval results

Recommendations

Understanding and Mitigating Gender Bias in Information Retrieval Systems
Advances in Information Retrieval
Abstract
Recent studies have shown that information retrieval systems may exhibit stereotypical gender biases in outcomes which may lead to discrimination against minority groups, such as different genders, and impact users’ decision making and judgements. ...
Analyzing the Influence of Bigrams on Retrieval Bias and Effectiveness
ICTIR '20: Proceedings of the 2020 ACM SIGIR on International Conference on Theory of Information Retrieval

Prior work on using retrievability measures in the evaluation of information retrieval (IR) systems has laid out the foundations for investigating the relationship between retrieval effectiveness and retrieval bias. While various factors influencing ...
Bias and Unfairness in Information Retrieval Systems: New Challenges in the LLM Era
KDD '24: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

With the rapid advancements of large language models (LLMs), information retrieval (IR) systems, such as search engines and recommender systems, have undergone a significant paradigm shift. This evolution, while heralding new opportunities, introduces ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR '22: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 2022

3569 pages

ISBN:9781450387323

DOI:10.1145/3477495

General Chairs:
Enrique Amigo
UNED
,
Pablo Castells
UAM and Amazon
,
Julio Gonzalo
UNED
,
Program Chairs:
Ben Carterette
Spotify
,
J. Shane Culpepper
RMIT University
,
Gabriella Kazai
Waseda University

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 July 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Tutorial

Conference

SIGIR '22

Sponsor:

SIGIR

SIGIR '22: The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 11 - 15, 2022

Madrid, Spain

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
419
Total Downloads

Downloads (Last 12 months)97
Downloads (Last 6 weeks)8

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Yang TXu ZAi Q(2023)Vertical Allocation-based Fair Exposure Amortizing in RankingProceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region10.1145/3624918.3625313(234-244)Online publication date: 26-Nov-2023
https://dl.acm.org/doi/10.1145/3624918.3625313
Yang TXu ZWang ZAi QFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)FARA: Future-aware Ranking Algorithm for Fairness OptimizationProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3614877(2906-2916)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3614877
Asyrofi RDewi MLutfhi MWibowo P(2023)Systematic Literature Review Langchain Proposed2023 International Electronics Symposium (IES)10.1109/IES59143.2023.10242497(533-537)Online publication date: 8-Aug-2023
https://doi.org/10.1109/IES59143.2023.10242497
Fang YLiu HTao ZYurochkin MAl Hasan MXiong L(2022)Fairness of Machine Learning in Search EnginesProceedings of the 31st ACM International Conference on Information & Knowledge Management10.1145/3511808.3557501(5132-5135)Online publication date: 17-Oct-2022
https://dl.acm.org/doi/10.1145/3511808.3557501

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten