research-article

A Critical Survey on Fairness Benefits of Explainable AI

Authors:

Jakob Schoeffer,

Maria De-Arteaga,

Niklas KühlAuthors Info & Claims

FAccT '24: Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency

Pages 1579 - 1595

https://doi.org/10.1145/3630106.3658990

Published: 05 June 2024 Publication History

Abstract

In this critical survey, we analyze typical claims on the relationship between explainable AI (XAI) and fairness to disentangle the multidimensional relationship between these two concepts. Based on a systematic literature review and a subsequent qualitative content analysis, we identify seven archetypal claims from 175 scientific articles on the alleged fairness benefits of XAI. We present crucial caveats with respect to these claims and provide an entry point for future discussions around the potentials and limitations of XAI for specific fairness desiderata. Importantly, we notice that claims are often (i) vague and simplistic, (ii) lacking normative grounding, or (iii) poorly aligned with the actual capabilities of XAI. We suggest to conceive XAI not as an ethical panacea but as one of many tools to approach the multidimensional, sociotechnical challenge of algorithmic fairness. Moreover, when making a claim about XAI and fairness, we emphasize the need to be more specific about what kind of XAI method is used, which fairness desideratum it refers to, how exactly it enables fairness, and who is the stakeholder that benefits from XAI.

References

[1]

Behnoush Abdollahi and Olfa Nasraoui. 2018. Transparency in fair machine learning: The case of explainable recommender systems. Human and Machine Learning: Visible, Explainable, Trustworthy and Transparent (2018), 21–35.

[2]

Amina Adadi and Mohammed Berrada. 2018. Peeking inside the black-box: A survey on explainable artificial intelligence (XAI). IEEE Access 6 (2018), 52138–52160.

[3]

Aniya Aggarwal, Pranay Lohia, Seema Nagar, Kuntal Dey, and Diptikalyan Saha. 2019. Black box fairness testing of machine learning models. In Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. ACM.

Digital Library

[4]

Sina Aghaei, Mohammad Javad Azizi, and Phebe Vayanos. 2019. Learning optimal and fair decision trees for non-discriminative decision-making. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 1418–1426.

Digital Library

[5]

Yongsu Ahn and Yu-Ru Lin. 2020. FairSight: Visual analytics for fairness in decision making. IEEE Transactions on Visualization and Computer Graphics 26, 1 (2020), 1086–1095.

[6]

Ulrich Aïvodji, Hiromi Arai, Olivier Fortineau, Sébastien Gambs, Satoshi Hara, and Alain Tapp. 2019. Fairwashing: The risk of rationalization. In Proceedings of the 36th International Conference on Machine Learning,. 161–170.

[7]

Ulrich Aïvodji, Hiromi Arai, Sébastien Gambs, and Satoshi Hara. 2021. Characterizing the risk of fairwashing. In Advances in Neural Information Processing Systems, Vol. 34. Curran Associates, Inc, 14822–14834.

[8]

Ulrich Aïvodji, Julien Ferry, Sébastien Gambs, Marie-José Huguet, and Mohamed Siala. 2021. FairCORELS, an open-source library for learning fair rule lists. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 4665–4669.

Digital Library

[9]

Kiana Alikhademi, Brianna Richardson, Emma Drobina, and Juan E Gilbert. 2021. Can explainable AI explain unfairness? A framework for evaluating explainable AI. arXiv preprint arXiv:2106.07483 (2021).

[10]

Yasmeen Alufaisan, Murat Kantarcioglu, and Yan Zhou. 2021. Robust transparency against model inversion attacks. IEEE Transactions on Dependable and Secure Computing 18, 5 (2021), 2061–2073.

Digital Library

[11]

Guilherme Alves, Maxime Amblard, Fabien Bernier, Miguel Couceiro, and Amedeo Napoli. 2021. Reducing unintended bias of ML models on tabular and textual data. In 2021 IEEE 8th International Conference on Data Science and Advanced Analytics (DSAA). IEEE, 1–10.

[12]

Guilherme Alves, Vaishnavi Bhargava, Fabien Bernier, Miguel Couceiro, and Amedeo Napoli. 2020. FixOut: An ensemble approach to fairer models. https://hal.archives-ouvertes.fr/hal-03033181/

[13]

Guilherme Alves, Vaishnavi Bhargava, Miguel Couceiro, and Amedeo Napoli. 2021. Making ML models fairer through explanations: The case of LimeOut. In Analysis of Images, Social Networks and Texts: 9th International Conference, AIST 2020, Skolkovo, Moscow, Russia, October 15–16, 2020, Revised Selected Papers 9. Springer, 3–18.

[14]

Christopher Anders, Plamen Pasliev, Ann-Kathrin Dombrowski, Klaus-Robert Müller, and Pan Kessel. 2020. Fairwashing explanations with off-manifold detergent. International Conference on Machine Learning (2020), 314–323.

[15]

Christopher J Anders, Leander Weber, David Neumann, Wojciech Samek, Klaus-Robert Müller, and Sebastian Lapuschkin. 2022. Finding and removing Clever Hans: Using explanation methods to debug and improve deep models. Information Fusion 77 (2022), 261–295.

Digital Library

[16]

Alessa Angerschmid, Jianlong Zhou, Kevin Theuermann, Fang Chen, and Andreas Holzinger. 2022. Fairness and explanation in AI-informed decision making. Machine Learning and Knowledge Extraction 4, 2 (2022), 556–579.

[17]

Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. 2022. Machine bias. In Ethics of data and analytics. Auerbach Publications, 254–264.

[18]

Ariful Islam Anik and Andrea Bunt. 2021. Data-centric explanations: Explaining training data of machine learning systems to promote transparency. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–13.

Digital Library

[19]

Falaah Arif Khan, Eleni Manis, and Julia Stoyanovich. 2022. Towards substantive conceptions of algorithmic fairness: Normative guidance from equal opportunity doctrines. In Proceedings of the 2nd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization. 1–10.

Digital Library

[20]

Alejandro Barredo Arrieta, Natalia Díaz-Rodríguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador García, Sergio Gil-López, Daniel Molina, Richard Benjamins, 2020. Explainable artificial intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information Fusion 58 (2020), 82–115.

Digital Library

[21]

Nicholas Asher, Lucas de Lara, Soumya Paul, and Chris Russell. 2022. Counterfactual models for fair and adequate explanations. Machine Learning and Knowledge Extraction 4, 2 (2022), 316–349.

[22]

Aparna Balagopalan, Haoran Zhang, Kimia Hamidieh, Thomas Hartvigsen, Frank Rudzicz, and Marzyeh Ghassemi. 2022. The road to explainability is paved with bias: Measuring the fairness of explanations. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 1194–1206.

Digital Library

[23]

Esma Balkir, Svetlana Kiritchenko, Isar Nejadgholi, and Kathleen C. Fraser. 2022. Challenges in applying explainability methods to improve the fairness of NLP models. In Proceedings of the 2nd Workshop on Trustworthy Natural Language Processing (TrustNLP 2022).

[24]

Solon Barocas, Moritz Hardt, and Arvind Narayanan. 2023. Fairness and machine learning: Limitations and opportunities. MIT Press.

[25]

Kevin Baum, Susanne Mantel, Eva Schmidt, and Timo Speith. 2022. From responsibility to reason-giving explainable artificial intelligence. Philosophy & Technology 35, 1 (2022), 12.

[26]

Tom Begley, Tobias Schwedes, Christopher Frye, and Ilya Feige. 2020. Explainability for fair machine learning. arXiv preprint arXiv:2010.07389 (2020).

[27]

Janelle Berscheid and Francois Roewer-Despres. 2019. Beyond transparency: A proposed framework for accountability in decision-making AI systems. AI Matters 5, 2 (2019), 13–22.

Digital Library

[28]

Vaishnavi Bhargava, Miguel Couceiro, and Amedeo Napoli. 2020. LimeOut: An ensemble approach to improve process fairness. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 475–491.

[29]

Umang Bhatt, Javier Antorán, Yunfeng Zhang, Q Vera Liao, Prasanna Sattigeri, Riccardo Fogliato, Gabrielle Melançon, Ranganath Krishnan, Jason Stanley, Omesh Tickoo, 2021. Uncertainty as a form of transparency: Measuring, communicating, and using uncertainty. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society. 401–413.

Digital Library

[30]

Reuben Binns. 2018. Fairness in machine learning: Lessons from political philosophy. In Conference on Fairness, Accountability and Transparency. PMLR, 149–159.

[31]

Reuben Binns. 2020. On the apparent conflict between individual and group fairness. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 514–524.

Digital Library

[32]

Reuben Binns. 2022. AI and the human in the loophole. RPA Humane AI UvA [Seminar presentation].

[33]

Reuben Binns, Max Van Kleek, Michael Veale, Ulrik Lyngs, Jun Zhao, and Nigel Shadbolt. 2018. ‘It’s reducing a human being to a percentage’; perceptions of justice in algorithmic decisions. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–14.

Digital Library

[34]

Emily Black, Manish Raghavan, and Solon Barocas. 2022. Model multiplicity: Opportunities, concerns, and solutions. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 850–863.

Digital Library

[35]

Emily Black, Samuel Yeom, and Matt Fredrikson. 2020. FlipTest: Fairness testing via optimal transport. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 111–121.

Digital Library

[36]

Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. 2020. Language (technology) is power: A critical survey of “bias” in NLP. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 5454–5476.

[37]

Pol Borrellas and Irene Unceta. 2021. The challenges of machine learning and their economic implications. Entropy 23, 3 (2021), 1–23.

[38]

Jenna Burrell. 2016. How the machine ‘thinks’: Understanding opacity in machine learning algorithms. Big Data & Society 3, 1 (2016), 1–11.

[39]

Roberta Calegari, Giovanni Ciatto, Enrico Denti, and Andrea Omicini. 2020. Logic-based technologies for intelligent systems: State of the art and perspectives. Information 11, 3 (2020), 167.

[40]

Roberta Calegari, Giovanni Ciatto, and Andrea Omicini. 2020. On the integration of symbolic and sub-symbolic techniques for XAI: A survey. Intelligenza Artificiale 14, 1 (2020), 7–32.

[41]

Diogo V Carvalho, Eduardo M Pereira, and Jaime S Cardoso. 2019. Machine learning interpretability: A survey on methods and metrics. Electronics 8, 8 (2019), 832.

[42]

Alessandro Castelnovo, Riccardo Crupi, Greta Greco, Daniele Regoli, Ilaria Giuseppina Penco, and Andrea Claudio Cosentini. 2022. A clarification of the nuances in the fairness metrics landscape. Scientific Reports 12, 1 (2022), 4209.

[43]

Alessandro Castelnovo, Lorenzo Malandri, Fabio Mercorio, Mario Mezzanzanica, and Andrea Cosentini. 2021. Towards fairness through time. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 647–663.

[44]

Corinne Cath. 2018. Governing artificial intelligence: Ethical, legal and technical opportunities and challenges. Philosophical Transactions. Series A, Mathematical, Physical, and Engineering Sciences 376, 2133 (2018), 1–8.

[45]

Simon Caton and Christian Haas. 2020. Fairness in machine learning: A survey. Comput. Surveys (2020).

[46]

Juliana Cesaro and Fabio Gagliardi Cozman. 2019. Measuring unfairness through game-theoretic interpretability. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 253–264.

[47]

Joymallya Chakraborty, Kewen Peng, and Tim Menzies. 2020. Making fair ML software using trustworthy explanation. In Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering. ACM, 1229–1233.

Digital Library

[48]

Gary K. Y. Chan. 2022. AI employment decision-making: Integrating the equal opportunity merit principle and explainable AI. AI & SOCIETY (2022), 1–12.

[49]

Silvia Chiappa. 2019. Path-specific counterfactual fairness. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 7801–7808.

Digital Library

[50]

Alexandra Chouldechova. 2017. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big Data 5, 2 (2017), 153–163.

[51]

Michael Chromik, Malin Eiband, Sarah Theres Völkel, and Daniel Buschek. 2019. Dark patterns of explainability, transparency, and user control for intelligent systems. In IUI Workshops, Vol. 2327.

[52]

Ylona Chun Tie, Melanie Birks, and Karen Francis. 2019. Grounded theory research: A design framework for novice researchers. SAGE Open Medicine 7 (2019), 2050312118822927.

[53]

Yeounoh Chung, Tim Kraska, Neoklis Polyzotis, Ki Hyun Tae, and Steven Euijong Whang. 2019. Automated data slicing for model validation: A big data-AI integration approach. IEEE Transactions on Knowledge and Data Engineering 32, 12 (2019), 2284–2296.

[54]

Nathan Colaner. 2022. Is explainable artificial intelligence intrinsically valuable?AI & SOCIETY 37, 1 (2022), 231–238.

[55]

Jason A. Colquitt. 2001. On the dimensionality of organizational justice: A construct validation of a measure. The Journal of Applied Psychology 86, 3 (2001), 386–400.

[56]

Giandomenico Cornacchia, Fedelucio Narducci, and Azzurra Ragone. 2021. A general model for fair and explainable recommendation in the loan domain. In Joint Workshop Proceedings of the 3rd Edition of Knowledge-aware and Conversational Recommender Systems (KaRS) and the 5th Edition of Recommendation in Complex Environments (ComplexRec) co-located with 15th ACM Conference on Recommender Systems (RecSys 2021).

[57]

Enyan Dai, Tianxiang Zhao, Huaisheng Zhu, Junjie Xu, Zhimeng Guo, Hui Liu, Jiliang Tang, and Suhang Wang. 2022. A comprehensive survey on trustworthy graph neural networks: Privacy, robustness, fairness, and explainability. arXiv preprint arXiv:2204.08570 (2022).

[58]

Jessica Dai, Sohini Upadhyay, Ulrich Aivodji, Stephen H Bach, and Himabindu Lakkaraju. 2022. Fairness via explanation quality: Evaluating disparities in the quality of post hoc explanations. In Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society. 203–214.

Digital Library

[59]

Abhisek Dash, Abhijnan Chakraborty, Saptarshi Ghosh, Animesh Mukherjee, and Krishna P Gummadi. 2022. Alexa, in you, I trust! Fairness and interpretability issues in e-commerce search through smart speakers. In Proceedings of the ACM Web Conference 2022. 3695–3705.

Digital Library

[60]

Saloni Dash, Vineeth N Balasubramanian, and Amit Sharma. 2022. Evaluating and mitigating bias in image classifiers: A causal perspective using counterfactuals. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 915–924.

[61]

Anupam Datta, Shayak Sen, and Yair Zick. 2016. Algorithmic transparency via quantitative input influence: Theory and experiments with learning systems. In 2016 IEEE Symposium on Security and Privacy. IEEE, 598–617.

[62]

Maria De-Arteaga, Stefan Feuerriegel, and Maytal Saar-Tsechansky. 2022. Algorithmic fairness in business analytics: Directions for research and practice. Production and Operations Management 31, 10 (2022), 3749–3770.

[63]

Karl de Fine Licht and Jenny de Fine Licht. 2020. Artificial intelligence, transparency, and public decision-making. AI & SOCIETY 35, 4 (2020), 917–926.

Digital Library

[64]

Joachim de Greeff, Maaike HT de Boer, Fieke HJ Hillerström, Freek Bomhof, Wiard Jorritsma, and Mark A Neerincx. 2021. The FATE system: Fair, transparent and explainable decision making. In AAAI Spring Symposium: Combining Machine Learning with Knowledge Engineering. 266–267.

[65]

Botty Dimanov, Umang Bhatt, Mateja Jamnik, and Adrian Weller. 2020. You shouldn’t trust me: Learning models which conceal unfairness from multiple explanation methods. In ECAI 2020. IOS Press, 2473–2480.

[66]

Jonathan Dodge, Q Vera Liao, Yunfeng Zhang, Rachel KE Bellamy, and Casey Dugan. 2019. Explaining models: An empirical study of how explanations impact fairness judgment. In Proceedings of the 24th International Conference on Intelligent User Interfaces. 275–285.

Digital Library

[67]

Finale Doshi-Velez and Been Kim. 2017. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608 (2017).

[68]

Mengnan Du, Fan Yang, Na Zou, and Xia Hu. 2020. Fairness in deep learning: A computational perspective. IEEE Intelligent Systems 36, 4 (2020), 25–34.

Digital Library

[69]

Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. 2012. Fairness through awareness. In Proceedings of the 3rd Innovations in Theoretical Computer Science Conference. 214–226.

Digital Library

[70]

Upol Ehsan and Mark O Riedl. 2021. Explainability pitfalls: Beyond dark patterns in explainable AI. arXiv preprint arXiv:2109.12480 (2021).

[71]

Malin Eiband, Daniel Buschek, Alexander Kremer, and Heinrich Hussmann. 2019. The impact of placebic explanations on trust in intelligent systems. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. 1–6.

Digital Library

[72]

Ming Fan, Wenying Wei, Wuxia Jin, Zijiang Yang, and Ting Liu. 2022. Explanation-guided fairness testing through genetic algorithm. In Proceedings of the 44th International Conference on Software Engineering. 871–882.

Digital Library

[73]

Sina Fazelpour and Zachary C Lipton. 2020. Algorithmic fairness from a non-ideal perspective. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society. 57–63.

Digital Library

[74]

Juliana Jansen Ferreira and Mateus de Souza Monteiro. 2020. Evidence-based explanation to promote fairness in AI systems. arXiv preprint arXiv:2003.01525 (2020).

[75]

Luciano Floridi, Josh Cowls, Monica Beltrametti, Raja Chatila, Patrice Chazerand, Virginia Dignum, Christoph Luetge, Robert Madelin, Ugo Pagallo, Francesca Rossi, 2018. AI4People—An ethical framework for a good AI society: Opportunities, risks, principles, and recommendations. Minds and Machines 28 (2018), 689–707.

Digital Library

[76]

Luciano Floridi, Josh Cowls, Thomas C King, and Mariarosaria Taddeo. 2021. How to design AI for social good: Seven essential factors. Ethics, Governance, and Policies in Artificial Intelligence (2021), 125–151.

[77]

Danilo Franco, Nicolò Navarin, Michele Donini, Davide Anguita, and Luca Oneto. 2022. Deep fair models for complex data: Graphs labeling and explainable face recognition. Neurocomputing 470 (2022), 318–334.

Digital Library

[78]

Ulrik Franke. 2022. First- and second-level bias in automated decision-making. Philosophy & Technology 35, 2 (2022), 21.

[79]

Sorelle A Friedler, Carlos Scheidegger, Suresh Venkatasubramanian, Sonam Choudhary, Evan P Hamilton, and Derek Roth. 2019. A comparative study of fairness-enhancing interventions in machine learning. In Proceedings of the Conference on Fairness, Accountability, and Transparency. 329–338.

Digital Library

[80]

Sainyam Galhotra, Romila Pradhan, and Babak Salimi. 2021. Explaining black-box algorithms using probabilistic contrastive counterfactuals. In Proceedings of the 2021 International Conference on Management of Data. 577–590.

Digital Library

[81]

Erick Galinkin. 2022. Towards a responsible AI development lifecycle: Lessons from information security. arXiv preprint arXiv:2203.02958 (2022).

[82]

Yingqiang Ge, Juntao Tan, Yan Zhu, Yinglong Xia, Jiebo Luo, Shuchang Liu, Zuohui Fu, Shijie Geng, Zelong Li, and Yongfeng Zhang. 2022. Explainable fairness in recommendation. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. 681–691.

Digital Library

[83]

Avijit Ghosh, Aalok Shanbhag, and Christo Wilson. 2022. FairCanary: Rapid continuous explainable fairness. In Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society. 307–316.

Digital Library

[84]

Bishwamittra Ghosh, Debabrota Basu, and Kuldeep S Meel. 2023. “How biased are your features?”: Computing fairness influence functions with global sensitivity analysis. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency. 138–148.

Digital Library

[85]

Navdeep Gill, Patrick Hall, Kim Montgomery, and Nicholas Schmidt. 2020. A responsible machine learning workflow with focus on interpretable models, post-hoc explanation, and discrimination testing. Information 11, 3 (2020), 137.

[86]

Leilani H Gilpin, David Bau, Ben Z Yuan, Ayesha Bajwa, Michael Specter, and Lalana Kagal. 2018. Explaining explanations: An overview of interpretability of machine learning. In 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA). IEEE, 80–89.

[87]

Bryce Goodman and Seth Flaxman. 2017. European Union regulations on algorithmic decision-making and a “right to explanation”. AI Magazine 38, 3 (2017), 50–57.

Digital Library

[88]

Przemyslaw A Grabowicz, Nicholas Perello, and Aarshee Mishra. 2022. Marrying fairness and explainability in supervised learning. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 1905–1916.

Digital Library

[89]

Nina Grgić-Hlača, Muhammad Bilal Zafar, Krishna P Gummadi, and Adrian Weller. 2018. Beyond distributive fairness in algorithmic decision making: Feature selection for procedurally fair learning. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 32.

[90]

Stephan Grimmelikhuijsen. 2023. Explaining why the computer says no: Algorithmic transparency affects the perceived trustworthiness of automated decision-making. Public Administration Review 83, 2 (2023), 241–262.

[91]

Jarek Gryz and Nima Shahbazi. 2020. Futility of a right to explanation. In EDBT/ICDT Workshops. 72–75.

[92]

Soumyajit Gupta, Sooyong Lee, Maria De-Arteaga, and Matthew Lease. 2023. Same same, but different: Conditional multi-task learning for demographic-specific toxicity detection. In Proceedings of the ACM Web Conference 2023. 3689–3700.

Digital Library

[93]

Vivek Gupta, Pegah Nokhiz, Chitradeep Dutta Roy, and Suresh Venkatasubramanian. 2019. Equalizing recourse across groups. arXiv preprint arXiv:1909.03166 (2019).

[94]

Philipp Hacker and Jan-Hendrik Passoth. 2020. Varieties of AI explanations under the law. From the GDPR to the AIA, and beyond. In XXAI: Extending Explainable AI Beyond Deep Models and Classifiers. Springer, 343–373.

[95]

Neal R. Haddaway, Matthew J. Grainger, and Charles T. Gray. 2022. Citationchaser: A tool for transparent and efficient forward and backward citation chasing in systematic searching. Research Synthesis Methods 13, 4 (2022), 533–545.

[96]

Patrick Hall and Navdeep Gill. 2017. Debugging the black-box COMPAS risk assessment instrument to diagnose and remediate bias. https://openreview.net/pdf?id=r1iWHVJ7Z

[97]

Patrick Hall, Navdeep Gill, and Nicholas Schmidt. 2019. Proposed guidelines for the responsible use of explainable machine learning. arXiv preprint arXiv:1906.03533 (2019).

[98]

Ronan Hamon, Henrik Junklewitz, Ignacio Sanchez, Gianclaudio Malgieri, and Paul De Hert. 2022. Bridging the gap between AI and explainability in the GDPR: Towards trustworthiness-by-design in automated decision-making. IEEE Computational Intelligence Magazine 17, 1 (2022), 72–85.

Digital Library

[99]

Michaela Hardt, Xiaoguang Chen, Xiaoyi Cheng, Michele Donini, Jason Gelman, Satish Gollaprolu, John He, Pedro Larroy, Xinyu Liu, Nick McCarthy, 2021. Amazon SageMaker Clarify: Machine learning bias detection and explainability in the cloud. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 2974–2983.

Digital Library

[100]

Bernease Herman. 2017. The promise and peril of human evaluation for model interpretability. arXiv preprint arXiv:1711.07414 (2017).

[101]

James M Hickey, Pietro G Di Stefano, and Vlasios Vasileiou. 2021. Fairness by explicability and adversarial SHAP learning. In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2020, Ghent, Belgium, September 14–18, 2020, Proceedings, Part III. Springer, 174–190.

Digital Library

[102]

Michael Hind, Dennis Wei, Murray Campbell, Noel CF Codella, Amit Dhurandhar, Aleksandra Mojsilović, Karthikeyan Natesan Ramamurthy, and Kush R Varshney. 2019. TED: Teaching AI to explain its decisions. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society. 123–129.

Digital Library

[103]

Fred Hohman, Andrew Head, Rich Caruana, Robert DeLine, and Steven M Drucker. 2019. GAMUT: A design probe to understand how data scientists understand machine learning models. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–13.

Digital Library

[104]

Lily Hu and Issa Kohler-Hausmann. 2020. What’s sex got to do with machine learning?Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (2020), 513–513.

[105]

John E Hunter, Frank L Schmidt, and Ronda Hunter. 1979. Differential validity of employment tests by race: A comprehensive review and analysis. Psychological Bulletin 86, 4 (1979), 721.

[106]

Alexey Ignatiev, Martin C Cooper, Mohamed Siala, Emmanuel Hebrard, and Joao Marques-Silva. 2020. Towards formal fairness in machine learning. In International Conference on Principles and Practice of Constraint Programming. Springer, 846–867.

Digital Library

[107]

Aditya Jain, Manish Ravula, and Joydeep Ghosh. 2020. Biased models have biased explanations. arXiv preprint arXiv:2012.10986 (2020).

[108]

Jean-Marie John-Mathews. 2022. Some critical and ethical perspectives on the empirical turn of AI interpretability. Technological Forecasting and Social Change 174 (2022), 1–29.

[109]

Amir-Hossein Karimi, Gilles Barthe, Bernhard Schölkopf, and Isabel Valera. 2022. A survey of algorithmic recourse: Contrastive explanations and consequential recommendations. Comput. Surveys (2022), 1–26.

[110]

Atoosa Kasirzadeh and Andrew Smart. 2021. The use and misuse of counterfactuals in ethical machine learning. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. 228–236.

Digital Library

[111]

Thomas Kehrenberg, Myles Bartlett, Oliver Thomas, and Novi Quadrianto. 2020. Null-sampling for interpretable and fair representations. In European Conference on Computer Vision. Springer, 565–580.

Digital Library

[112]

Barbara Kitchenham and Stuart M. Charters. 2007. Guidelines for performing systematic literature reviews in software engineering. Technical Report.

[113]

René F Kizilcec. 2016. How much information? Effects of transparency on trust in an algorithmic interface. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. 2390–2395.

Digital Library

[114]

Jon Kleinberg, Jens Ludwig, Sendhil Mullainathan, and Ashesh Rambachan. 2018. Algorithmic fairness. In AEA Papers and Proceedings, Vol. 108. American Economic Association 2014 Broadway, Suite 305, Nashville, TN 37203, 22–27.

[115]

Jon Kleinberg and Sendhil Mullainathan. 2019. Simplicity creates inequity: Implications for fairness, stereotypes, and interpretability. In Proceedings of the 2019 ACM Conference on Economics and Computation. 807–808.

Digital Library

[116]

Jon Kleinberg, Sendhil Mullainathan, and Manish Raghavan. 2017. Inherent trade-offs in the fair determination of risk scores. In 8th Innovations in Theoretical Computer Science Conference (ITCS 2017).

[117]

Joshua A. Kroll, Joanna Huey, Solon Barocas, Edward W. Felten, Joel R. Reidenberg, David G. Robinson, and Harlan Yu. 2017. Accountable algorithms. University of Pennsylvania Law Review 165, 633 (2017), 633–705.

[118]

Udo Kuckartz and Stefan Rädiker. 2019. Analyzing qualitative data with MAXQDA: Text, audio, and video. Springer.

[119]

Catherine Kung and Renzhe Yu. 2020. Interpretable models do not compromise accuracy or fairness in predicting college success. In Proceedings of the Seventh ACM Conference on Learning @ Scale. 413–416.

Digital Library

[120]

Matt J Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. 2017. Counterfactual fairness. Advances in Neural Information Processing Systems 30 (2017).

[121]

Himabindu Lakkaraju and Osbert Bastani. 2020. “How do I fool you?” Manipulating user trust via misleading black box explanations. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society. 79–85.

Digital Library

[122]

Markus Langer, Daniel Oster, Timo Speith, Holger Hermanns, Lena Kästner, Eva Schmidt, Andreas Sesing, and Kevin Baum. 2021. What do we want from explainable artificial intelligence (XAI)? – A stakeholder perspective on XAI and a conceptual model guiding interdisciplinary XAI research. Artificial Intelligence 296 (2021), 103473.

[123]

Connor Lawless, Sanjeeb Dash, Oktay Gunluk, and Dennis Wei. 2023. Interpretable and fair Boolean rule sets via column generation. Journal of Machine Learning Research 24, 229 (2023), 1–50.

[124]

Erwan Le Merrer and Gilles Trédan. 2020. Remote explainability faces the bouncer problem. Nature Machine Intelligence 2, 9 (2020), 529–539.

[125]

Min Kyung Lee, Anuraag Jain, Hea Jin Cha, Shashank Ojha, and Daniel Kusbit. 2019. Procedural justice in algorithmic fairness: Leveraging transparency and outcome control for fair algorithmic mediation. Proceedings of the ACM on Human-Computer Interaction 3, CSCW (2019), 1–26.

Digital Library

[126]

Bruno Lepri, Nuria Oliver, Emmanuel Letouzé, Alex Pentland, and Patrick Vinck. 2018. Fair, transparent, and accountable algorithmic decision-making processes: The premise, the proposed solutions, and the open challenges. Philosophy & Technology 31, 4 (2018), 611–627.

[127]

David Leslie. 2019. Understanding artificial intelligence ethics and safety. arXiv preprint arXiv:1906.05684 (2019).

[128]

Gerald S Leventhal. 1980. What should be done with equity theory? New approaches to the study of fairness in social relationships. In Social exchange: Advances in theory and research. Springer, 27–55.

[129]

Brian Y Lim, Anind K Dey, and Daniel Avrahami. 2009. Why and why not explanations improve the intelligibility of context-aware intelligent systems. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2119–2128.

Digital Library

[130]

Pantelis Linardatos, Vasilis Papastefanopoulos, and Sotiris Kotsiantis. 2020. Explainable AI: A review of machine learning interpretability methods. Entropy 23, 1 (2020), 18.

[131]

Zachary Lipton, Julian McAuley, and Alexandra Chouldechova. 2018. Does mitigating ML’s impact disparity require treatment disparity?Advances in Neural Information Processing Systems 31 (2018).

[132]

Zachary C Lipton. 2018. The mythos of model interpretability: In machine learning, the concept of interpretability is both important and slippery. Queue 16, 3 (2018), 31–57.

Digital Library

[133]

Michele Loi, Andrea Ferrario, and Eleonora Viganò. 2021. Transparency as design publicity: Explaining and justifying inscrutable algorithms. Ethics and Information Technology 23, 3 (2021), 253–263.

Digital Library

[134]

Scott M Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems 30 (2017).

[135]

Henrietta Lyons, Eduardo Velloso, and Tim Miller. 2021. Conceptualising contestability: Perspectives on contesting algorithmic decisions. Proceedings of the ACM on Human-Computer Interaction 5, CSCW1 (2021), 1–25.

Digital Library

[136]

Jocelyn Maclure. 2021. AI, explainability and public reason: The argument from the limitations of the human mind. Minds and Machines 31, 3 (2021), 421–438.

Digital Library

[137]

Marta Marchiori Manerba and Riccardo Guidotti. 2021. FairShades: Fairness auditing via explainability in abusive language detection systems. In 2021 IEEE Third International Conference on Cognitive Machine Intelligence (CogMI). IEEE, 34–43.

[138]

Charles Marx, Flavio Calmon, and Berk Ustun. 2020. Predictive multiplicity in classification. In International Conference on Machine Learning. PMLR, 6765–6774.

[139]

Sandra G Mayson. 2018. Bias in, bias out. The Yale Law Journal 128 (2018), 2218–2300.

[140]

Chuizheng Meng, Loc Trinh, Nan Xu, James Enouen, and Yan Liu. 2022. Interpretability and fairness evaluation of deep learning models on MIMIC-IV dataset. Scientific Reports 12, 1 (2022), 7166.

[141]

Loizos Michael. 2019. Machine coaching. In Proceedings of the IJCAI 2019 Workshop on Explainable Artificial Intelligence (XAI). 80–86.

[142]

Marius Miron, Songül Tolan, Emilia Gómez, and Carlos Castillo. 2021. Evaluating causes of algorithmic bias in juvenile criminal recidivism. Artificial Intelligence and Law 29, 2 (2021), 111–147.

Digital Library

[143]

Alexey Miroshnikov, Konstandinos Kotsiopoulos, Ryan Franks, and Arjun Ravi Kannan. 2022. Wasserstein-based fairness interpretability framework for machine learning models. Machine Learning (2022), 1–51.

[144]

Brent Daniel Mittelstadt, Patrick Allo, Mariarosaria Taddeo, Sandra Wachter, and Luciano Floridi. 2016. The ethics of algorithms: Mapping the debate. Big Data & Society 3, 2 (2016), 1–21.

[145]

Lily Morse, Mike Horia M Teodorescu, Yazeed Awwad, and Gerald C Kane. 2021. Do the ends justify the means? Variation in the distributive and procedural fairness of machine learning algorithms. Journal of Business Ethics (2021), 1–13.

[146]

Erica Mourão, João Felipe Pimentel, Leonardo Murta, Marcos Kalinowski, Emilia Mendes, and Claes Wohlin. 2020. On the performance of hybrid search strategies for systematic literature reviews in software engineering. Information and Software Technology 123 (2020), 106294.

[147]

Deirdre K Mulligan, Joshua A Kroll, Nitin Kohli, and Richmond Y Wong. 2019. This thing called fairness: Disciplinary confusion realizing a value in technology. Proceedings of the ACM on Human-Computer Interaction 3, CSCW (2019), 1–36.

Digital Library

[148]

Ece Çiğdem Mutlu, Niloofar Yousefi, and Ozlem Ozmen Garibay. 2022. Contrastive counterfactual fairness in algorithmic decision-making. In Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society. 499–507.

Digital Library

[149]

Yuri Nakao, Simone Stumpf, Subeida Ahmed, Aisha Naseer, and Lorenzo Strappelli. 2022. Toward involving end-users in interactive human-in-the-loop AI fairness. ACM Transactions on Interactive Intelligent Systems (TiiS) 12, 3 (2022), 1–30.

Digital Library

[150]

Rym Nassih and Abdelaziz Berrado. 2020. State of the art of fairness, interpretability and explainability in machine learning: Case of PRIM. In Proceedings of the 13th International Conference on Intelligent Systems: Theories and Applications. 1–5.

Digital Library

[151]

Eirini Ntoutsi, Pavlos Fafalios, Ujwal Gadiraju, Vasileios Iosifidis, Wolfgang Nejdl, Maria-Esther Vidal, Salvatore Ruggieri, Franco Turini, Symeon Papadopoulos, Emmanouil Krasanakis, 2020. Bias in data-driven artificial intelligence systems—An introductory survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 10, 3 (2020), e1356.

[152]

Julian Nyarko, Sharad Goel, and Roseanna Sommers. 2021. Breaking taboos in fair machine learning: An experimental study. In Proceedings of the 1st ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization. 1–11.

Digital Library

[153]

Alfonso Ortega, Julian Fierrez, Aythami Morales, Zilong Wang, Marina de la Cruz, César Luis Alonso, and Tony Ribeiro. 2021. Symbolic AI for XAI: Evaluating LFIT inductive programming for explaining biases in machine learning. Computers 10, 11 (2021), 154.

[154]

Deepak Padmanabhan, V Sanil, and Joemon Jose. 2020. On fairness and interpretability. In IJCAI 2020 AI for Social Good Workshop.

[155]

Matthew J Page, Joanne E McKenzie, Patrick M Bossuyt, Isabelle Boutron, Tammy C Hoffmann, Cynthia D Mulrow, Larissa Shamseer, Jennifer M Tetzlaff, Elie A Akl, Sue E Brennan, 2021. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ 372 (2021).

[156]

Weishen Pan, Sen Cui, Jiang Bian, Changshui Zhang, and Fei Wang. 2021. Explaining algorithmic fairness through fairness-aware causal path decomposition. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 1287–1297.

Digital Library

[157]

Cecilia Panigutti, Alan Perotti, André Panisson, Paolo Bajardi, and Dino Pedreschi. 2021. FairLens: Auditing black-box clinical decision support systems. Information Processing & Management 58, 5 (2021), 1–17.

Digital Library

[158]

Andrea Papenmeier, Gwenn Englebienne, and Christin Seifert. 2019. How model accuracy and explanation fidelity influence user trust in AI. In IJCAI Workshop on Explainable Artificial Intelligence (XAI) 2019.

[159]

Andrea Papenmeier, Dagmar Kern, Gwenn Englebienne, and Christin Seifert. 2022. It’s complicated: The relationship between user trust, model accuracy and explanations in AI. ACM Transactions on Computer-Human Interaction (TOCHI) 29, 4 (2022), 1–33.

Digital Library

[160]

Álvaro Parafita and Jordi Vitrià. 2021. Deep causal graphs for causal inference, black-box explainability and fairness. In Artificial Intelligence Research and Development: Proceedings of the 23rd International Conference of the Catalan Association for Artificial Intelligence. 415–424.

[161]

Hyanghee Park, Daehwan Ahn, Kartik Hosanagar, and Joonhwan Lee. 2021. Human-AI interaction in human resource management: Understanding why employees resist algorithmic evaluation at workplaces and how to mitigate burdens. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–15.

Digital Library

[162]

Hyanghee Park, Daehwan Ahn, Kartik Hosanagar, and Joonhwan Lee. 2022. Designing fair AI in human resource management: Understanding tensions surrounding algorithmic evaluation and envisioning stakeholder-centered solutions. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–22.

Digital Library

[163]

Andrija Petrović, Mladen Nikolić, Sandro Radovanović, Boris Delibašić, and Miloš Jovanović. 2022. FAIR: Fair adversarial instance re-weighting. Neurocomputing 476 (2022), 14–37.

Digital Library

[164]

Emma Pierson, Camelia Simoiu, Jan Overgoor, Sam Corbett-Davies, Daniel Jenson, Amy Shoemaker, Vignesh Ramachandran, Phoebe Barghouty, Cheryl Phillips, Ravi Shroff, 2020. A large-scale analysis of racial disparities in police stops across the United States. Nature Human Behaviour 4, 7 (2020), 736–745.

[165]

Romila Pradhan, Jiongli Zhu, Boris Glavic, and Babak Salimi. 2022. Interpretable data-based explanations for fairness debugging. In Proceedings of the 2022 International Conference on Management of Data. 247–261.

Digital Library

[166]

Novi Quadrianto, Viktoriia Sharmanska, and Oliver Thomas. 2019. Discovering fair representations in the data domain. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 8227–8236.

[167]

Bilal Qureshi, Faisal Kamiran, Asim Karim, Salvatore Ruggieri, and Dino Pedreschi. 2020. Causal inference for social discrimination reasoning. Journal of Intelligent Information Systems 54, 2 (2020), 425–437.

[168]

Emilee Rader, Kelley Cotter, and Janghee Cho. 2018. Explanations as mechanisms for supporting algorithmic transparency. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–13.

Digital Library

[169]

Edward Raff, Jared Sylvester, and Steven Mills. 2018. Fair forests: Regularized tree induction to minimize model bias. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society. 243–250.

Digital Library

[170]

Gabriëlle Ras, Marcel van Gerven, and Pim Haselager. 2018. Explanation methods in deep learning: Users, values, concerns and challenges. In Explainable and interpretable models in computer vision and machine learning. Springer, 19–36.

[171]

Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. “Why should I trust you?” Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.

Digital Library

[172]

Jake Robertson, Catherine Stinson, and Ting Hu. 2022. A bio-inspired framework for machine bias interpretation. In Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society. 588–598.

Digital Library

[173]

Avi Rosenfeld and Ariella Richardson. 2019. Explainability in human-agent systems. Autonomous Agents and Multi-Agent Systems 33 (2019), 673–705.

Digital Library

[174]

Cynthia Rudin. 2019. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence 1, 5 (2019), 206–215.

[175]

Chris Russell, Matt J Kusner, Joshua Loftus, and Ricardo Silva. 2017. When worlds collide: Integrating different counterfactual assumptions in fairness. Advances in Neural Information Processing Systems 30 (2017).

[176]

Laura Sartori and Andreas Theodorou. 2022. A sociotechnical perspective for the future of AI: Narratives, inequalities, and human control. Ethics and Information Technology 24, 1 (2022), 1–11.

Digital Library

[177]

Benjamin Saunders, Julius Sim, Tom Kingstone, Shula Baker, Jackie Waterfield, Bernadette Bartlam, Heather Burroughs, and Clare Jinks. 2018. Saturation in qualitative research: Exploring its conceptualization and operationalization. Quality & Quantity 52 (2018), 1893–1907.

[178]

Max Schemmer, Niklas Kuehl, Carina Benz, Andrea Bartos, and Gerhard Satzger. 2023. Appropriate reliance on AI advice: Conceptualization and the effect of explanations. In Proceedings of the 28th International Conference on Intelligent User Interfaces. 410–422.

Digital Library

[179]

Nadine Schlicker, Markus Langer, Sonja K. Ötting, Kevin Baum, Cornelius J. König, and Dieter Wallach. 2021. What to expect from opening up ‘black boxes’? Comparing perceptions of justice between human and automated agents. Computers in Human Behavior 122 (2021), 1–16.

[180]

Jakob Schoeffer, Maria De-Arteaga, and Niklas Kuehl. 2022. On explanations, fairness, and appropriate reliance in human-AI decision-making. arXiv preprint arXiv:2209.11812 (2022).

[181]

Jakob Schoeffer, Maria De-Arteaga, and Niklas Kuehl. 2022. On the relationship between explanations, fairness perceptions, and decisions. ACM CHI 2022 Workshop on Human-Centered XAI (HCXAI) (2022).

[182]

Jakob Schoeffer and Niklas Kuehl. 2021. Appropriate fairness perceptions? On the effectiveness of explanations in enabling people to assess the fairness of automated decision systems. In Companion Publication of the 2021 Conference on Computer Supported Cooperative Work and Social Computing. 153–157.

Digital Library

[183]

Jakob Schoeffer, Niklas Kuehl, and Yvette Machowski. 2022. “There is not enough information”: On the effects of explanations on perceptions of informational fairness and trustworthiness in automated decision-making. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 1616–1628.

Digital Library

[184]

Jakob Schoeffer, Yvette Machowski, and Niklas Kuehl. 2021. A study on fairness and trust perceptions in automated decision making. In Joint Proceedings of the ACM IUI 2021 Workshops.

[185]

Jakob Schoeffer, Yvette Machowski, and Niklas Kuehl. 2022. Perceptions of fairness and trustworthiness based on explanations in human vs. automated decision-making. In Proceedings of the 55th Hawaii International Conference on System Sciences (HICSS).

[186]

Andrew D Selbst and Solon Barocas. 2018. The intuitive appeal of explainable machines. Fordham Law Review 87 (2018), 1085–1139.

[187]

William Seymour. 2018. Detecting bias: Does an algorithm have to be transparent in order to be fair?BIAS 2018 (2018).

[188]

Shubham Sharma, Jette Henderson, and Joydeep Ghosh. 2020. CERTIFAI: A common framework to provide explanations and analyse the fairness and robustness of black-box models. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society. 166–172.

Digital Library

[189]

Donghee Shin. 2020. User perceptions of algorithmic decisions in the personalized AI system: Perceptual evaluation of fairness, accountability, transparency, and explainability. Journal of Broadcasting & Electronic Media 64, 4 (2020), 541–565.

[190]

Donghee Shin. 2021. The effects of explainability and causability on perception, trust, and acceptance: Implications for explainable AI. International Journal of Human-Computer Studies 146 (2021).

[191]

Donghee Shin. 2021. Why does explainability matter in news analytic systems? Proposing explainable analytic journalism. Journalism Studies 22, 8 (2021), 1047–1065.

[192]

Donghee Shin, Joon Soo Lim, Norita Ahmad, and Mohammed Ibahrine. 2022. Understanding user sensemaking in fairness and transparency in algorithms: Algorithmic sensemaking in over-the-top platform. AI & SOCIETY (2022), 1–14.

[193]

Avital Shulner-Tal, Tsvi Kuflik, and Doron Kliger. 2022. Fairness, explainability and in-between: Understanding the impact of different explanation methods on non-expert users’ perceptions of fairness toward an algorithmic system. Ethics and Information Technology 24, 1 (2022), 1–13.

Digital Library

[194]

Avital Shulner-Tal, Tsvi Kuflik, and Doron Kliger. 2023. Enhancing fairness perception – Towards human-centred AI and personalized explanations understanding the factors influencing laypeople’s fairness perceptions of algorithmic decisions. International Journal of Human–Computer Interaction 39, 7 (2023), 1455–1482.

[195]

Michael Siering. 2022. Explainability and fairness of RegTech for regulatory enforcement: Automated monitoring of consumer complaints. Decision Support Systems 158 (2022), 1–12.

Digital Library

[196]

Jennifer Skeem, John Monahan, and Christopher Lowenkamp. 2016. Gender, risk assessment, and sanctioning: The cost of treating women like men. Law and Human Behavior 40, 5 (2016), 580.

[197]

Dylan Slack, Sorelle A Friedler, and Emile Givental. 2020. Fairness warnings and Fair-MAML: Learning fairly with minimal data. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 200–209.

Digital Library

[198]

Dylan Slack, Anna Hilgard, Himabindu Lakkaraju, and Sameer Singh. 2021. Counterfactual explanations can be manipulated. Advances in Neural Information Processing Systems 34 (2021), 62–75.

[199]

Dylan Slack, Sophie Hilgard, Emily Jia, Sameer Singh, and Himabindu Lakkaraju. 2020. Fooling LIME and SHAP: Adversarial attacks on post hoc explanation methods. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society. 180–186.

Digital Library

[200]

Kacper Sokol and Peter Flach. 2019. Counterfactual explanations of machine learning predictions: Opportunities and challenges for AI safety. In 2019 AAAI Workshop on Artificial Intelligence Safety, SafeAI 2019.

[201]

Kacper Sokol and Peter Flach. 2020. Explainability fact sheets: A framework for systematic assessment of explainable approaches. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 56–67.

Digital Library

[202]

Timo Speith. 2022. How to evaluate explainability? A case for three criteria. In 2022 IEEE 30th International Requirements Engineering Conference Workshops (REW). IEEE, 92–97.

[203]

Aaron Springer and Steve Whittaker. 2019. Making transparency clear. In Joint Proceedings of the ACM IUI 2019 Workshops.

[204]

Megha Srivastava, Hoda Heidari, and Andreas Krause. 2019. Mathematical notions vs. human perception of fairness: A descriptive approach to fairness for machine learning. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2459–2468.

Digital Library

[205]

Christopher Starke, Janine Baleis, Birte Keller, and Frank Marcinkowski. 2022. Fairness perceptions of algorithmic decision-making: A systematic review of the empirical literature. Big Data & Society 9, 2 (2022), 20539517221115189.

[206]

Georg Starke, Benedikt Schmidt, Eva De Clercq, and Bernice Simone Elger. 2022. Explainability as fig leaf? An exploration of experts’ ethical expectations towards machine learning in psychiatry. AI and Ethics 3, 1 (2022), 303–314.

[207]

Cor Steging, Silja Renooij, and Bart Verheij. 2021. Discovering the rationale of decisions: Towards a method for aligning learning and reasoning. In Proceedings of the Eighteenth International Conference on Artificial Intelligence and Law. 235–239.

Digital Library

[208]

Alexander Stevens, Peter Deruyck, Ziboud Van Veldhoven, and Jan Vanthienen. 2020. Explainability and fairness in machine learning: Improve fair end-to-end lending for Kiva. In 2020 IEEE Symposium Series on Computational Intelligence (SSCI). IEEE, 1241–1248.

[209]

Simone Stumpf, Lorenzo Strappelli, Subeida Ahmed, Yuri Nakao, Aisha Naseer, Giulia Del Gamba, and Daniele Regoli. 2021. Design methods for artificial intelligence fairness and transparency. In Joint Proceedings of the ACM IUI 2021 Workshops.

[210]

Gero Szepannek and Karsten Lübke. 2021. Facing the challenges of developing fair risk scoring models. Frontiers in Artificial Intelligence 4 (2021), 681915.

[211]

Songül Tolan, Marius Miron, Emilia Gómez, and Carlos Castillo. 2019. Why machine learning may lead to unfairness: Evidence from risk assessment for juvenile justice in Catalonia. In Proceedings of the Seventeenth International Conference on Artificial Intelligence and Law. 83–92.

Digital Library

[212]

Florian Tramer, Vaggelis Atlidakis, Roxana Geambasu, Daniel Hsu, Jean-Pierre Hubaux, Mathias Humbert, Ari Juels, and Huang Lin. 2017. FairTest: Discovering unwarranted associations in data-driven applications. In 2017 IEEE European Symposium on Security and Privacy (EuroS&P). IEEE, 401–416.

[213]

Niels Van Berkel, Jorge Goncalves, Danula Hettiachchi, Senuri Wijenayake, Ryan M Kelly, and Vassilis Kostakos. 2019. Crowdsourcing perceptions of fair predictors for machine learning: A recidivism case study. Proceedings of the ACM on Human-Computer Interaction 3, CSCW (2019), 1–21.

Digital Library

[214]

Niels Van Berkel, Jorge Goncalves, Daniel Russo, Simo Hosio, and Mikael B Skov. 2021. Effect of information presentation on fairness perceptions of machine learning predictors. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–13.

Digital Library

[215]

Lingraj S Vannur, Balaji Ganesan, Lokesh Nagalapatti, Hima Patel, and MN Tippeswamy. 2021. Data augmentation for fairness in personal knowledge base population. In Trends and Applications in Knowledge Discovery and Data Mining: PAKDD 2021 Workshops, WSPA, MLMEIN, SDPRA, DARAI, and AI4EPT, Delhi, India, May 11, 2021 Proceedings 25. Springer, 143–152.

[216]

Carla Piazzon Vieira and Luciano Antonio Digiampietri. 2022. Machine learning post-hoc interpretability: A systematic mapping study. In Proceedings of the XVIII Brazilian Symposium on Information Systems. 1–8.

Digital Library

[217]

Kate Vredenburgh. 2022. The right to explanation. Journal of Political Philosophy 30, 2 (2022), 209–229.

[218]

Sandra Wachter, Brent Mittelstadt, and Chris Russell. 2017. Counterfactual explanations without opening the black box: Automated decisions and the GDPR. Harv. JL & Tech. 31 (2017), 841.

[219]

B. Wagner and A. S. d’Avila Garcez. 2021. Neural-symbolic integration for fairness in AI. In Proceedings of the AAAI 2021 Spring Symposium on Combining Machine Learning and Knowledge Engineering.

[220]

Robyn Repko Waller and Russell L Waller. 2022. Assembled bias: Beyond transparent algorithmic bias. Minds and Machines 32, 3 (2022), 533–562.

Digital Library

[221]

Joel Walmsley. 2021. Artificial intelligence and the value of transparency. AI & SOCIETY 36, 2 (2021), 585–595.

Digital Library

[222]

Caroline Wang, Bin Han, Bhrij Patel, and Cynthia Rudin. 2023. In pursuit of interpretable, fair and accurate machine learning for criminal recidivism prediction. Journal of Quantitative Criminology 39, 2 (2023), 519–581.

[223]

Jianhong Wang, Yuan Zhang, Tae-Kyun Kim, and Yunjie Gu. 2020. Shapley Q-value: A local reward approach to solve global reward games. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 7285–7292.

[224]

Ruotong Wang, F Maxwell Harper, and Haiyi Zhu. 2020. Factors influencing perceived fairness in algorithmic decision-making: Algorithm outcomes, development procedures, and individual differences. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–14.

Digital Library

[225]

Ziming Wang, Changwu Huang, and Xin Yao. 2024. Procedural fairness in machine learning. arXiv preprint arXiv:2404.01877 (2024).

[226]

Richard Warner and Robert H Sloan. 2021. Making artificial intelligence transparent: Fairness and the problem of proxy variables. Criminal Justice Ethics 40, 1 (2021), 23–39.

[227]

David S Watson and Luciano Floridi. 2021. The explanation game: A formal framework for interpretable machine learning. In Ethics, governance, and policies in artificial intelligence. Springer, 185–219.

[228]

Claes Wohlin. 2014. Guidelines for snowballing in systematic literature studies and a replication in software engineering. In Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering. ACM, 1–10.

Digital Library

[229]

Claes Wohlin, Marcos Kalinowski, Katia Romero Felizardo, and Emilia Mendes. 2022. Successful combination of database search and snowballing for identification of primary studies in systematic literature studies. Information and Software Technology 147 (2022), 106908.

Digital Library

[230]

Joost F. Wolfswinkel, Elfi Furtmueller, and Celeste P. M. Wilderom. 2013. Using grounded theory as a method for rigorously reviewing literature. European Journal of Information Systems 22, 1 (2013), 1–11.

[231]

Wanqian Yang, Lars Lorch, Moritz Graule, Himabindu Lakkaraju, and Finale Doshi-Velez. 2020. Incorporating interpretable output constraints in Bayesian neural networks. Advances in Neural Information Processing Systems 33 (2020), 12721–12731.

[232]

Carlos Vladimiro González Zelaya. 2019. Towards explaining the effects of data preprocessing on machine learning. In 2019 IEEE 35th International Conference on Data Engineering (ICDE). IEEE, 2086–2090.

[233]

Hantian Zhang, Nima Shahbazi, Xu Chu, and Abolfazl Asudeh. 2021. FairRover: Explorative model building for fair and responsible machine learning. In Proceedings of the Fifth Workshop on Data Management for End-To-End Machine Learning. 1–10.

Digital Library

[234]

Junzhe Zhang and Elias Bareinboim. 2018. Fairness in decision-making – The causal explanation formula. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 32.

[235]

Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 295–305.

Digital Library

[236]

Haibin Zheng, Zhiqing Chen, Tianyu Du, Xuhong Zhang, Yao Cheng, Shouling Ji, Jingyi Wang, Yue Yu, and Jinyin Chen. 2022. NeuronFair: Interpretable white-box fairness testing through biased neuron identification. In Proceedings of the 44th International Conference on Software Engineering. 1519–1531.

Digital Library

[237]

Jianlong Zhou, Fang Chen, and Andreas Holzinger. 2020. Towards explainability for AI fairness. In International Workshop on Extending Explainable AI Beyond Deep Models and Classifiers. Springer, 375–386.

[238]

Julian Zucker and Myraeka d’Leeuwen. 2020. Arbiter: A domain-specific language for ethical machine learning. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society. 421–425.

Digital Library

Cited By

Bohlens C(2025)The Utilization of Artificial Intelligence in Higher Education Institutions in GermanyAI Adoption and Diffusion in Education10.4018/979-8-3693-7949-3.ch012(321-358)Online publication date: 3-Jan-2025
https://doi.org/10.4018/979-8-3693-7949-3.ch012
Nguyen QLe LTran XDo TLin CKostakos VKay JHoang T(2024)FairAD-XAI: Evaluation Framework for Explainable AI Methods in Alzheimer's Disease Detection with Fairness-in-the-loopCompanion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing10.1145/3675094.3678998(870-876)Online publication date: 5-Oct-2024
https://dl.acm.org/doi/10.1145/3675094.3678998
Schoeffer JDe-Arteaga MKühl N(2024)Explanations, Fairness, and Appropriate Reliance in Human-AI Decision-MakingProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642621(1-18)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642621
Show More Cited By

Index Terms

A Critical Survey on Fairness Benefits of Explainable AI

Recommendations

Explanations, Fairness, and Appropriate Reliance in Human-AI Decision-Making
CHI '24: Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems

In this work, we study the effects of feature-based explanations on distributive fairness of AI-assisted decisions, specifically focusing on the task of predicting occupations from short textual bios. We also investigate how any effects are mediated by ...
Methods and standards for research on explainable artificial intelligence: Lessons from intelligent tutoring systems
Abstract
The DARPA Explainable Artificial Intelligence (AI) (XAI) Program focused on generating explanations for AI programs that use machine learning techniques. This article highlights progress during the DARPA Program (2017‐2021) relative to research ...

Lessons learned in the work on intelligent tutoring systems that apply to system design in Explainable AI. image image
Explainable Argumentation for Wellness Consultation
Explainable, Transparent Autonomous Agents and Multi-Agent Systems
Abstract
There has been a recent resurgence in the area of explainable artificial intelligence as researchers and practitioners seek to provide more transparency to their algorithms. Much of this research is focused on explicitly explaining decisions or ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

FAccT '24: Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency

June 2024

2580 pages

ISBN:9798400704505

DOI:10.1145/3630106

Copyright © 2024 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 June 2024

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

National Institutes of Health

Conference

FAccT '24

FAccT '24: The 2024 ACM Conference on Fairness, Accountability, and Transparency

June 3 - 6, 2024

Rio de Janeiro, Brazil

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
600
Total Downloads

Downloads (Last 12 months)600
Downloads (Last 6 weeks)140

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Bohlens C(2025)The Utilization of Artificial Intelligence in Higher Education Institutions in GermanyAI Adoption and Diffusion in Education10.4018/979-8-3693-7949-3.ch012(321-358)Online publication date: 3-Jan-2025
https://doi.org/10.4018/979-8-3693-7949-3.ch012
Nguyen QLe LTran XDo TLin CKostakos VKay JHoang T(2024)FairAD-XAI: Evaluation Framework for Explainable AI Methods in Alzheimer's Disease Detection with Fairness-in-the-loopCompanion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing10.1145/3675094.3678998(870-876)Online publication date: 5-Oct-2024
https://dl.acm.org/doi/10.1145/3675094.3678998
Schoeffer JDe-Arteaga MKühl N(2024)Explanations, Fairness, and Appropriate Reliance in Human-AI Decision-MakingProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642621(1-18)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642621
Speith TCrook BMann SSchomäcker ALanger M(2024)Conceptualizing understanding in explainable artificial intelligence (XAI): an abilities-based approachEthics and Information Technology10.1007/s10676-024-09769-326:2Online publication date: 15-Jun-2024
https://dl.acm.org/doi/10.1007/s10676-024-09769-3
Alvarez JColmenarejo AElobaid AFabbrizzi SFahimi MFerrara AGhodsi SMougan CPapageorgiou IReyero PRusso MScott KState LZhao XRuggieri S(2024)Policy advice and best practices on bias and fairness in AIEthics and Information Technology10.1007/s10676-024-09746-w26:2Online publication date: 29-Apr-2024
https://dl.acm.org/doi/10.1007/s10676-024-09746-w
Speith TXu J(2024)Explainability and Transparency in Practice: A Comparison Between Corporate and National AI Ethics Guidelines in Germany and ChinaExplainable and Transparent AI and Multi-Agent Systems10.1007/978-3-031-70074-3_12(205-223)Online publication date: 6-May-2024
https://dl.acm.org/doi/10.1007/978-3-031-70074-3_12

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten