skip to main content
research-article
Free access

Biases in AI systems

Published: 26 July 2021 Publication History

Abstract

A survey for practitioners.

References

[1]
Ali, M., Sapiezynsk, P., Bogen, M., Korolova, A., Mislove, A., Rieke, A. Discrimination through optimization: how Facebook's ad delivery can lead to biased outcomes. In Proceedings of the ACM on Human-Computer Interaction 3 (2019)
[2]
Amatriain, X. What does the concept of presentation-feedback bias refer to in the context of machine learning? Quora, 2015; https://www.quora.com/What-does-the-concept-of-presentation-feedback-bias-refer-to-in-the-context-of-machine-learning.
[3]
Austin, P.C., Platt, R.W. Survivor treatment bias, treatment selection bias, and propensity scores in observational research., 2 (2010), 136--138; https://www.jclinepi.com/article/S0895-4356(09)00247-9/fulltext.
[4]
Barocas, S., Hardt, M., Narayanan, A. Fairness and machine learning: limitations and opportunities, 2019; https://fairmlbook.org.
[5]
Bellamy, R.K.E. et al. AI Fairness 360: An extensible toolkit for detecting, understanding, and mitigating unwanted algorithmic bias. 2018, arXiv; https://arxiv.org/abs/1810.01943.
[6]
Buolamwini, J., Gebru, T. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Proceedings of Machine Learning Research 81 (2018), 1--15; http://proceedings.mlr.press/v81/buolamwini18a/buolamwini18a.pdf.
[7]
Crawford, K., Paglen, T. Excavating AI: The politics of images in machine learning training sets. The AI Now Institute, New York University, 2019; https://www.excavating.ai.
[8]
Dressel, J., Farid, H. The accuracy, fairness, and limits of predicting recidivism. Science Advances 4, 1 (2018); https://advances.sciencemag.org/content/4/1/eaao5580.
[9]
Elwert, F., Winship, C. Endogenous selection bias: The problem of conditioning on a collider variable. Annual Review of Sociology 40 (2014), 31--53
[10]
Friedman, B., Nissenbaum, H. Bias in computer systems. In ACM Trans. Information Systems 14, 3 (1996)
[11]
Gajane, P., Pechenizkiy, M. On formalizing fairness in prediction with machine learning. In Proceedings of the Intern. Conf. Machine Learning, Fairness Accountability and Transparency Workshop, 2018; https://www.fatml.org/media/documents/formalizing_fairness_in_prediction_with_ml.pdf.
[12]
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé III, H., Crawford, K. Datasheets for datasets. In Proceedings of the 5th Workshop on Fairness, Accountability, and Transparency in Machine Learning, 2018; https://www.microsoft.com/en-us/research/uploads/prod/2019/01/1803.09010.pdf.
[13]
Hao, K. This is how AI bias really happens---and why it's so hard to fix. MIT Technology Review; https://www.technologyreview.com/2019/02/04/137602/this-is-how-ai-bias-really-happensand-why-its-so-hard-to-fix/.
[14]
Holstein, K., Vaughan, J.W., Daumé III, H., Dudik, M., Wallach, H. Improving fairness in machine learning systems: What do industry practitioners need? In Proceedings of the 2019 SIGCHI Con. Human Factors in Computing Systems, 1--16
[15]
Kahneman, D. Evaluation by moments: past and future. Choices, Values and Frames. D. Kahneman and A. Tversky, Eds. Cambridge University Press, New York, 2000.
[16]
Kilbertus, N., Ball, P.J., Kusner, M.J., Weller, A., Silva, R. The sensitivity of counterfactual fairness to unmeasured confounding. In Proceedings of the 2019 Conf. Uncertainty in Artificial Intelligence; http://auai.org/uai2019/proceedings/papers/213.pdf.
[17]
Lowry, S., Macpherson, G. 1988. A blot on the profession. British Medical J. Clinical Research Ed. 296, 6623 (1988), 657; https://www.bmj.com/content/296/6623/657.
[18]
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., Galstyan, A. A survey on bias and fairness in machine learning. 2019, arXiv; https://arxiv.org/abs/1908.09635.
[19]
Mitchell, M. et al. Model cards for model reporting. In Proceedings of the 2019 AAAI/ACM Conf. AI, Ethics, and Society; arXiv; https://arxiv.org/abs/1810.03993.
[20]
Pearl, J., Mackenzie, D. The Book of Why: The New Science of Cause and Effect. Basic Books, 2018.
[21]
Plous, S. The Psychology of Judgment and Decision Making. McGraw-Hill, 1993.
[22]
Raji, I., Buolamwini, J. Actionable auditing: investigating the impact of publicly naming biased performance results of commercial AI products. In Proceedings of the 2019 AAAI/ACM Conf. AI, Ethics, and Society, 429--435
[23]
Small, Z. 600,000 images removed from AI database after art project exposes racist bias. Hyperallergic, 2019; https://hyperallergic.com/518822/600000-imagesremoved-from-ai-database-after-art-project-exposesracist-bias/.
[24]
Srinivasan, R., Chander, A. Crowdsourcing in the absence of ground truth---a case study. In Proceedings of the 2019 Intern. Conf. Machine Learning Workshop on Human in the Loop Learning; https://arxiv.org/abs/1906.07254.
[25]
Torralba, A., Efros, A.A. Unbiased look at dataset bias. In Proceedings of the 2011 IEEE Con. Computer Vision and Pattern Recognition, 1521--1528; https://ieeexplore.ieee.org/document/5995347.

Cited By

View all
  • (2025)Ethical Implications of AIUsing AI Tools in Text Analysis, Simplification, Classification, and Synthesis10.4018/979-8-3693-9511-0.ch014(411-438)Online publication date: 14-Feb-2025
  • (2025)The Philosophy of Intelligenticism and Its Implications on Potential Future Directions for IS Research and PracticeArtificial Intelligence for Design and Process Science10.1007/978-3-031-67886-8_10(159-173)Online publication date: 14-Jan-2025
  • (2024)Fair classification with partial feedbackProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3693017(23547-23576)Online publication date: 21-Jul-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image Communications of the ACM
Communications of the ACM  Volume 64, Issue 8
August 2021
116 pages
ISSN:0001-0782
EISSN:1557-7317
DOI:10.1145/3477555
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 July 2021
Published in CACM Volume 64, Issue 8

Permissions

Request permissions for this article.

Check for updates

Qualifiers

  • Research-article
  • Popular
  • Refereed

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)3,620
  • Downloads (Last 6 weeks)408
Reflects downloads up to 14 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Ethical Implications of AIUsing AI Tools in Text Analysis, Simplification, Classification, and Synthesis10.4018/979-8-3693-9511-0.ch014(411-438)Online publication date: 14-Feb-2025
  • (2025)The Philosophy of Intelligenticism and Its Implications on Potential Future Directions for IS Research and PracticeArtificial Intelligence for Design and Process Science10.1007/978-3-031-67886-8_10(159-173)Online publication date: 14-Jan-2025
  • (2024)Fair classification with partial feedbackProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3693017(23547-23576)Online publication date: 21-Jul-2024
  • (2024)Transparency and AccountabilityChallenges in Large Language Model Development and AI Ethics10.4018/979-8-3693-3860-5.ch006(178-211)Online publication date: 30-Aug-2024
  • (2024)The Future of Ethical AIChallenges in Large Language Model Development and AI Ethics10.4018/979-8-3693-3860-5.ch005(145-177)Online publication date: 30-Aug-2024
  • (2024)Enhancing Assessment Systems in Higher EducationUtilizing AI for Assessment, Grading, and Feedback in Higher Education10.4018/979-8-3693-2145-4.ch002(28-56)Online publication date: 19-Apr-2024
  • (2024)Artificial Intelligence and Health in AfricaExamining the Rapid Advance of Digital Technology in Africa10.4018/978-1-6684-9962-7.ch006(105-125)Online publication date: 23-Feb-2024
  • (2024)The Sociodemographic Biases in Machine Learning Algorithms: A Biomedical Informatics PerspectiveLife10.3390/life1406065214:6(652)Online publication date: 21-May-2024
  • (2024)Strategies to improve fairness in artificial intelligence:A systematic literature reviewEducation for Information10.3233/EFI-240045(1-24)Online publication date: 25-Jul-2024
  • (2024)Assessing and Mitigating Bias in Artificial Intelligence: A ReviewRecent Advances in Computer Science and Communications10.2174/266625581666623052311442517:1Online publication date: Jan-2024
  • Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Digital Edition

View this article in digital edition.

Digital Edition

Magazine Site

View this article on the magazine site (external)

Magazine Site

Login options

Full Access

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media