skip to main content
10.1145/2939672.2945386acmconferencesArticle/Chapter ViewAbstractPublication PageskddConference Proceedingsconference-collections
tutorial

Algorithmic Bias: From Discrimination Discovery to Fairness-aware Data Mining

Published: 13 August 2016 Publication History

Abstract

Algorithms and decision making based on Big Data have become pervasive in all aspects of our daily lives lives (offline and online), as they have become essential tools in personal finance, health care, hiring, housing, education, and policies. It is therefore of societal and ethical importance to ask whether these algorithms can be discriminative on grounds such as gender, ethnicity, or health status. It turns out that the answer is positive: for instance, recent studies in the context of online advertising show that ads for high-income jobs are presented to men much more often than to women [Datta et al., 2015]; and ads for arrest records are significantly more likely to show up on searches for distinctively black names [Sweeney, 2013]. This algorithmic bias exists even when there is no discrimination intention in the developer of the algorithm. Sometimes it may be inherent to the data sources used (software making decisions based on data can reflect, or even amplify, the results of historical discrimination), but even when the sensitive attributes have been suppressed from the input, a well trained machine learning algorithm may still discriminate on the basis of such sensitive attributes because of correlations existing in the data. These considerations call for the development of data mining systems which are discrimination-conscious by-design. This is a novel and challenging research area for the data mining community.
The aim of this tutorial is to survey algorithmic bias, presenting its most common variants, with an emphasis on the algorithmic techniques and key ideas developed to derive efficient solutions. The tutorial covers two main complementary approaches: algorithms for discrimination discovery and discrimination prevention by means of fairness-aware data mining. We conclude by summarizing promising paths for future research.

Supplementary Material

Part 1 of 3 (kdd2016_tutorial_algorithmic_bias_01-acm.mp4)
Part 2 of 3 (kdd2016_tutorial_algorithmic_bias_02-acm.mp4)
Part 3 of 3 (kdd2016_tutorial_algorithmic_bias_03-acm.mp4)

References

[1]
S. Barocas and A. D. Selbst. Big data's disparate impact. SSRN Pre-Print 2477899, 2014.
[2]
F. Bonchi, S. Hajian, B. Mishra, and D. Ramazzotti. Exposing the probabilistic causal structure of discrimination. arXiv:1510.00552, 2015.
[3]
T. Calders and S. Verwer. Three naive bayes approaches for discrimination-free classi cation. DMKD, 21(2):277--292, 2010.
[4]
B. Custers, T. Calders, B. Schermer, and T. Zarsky, editors. Discrimination and Privacy in the Information Society. Springer, 2013.
[5]
A. Datta, M. C. Tschantz, and A. Datta. Automated experiments on ad privacy settings. Proc. Privacy Enhancing Technologies, 2015(1):92--112, 2015.
[6]
C. Dwork, M. Hardt, T. Pitassi, O. Reingold, and R. Zemel. Fairness through awareness. In ITCS, 2012.
[7]
M. Feldman, S. A. Friedler, J. Moeller, C. Scheidegger, and S. Venkatasubramanian. Certifying and removing disparate impact. In KDD, 2015.
[8]
S. Hajian and J. Domingo-Ferrer. A methodology for direct and indirect discrimination prevention in data mining. TKDE, 25(7):1445--1459, 2013.
[9]
S. Hajian, J. Domingo-Ferrer, and O. Farras. Generalization-based privacy preservation and discrimination prevention in data publishing and mining. DMKD, 28(5-6):1158--1188, 2014.
[10]
S. Hajian, J. Domingo-Ferrer, A. Monreale, D. Pedreschi, and F. Giannotti. Discrimination- and privacy-aware patterns. DMKD, 29(6):1733--1782, 2015.
[11]
F. Kamiran and T. Calders. Data preprocessing techniques for classi cation without discrimination. KAIS, 33(1):1--33, 2012.
[12]
B. T. Luong, S. Ruggieri, and F. Turini. k-nn as an implementation of situation testing for discrimination discovery and prevention. In KDD, 2011.
[13]
D. Pedreshi, S. Ruggieri, and F. Turini. Discrimination-aware data mining. In KDD, 2008.
[14]
S. Ruggieri, S. Hajian, F. Kamiran, and X. Zhang. Anti-discrimination analysis using privacy attack strategies. In ECML-PKDD, 2014.
[15]
N. Savage. When computers stand in the schoolhouse door. Commun. ACM, 59(3):19--21, Feb. 2016.
[16]
L. Sweeney. Discrimination in online ad delivery. Queue, 11(3):10, 2013.

Cited By

View all

Index Terms

  1. Algorithmic Bias: From Discrimination Discovery to Fairness-aware Data Mining

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    KDD '16: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
    August 2016
    2176 pages
    ISBN:9781450342322
    DOI:10.1145/2939672
    Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 13 August 2016

    Check for updates

    Author Tags

    1. algorithmic bias
    2. discrimination discovery
    3. discrimination prevention

    Qualifiers

    • Tutorial

    Conference

    KDD '16
    Sponsor:

    Acceptance Rates

    KDD '16 Paper Acceptance Rate 66 of 1,115 submissions, 6%;
    Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

    Upcoming Conference

    KDD '25

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)322
    • Downloads (Last 6 weeks)30
    Reflects downloads up to 30 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2025)The private sector is hoarding AI researchers: what implications for science?∗AI & SOCIETY10.1007/s00146-024-02171-zOnline publication date: 1-Feb-2025
    • (2024)Exploring Generative AI as Personally Effective Decision-Making ToolsEnhancing Automated Decision-Making Through AI10.4018/979-8-3693-6230-3.ch014(451-492)Online publication date: 29-Nov-2024
    • (2024)Unveiling the Transformative PowerFuture of Digital Technology and AI in Social Sectors10.4018/979-8-3693-5533-6.ch001(1-32)Online publication date: 27-Sep-2024
    • (2024)Bias and Fairness in AI TechnologyExploring the Ethical Implications of Generative AI10.4018/979-8-3693-1565-1.ch003(34-48)Online publication date: 19-Apr-2024
    • (2024)Bias Detection and Mitigation in AI-Driven Target Marketing: Exploring Fairness in Automated Consumer ProfilingInternational Journal of Innovative Science and Research Technology (IJISRT)10.38124/ijisrt/IJISRT24MAY2203(2574-2584)Online publication date: 11-Jun-2024
    • (2024)Strategic Overhaul: Reframing Data Integration for Optimal AI UtilizationSSRN Electronic Journal10.2139/ssrn.4892176Online publication date: 2024
    • (2024)OTClean: Data Cleaning for Conditional Independence Violations using Optimal TransportProceedings of the ACM on Management of Data10.1145/36549632:3(1-26)Online publication date: 30-May-2024
    • (2024)Do Recommender Systems Promote Local Music? A Reproducibility Study Using Music Streaming DataProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3688065(148-157)Online publication date: 8-Oct-2024
    • (2024)The Impact of Group Membership Bias on the Quality and Fairness of Exposure in RankingProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657752(1514-1524)Online publication date: 10-Jul-2024
    • (2024)Software Engineering Education Must Adapt and Evolve for an LLM EnvironmentProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 110.1145/3626252.3630927(666-672)Online publication date: 7-Mar-2024
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media