research-article

Open access

Machine Learning and the Meaning of Equal Treatment

Authors:

Sophia Adams Bhatti,

Adrian WellerAuthors Info & Claims

AIES '21: Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society

Pages 956 - 966

https://doi.org/10.1145/3461702.3462556

Published: 30 July 2021 Publication History

Abstract

Approaches to non-discrimination are generally informed by two principles: striving for equality of treatment, and advancing various notions of equality of outcome. We consider when and why there are trade-offs in machine learning between respecting formalistic interpretations of equal treatment and advancing equality of outcome. Exploring a hypothetical discrimination suit against Facebook, we argue that interpretations of equal treatment which require blindness to difference may constrain how machine learning can be deployed to advance equality of outcome. When machine learning models predict outcomes that are unevenly distributed across racial groups, using those models to advance racial justice will often require deliberately taking race into account. We then explore the normative stakes of this tension. We describe three pragmatic policy options underpinned by distinct interpretations and applications of equal treatment. A status quo approach insists on blindness to difference, permitting the design of machine learning models that compound existing patterns of disadvantage. An industry-led approach would specify a narrow set of domains in which institutions were permitted to use protected characteristics to actively reduce inequalities of outcome. A government-led approach would impose positive duties that require institutions to consider how best to advance equality of outcomes and permit the use of protected characteristics to achieve that goal. We argue that while machine learning offers significant possibilities for advancing racial justice and outcome-based equality, harnessing those possibilities will require a shift in the normative commitments that underpin the interpretation and application of equal treatment in non-discrimination law and the governance of machine learning.

References

[1]

1999. Matadeen v. Pointu.

[2]

2009. Ricci v. DeStefano.

[3]

2017. R (on the application of Coll) v. Secretary of State for Justice.

[4]

Morris B Abram. 1986. Affirmative Action: Fair Shakers and Social Engineers. Harvard law review, Vol. 99, 6 (1986), 1312--1326.

[5]

Ifeoma Ajunwa. 2019 a. Automated Employment Discrimination 34 HARV. JL & TECH. _ (forthcoming 2021). JL & TECH. __ (forthcoming 2021).(March 15, 2019), Vol. 34 (2019).

[6]

Ifeoma Ajunwa. 2019 b. The paradox of automation as anti-bias intervention. Cardozo L. Rev., Vol. 41 (2019), 1671.

[7]

Sven Patric Aldeberg, Fiona Jane Liebe Saunders Watson, Malcolm Clive North, et al. 1999. National Coalition for Gay and Lesbian Equality V. Minister of Home Affairs*[December 2, 1999]+ Cite as 39 ILM 798 (2000). (1999).

[8]

Muhammad Ali, Piotr Sapiezynski, Miranda Bogen, Aleksandra Korolova, Alan Mislove, and Aaron Rieke. 2019. Discrimination through Optimization: How Facebook's Ad Delivery Can Lead to Biased Outcomes. Proceedings of the ACM on Human-Computer Interaction, Vol. 3, CSCW (2019), 1--30.

Digital Library

[9]

Danielle Allen. 2014. Our declaration: A reading of the Declaration of Independence in defense of equality .WW Norton & Company.

[10]

Danielle S Allen. 2003. The world of Prometheus: The politics of punishing in democratic Athens. Princeton University Press.

[11]

Danielle S Allen. 2006. Talking to strangers. anxieties of citizenship since brown v. board of education. Number 23. Editions Hermann.

[12]

Elizabeth Anderson. 2013. The imperative of integration .Princeton University Press.

[13]

Elizabeth S Anderson. 1999. What is the Point of Equality? Ethics, Vol. 109, 2 (1999), 287--337.

[14]

Equality Impact Assessment. 2012. Equality Impact Assessment. Dementia, Vol. 1 (2012), 2--135.

[15]

Financial Conduct Authority. 2015. Regulatory sandbox. online last accessed 1 October 2016) (2015).

[16]

Jack M Balkin and Reva B Siegel. 2003. The American civil rights tradition: Anticlassification or antisubordination. Issues in Legal Scholarship, Vol. 2, 1 (2003).

[17]

Solon Barocas and Andrew D Selbst. 2016. Big data's disparate impact. Calif. L. Rev., Vol. 104 (2016), 671.

[18]

Herman Belz. 1990. Equality transformed: A quarter-century of affirmative action. Vol. 15. Transaction Publishers.

[19]

Jason R Bent. 2019. Is Algorithmic Affirmative Action Legal? Georgetown Law Journal, Forthcoming (2019).

[20]

Reuben Binns. 2017. Data protection impact assessments: a meta-regulatory approach. International Data Privacy Law, Vol. 7, 1 (2017), 22--35.

[21]

Reuben Binns. 2020. On the apparent conflict between individual and group fairness. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 514--524.

Digital Library

[22]

Henry Blackmun. 1978. Regents of the University of California v. Bakke.

[23]

Alexandra Chouldechova. 2017. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big data, Vol. 5, 2 (2017), 153--163.

[24]

Hugh Collins and Tarunabh Khaitan. 2018. Foundations of indirect discrimination law .Bloomsbury Publishing.

[25]

Sam Corbett-Davies and Sharad Goel. 2018. The measure and mismeasure of fairness: A critical review of fair machine learning. arXiv preprint arXiv:1808.00023 (2018).

[26]

Roger Crisp. 2014. Aristotle: Nicomachean Ethics .Cambridge University Press.

[27]

William Dieterich, Christina Mendoza, and Tim Brennan. 2016. COMPAS risk scales: Demonstrating accuracy equity and predictive parity. Northpointe Inc (2016).

[28]

Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. 2012. Fairness through awareness. In Proceedings of the 3rd innovations in theoretical computer science conference. 214--226.

Digital Library

[29]

Cynthia Dwork, Nicole Immorlica, Adam Tauman Kalai, and Max Leiserson. 2018. Decoupled Classifiers for Group-Fair and Efficient Machine Learning (Proceedings of Machine Learning Research, Vol. 81), Sorelle A. Friedler and Christo Wilson (Eds.). PMLR, 119--133.

[30]

Benjamin Eidelson. 2015. Discrimination and disrespect .Oxford University Press.

[31]

Virginia Eubanks. 2018. Automating inequality: How high-tech tools profile, police, and punish the poor .St. Martin's Press.

Digital Library

[32]

Michael Feldman, Sorelle A Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubramanian. 2015. Certifying and removing disparate impact. In proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining. 259--268.

Digital Library

[33]

Owen M Fiss. 1976. Groups and the equal protection clause. Philosophy & Public Affairs (1976), 107--177.

[34]

Sandra Fredman. 2001. Equality: a new generation? Industrial Law Journal, Vol. 30, 2 (2001), 145--168.

[35]

Sandra Fredman. 2014. Addressing disparate impact: Indirect discrimination and the public sector equality duty. Industrial Law Journal, Vol. 43, 3 (2014), 349--363.

[36]

Sandra Fredman. 2016. Substantive equality revisited. International Journal of Constitutional Law, Vol. 14, 3 (2016), 712--738.

[37]

Sorelle A Friedler, Carlos Scheidegger, and Suresh Venkatasubramanian. 2016. On the (im) possibility of fairness. arXiv preprint arXiv:1609.07236 (2016).

[38]

Raymond Geuss, Quentin Skinner, et al. 1996. Aristotle: The Politics and the Constitution of Athens. Cambridge University Press.

[39]

Claudia Goldin. 2014. A grand gender convergence: Its last chapter. American Economic Review, Vol. 104, 4 (2014), 1091--1119.

[40]

Claudia Goldin, Sari Pekkala Kerr, Claudia Olivetti, and Erling Barth. 2017. The expanding gender earnings gap: Evidence from the LEHD-2000 Census. American Economic Review, Vol. 107, 5 (2017), 110--14.

[41]

Evadné Grant. 2007. Dignity and equality. (2007).

[42]

Lady Hale. 2017. Essop v. Home Office., 27 pages.

[43]

Deborah Hellman. 2008. When is discrimination wrong? Harvard University Press.

[44]

Deborah Hellman. 2020. Measuring algorithmic fairness. Va. L. Rev., Vol. 106 (2020), 811.

[45]

Deborah Hellman and Sophia Moreau. 2013. Philosophical foundations of discrimination law. OUP Oxford.

[46]

Anna Lauren Hoffmann. 2019. Where fairness fails: data, algorithms, and the limits of antidiscrimination discourse. Information, Communication & Society, Vol. 22, 7 (2019), 900--915.

[47]

Elisa Holmes. 2005. Anti-Discrimination Rights Without Equality. The Modern Law Review, Vol. 68, 2 (2005), 175--194.

[48]

HUD. 2019 a. Charge Sheet. https://www.hud.gov/sites/dfiles/Main/documents/HUD_v_Facebook.pdf.

[49]

HUD. 2019 b. HUD's Implementation of the Fair Housing Act's Disparate Impact Standard. https://www.federalregister.gov/documents/2019/08/19/2019--17542/huds-implementation-of-the-fair-housing-acts-disparate-impact-standard#h-9.

[50]

Faisal Kamiran, Toon Calders, and Mykola Pechenizkiy. 2010. Discrimination aware decision tree learning. In 2010 IEEE International Conference on Data Mining. IEEE, 869--874.

Digital Library

[51]

Ira Katznelson. 2005. When affirmative action was white: An untold history of racial inequality in twentieth-century America .WW Norton & Company.

[52]

Ibram X Kendi. 2019. How to be an antiracist .One world.

[53]

Randall Kennedy. 2013. For discrimination: Race, affirmative action, and the law .Pantheon.

[54]

Tarunabh Khaitan. 2015. A theory of discrimination law .OUP Oxford.

[55]

Pauline T Kim and Sharion Scott. 2018. Discrimination in online employment recruiting. Louis ULJ, Vol. 63 (2018), 93.

[56]

Jon Kleinberg, Jens Ludwig, Sendhil Mullainathan, and Ashesh Rambachan. 2018a. Algorithmic fairness. In AEA papers and proceedings, Vol. 108. 22--27.

[57]

Jon Kleinberg, Jens Ludwig, Sendhil Mullainathan, and Cass R Sunstein. 2018b. Discrimination in the Age of Algorithms. Journal of Legal Analysis, Vol. 10 (2018).

[58]

Jon Kleinberg and Sendhil Mullainathan. 2019. Simplicity creates inequity: implications for fairness, stereotypes, and interpretability. In Proceedings of the 2019 ACM Conference on Economics and Computation. 807--808.

Digital Library

[59]

Jon Kleinberg, Sendhil Mullainathan, and Manish Raghavan. 2016. Inherent trade-offs in the fair determination of risk scores. arXiv preprint arXiv:1609.05807 (2016).

[60]

Kathryn Kolbert and David H Gans. 1993. Responding to Planned Parenthood v. Casey: Establishing Neutrality Principles in State Constitutional Law. Temple L. Rev., Vol. 66 (1993), 1151.

[61]

Joshua A Kroll, Solon Barocas, Edward W Felten, Joel R Reidenberg, David G Robinson, and Harlan Yu. 2016. Accountable algorithms. U. Pa. L. Rev., Vol. 165 (2016), 633.

[62]

Jill Lepore. 2018. These truths: A history of the United States .WW Norton & Company.

[63]

Meira Levinson. 2012. No citizen left behind. Vol. 13. Harvard University Press.

[64]

Zachary Lipton, Julian McAuley, and Alexandra Chouldechova. 2018. Does mitigating ML's impact disparity require treatment disparity?. In Advances in Neural Information Processing Systems. 8125--8135.

[65]

Lydia T Liu, Sarah Dean, Esther Rolf, Max Simchowitz, and Moritz Hardt. 2018. Delayed impact of fair machine learning. In International Conference on Machine Learning.

[66]

Catharine A MacKinnon. 2016. Substantive equality revisited: A reply to Sandra Fredman. International journal of constitutional law, Vol. 14, 3 (2016), 739--746.

[67]

Catharine A MacKinnon. 2017. Substantive equality revisited: A rejoinder to Sandra Fredman. International Journal of Constitutional Law, Vol. 15, 4 (2017), 1174--1177.

[68]

Dee Masters. 2017. Identifying Direct Discrimination in 'Proxy Cases' after R (on the Application of Coll) v Secretary of State for Justice. Cloisters. https://www.cloisters.com/identifying-direct-discrimination-in-proxy-cases-after-r-on-the-application-of-coll-v-secretary-of-state-for-justice/ (2017).

[69]

Aileen McColgan. 2014. Discrimination, equality and the law .Bloomsbury Publishing.

[70]

Christopher McCrudden. 1982. Institutional discrimination. Oxford J. Legal Stud., Vol. 2 (1982), 303.

[71]

Christopher McCrudden and Sacha Prechal. 2009. The Concepts of Equality and Non-discrimination in Europe: A practical approach. European Commission, Directorate-General for Employment, Social Affairs and Equal Opportunities, Unit G, Vol. 2 (2009).

[72]

Arthur S Miller and Ronald F Howell. 1960. The Myth of Neutrality in Constitutional Adjudication. The University of Chicago Law Review, Vol. 27, 4 (1960), 661--695.

[73]

Martha Minow. 1990. Making all the difference: Inclusion, exclusion, and American law. Cornell University Press.

[74]

Thomas Nagel. 1997. Introduction of equality and preferential treatment. A Philosophy (1997).

[75]

Cathy O'Neil. 2019. Comment Regarding Docket No. FR-6111-P-02. https://clinic.cyber.harvard.edu/files/2019/10/HUD-Rule-Comment-ONEIL-10--18--2019-FINAL.pdf.

[76]

Dino Pedreshi, Salvatore Ruggieri, and Franco Turini. 2008. Discrimination-aware data mining. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining. 560--568.

Digital Library

[77]

Juan C Perdomo, Tijana Zrnic, Celestine Mendler-Dünner, and Moritz Hardt. 2020. Performative prediction. arXiv preprint arXiv:2002.06673 (2020).

[78]

Manish Raghavan, Solon Barocas, Jon Kleinberg, and Karen Levy. 2020. Mitigating bias in algorithmic hiring: Evaluating claims and practices. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 469--481.

Digital Library

[79]

John Roberts. 2007. Parents Involved in Community Schools v. Seattle School Dist.

[80]

Piotr Sapiezynski, Avijit Gosh, Levi Kaplan, Alan Mislove, and Aaron Rieke. 2019. Algorithms that "Don't See Color": Comparing Biases in Lookalike and Special Ad Audiences. arXiv preprint arXiv:1912.07579 (2019).

[81]

Marshall Cohen Thomas Nagel Thomas Scanlon, Ronald Dworkin, et al. 1977. Equality and preferential treatment .Princeton University Press.

[82]

Frederick Schauer. 2018. On Treating Unlike Cases Alike.

[83]

Andrew Selbst. 2019. A New HUD Rule Would Effectively Encourage Discrimination by Algorithm. https://slate.com/technology/2019/08/hud-disparate-impact-discrimination-algorithm.html (2019).

[84]

Andrew D Selbst, Danah Boyd, Sorelle A Friedler, Suresh Venkatasubramanian, and Janet Vertesi. 2019. Fairness and abstraction in sociotechnical systems. In Proceedings of the Conference on Fairness, Accountability, and Transparency. 59--68.

Digital Library

[85]

Tommie Shelby. 2016. Dark ghettos: Injustice, dissent, and reform. Harvard University Press.

[86]

Reva B Siegel. 2017. Blind Justice: Why the Court Refused to Accept Statistical Evidence of Discriminatory Purpose in McCleskey v. Kemp-And Some Pathways for Charge. Nw. UL Rev., Vol. 112 (2017), 1269.

[87]

Reva B Siegel. 2018. The Constitutionalism of Disparate Impact-Court-Centered and Popular Pathways: a Comment on Owen Fiss's Brennan Lecture. Calif. L. Rev., Vol. 106 (2018), 2001.

[88]

John David Skrentny. 2018. The ironies of affirmative action: Politics, culture, and justice in America. University of Chicago Press.

[89]

Sonya Sotomayor. 2014. Schuette v. Coalition to Defend Affirmative Action.

[90]

Cass R Sunstein. 1992. Neutrality in constitutional law (with special reference to pornography, abortion, and surrogacy). Columbia Law Review, Vol. 92, 1 (1992), 1--52.

[91]

Beverly Daniel Tatum. 2017. Why are all the Black kids sitting together in the cafeteria?: And other conversations about race. Basic Books.

[92]

Christa Tobler. 2008. Limits and potential of the concept of indirect discrimination. Office for Official Publications of the European Communities.

[93]

Jon Truby. 2018. Fintech and the city: Sandbox 2.0 policy and regulatory reform proposals. International Review of Law, Computers & Technology (2018), 1--33.

[94]

Upturn. 2018. Brief as Amicus Curiae Supporting Petitioners. https://www.upturn.org/static/files/2018--11--16_Upturn_Facebook_Amicus.pdf.

[95]

Berk Ustun, Yang Liu, and David Parkes. 2019. Fairness without harm: Decoupled classifiers with preference guarantees. In International Conference on Machine Learning. 6373--6382.

[96]

Michael Veale, Reuben Binns, and Jef Ausloos. 2018. When data protection by design and data subject rights clash. International Data Privacy Law, Vol. 8, 2 (2018), 105--123.

[97]

Wolfgang Von Leyden. 1985. Aristotle on equality and justice: His political argument. Springer.

[98]

Michael Walzer. 2008. Spheres of justice: A defense of pluralism and equality. Basic books.

[99]

Alice Xiang. 2021. Reconciling legal and technical approaches to algorithmic bias. Tennessee Law Review, Vol. 88, 3 (2021).

[100]

Iris Marion Young. 2011. Justice and the Politics of Difference. Princeton University Press.

[101]

Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rodriguez, and Krishna P Gummadi. 2017a. Fairness beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment. In Proceedings of the 26th international conference on world wide web. 1171--1180.

Digital Library

[102]

Muhammad Bilal Zafar, Isabel Valera, Manuel Rodriguez, Krishna Gummadi, and Adrian Weller. 2017b. From parity to preference-based notions of fairness in classification. In Advances in Neural Information Processing Systems. 229--239.

[103]

Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rogriguez, and Krishna P Gummadi. 2017c. Fairness constraints: Mechanisms for fair classification. In Artificial Intelligence and Statistics. PMLR, 962--970.

Cited By

Stanojevic A(2024)Algorithmic Governance and Social Vulnerability: A Value Analysis of Equality and TrustSSRN Electronic Journal10.2139/ssrn.4685312Online publication date: 2024
https://doi.org/10.2139/ssrn.4685312
Wen HChen TChai LSadiq SGao JYin H(2024)Variational Counterfactual Prediction Under Runtime Domain CorruptionIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2023.332189336:5(2271-2284)Online publication date: May-2024
https://doi.org/10.1109/TKDE.2023.3321893
Hertwig RHerzog SKozyreva A(2023)Blinding to Circumvent Human Biases: Deliberate Ignorance in Humans, Institutions, and MachinesPerspectives on Psychological Science10.1177/17456916231188052Online publication date: 5-Sep-2023
https://doi.org/10.1177/17456916231188052
Show More Cited By

Index Terms

Machine Learning and the Meaning of Equal Treatment
1. Applied computing
  1. Law, social and behavioral sciences
    1. Law

Recommendations

Measuring justice in machine learning
FAT* '20: Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency

How can we build more just machine learning systems? To answer this question, we need to know both what justice is and how to tell whether one system is more or less just than another. That is, we need both a definition and a measure of justice. ...
Racial categories in machine learning
FAT* '19: Proceedings of the Conference on Fairness, Accountability, and Transparency

Controversies around race and machine learning have sparked debate among computer scientists over how to design machine learning systems that guarantee fairness. These debates rarely engage with how racial identity is embedded in our social experience, ...
The Use and Misuse of Counterfactuals in Ethical Machine Learning
FAccT '21: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency

The use of counterfactuals for considerations of algorithmic fairness and explainability is gaining prominence within the machine learning community and industry. This paper argues for more caution with the use of counterfactuals when the facts to be ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

AIES '21: Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society

July 2021

1077 pages

ISBN:9781450384735

DOI:10.1145/3461702

Program Chairs:
Marion Fourcade
University of California Berkeley, USA
,
Benjamin Kuipers
University of Michigan, USA
,
Seth Lazar
Australian National University, Australia
,
Deirdre Mulligan
University of California Berkeley, USA

Copyright © 2021 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 July 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

EPSRC

Conference

AIES '21

Sponsor:

SIGAI

AIES '21: AAAI/ACM Conference on AI, Ethics, and Society

May 19 - 21, 2021

Virtual Event, USA

Acceptance Rates

Overall Acceptance Rate 61 of 162 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
749
Total Downloads

Downloads (Last 12 months)265
Downloads (Last 6 weeks)29

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Stanojevic A(2024)Algorithmic Governance and Social Vulnerability: A Value Analysis of Equality and TrustSSRN Electronic Journal10.2139/ssrn.4685312Online publication date: 2024
https://doi.org/10.2139/ssrn.4685312
Wen HChen TChai LSadiq SGao JYin H(2024)Variational Counterfactual Prediction Under Runtime Domain CorruptionIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2023.332189336:5(2271-2284)Online publication date: May-2024
https://doi.org/10.1109/TKDE.2023.3321893
Hertwig RHerzog SKozyreva A(2023)Blinding to Circumvent Human Biases: Deliberate Ignorance in Humans, Institutions, and MachinesPerspectives on Psychological Science10.1177/17456916231188052Online publication date: 5-Sep-2023
https://doi.org/10.1177/17456916231188052
Zilka MFogliato RHron JButcher BAshurst CWeller A(2023)The Progression of Disparities within the Criminal Justice System: Differential Enforcement and Risk Assessment InstrumentsProceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency10.1145/3593013.3594099(1553-1569)Online publication date: 12-Jun-2023
https://dl.acm.org/doi/10.1145/3593013.3594099
Patel OKundu K(2022)Oral Cancer Detection and Diagnosis: A New Frontier in Artificial Intelligence2022 4th International Conference on Artificial Intelligence and Speech Technology (AIST)10.1109/AIST55798.2022.10065177(1-4)Online publication date: 9-Dec-2022
https://doi.org/10.1109/AIST55798.2022.10065177

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten