research-article

The scales of (algorithmic) justice: tradeoffs and remedies

Authors:

Marissa GerchickAuthors Info & Claims

AI Matters, Volume 5, Issue 2

Pages 30 - 40

https://doi.org/10.1145/3340470.3340478

Published: 05 August 2019 Publication History

Abstract

Every day, governmental and federally funded agencies --- including criminal courts, welfare agencies, and educational institutions --- make decisions about resource allocation using automated decision-making tools (Lecher, 2018; Fishel, Flack, & DeMatteo, 2018). Important factors surrounding the use of these tools are embedded both in their design and in the policies and practices of the various agencies that implement them. As the use of such tools is becoming more common, a number of questions have arisen about whether using these tools is fair, or in some cases, even legal (K.W. v. Armstrong, 2015; ACLU, Outten & Golden LLP, and the Communications Workers of America, 2019).

References

[1]

17/EN. WP 251. (2017). Guidelines on Automated individual decision-making and Profiling for the purposes of Regulation 2016/679

[2]

ACLU, Outten & Golden LLP, and the Communications Workers of America. (2019). Facebook EEOC complaints. https://www.aclu.org/cases/facebook-eeoc-complaints. (Online; accessed June 1, 2019)

[3]

Alexander v. Choate, 469 U.S. 287 (1985)

[4]

Allegheny County. (2019). DHS funding. https://www.county.allegheny.pa.us/Human-Services/About/Funding-Sources.aspx. (Online; accessed May 18, 2019)

[5]

Alpaydin, E. (2009). Introduction to machine learning. MIT press.

Digital Library

[6]

Angwin, J., Larson, J., Mattu, S., & Kirchner, L. (2016). Machine bias. ProPublica, May, 23.

[7]

Arlington Heights v. Metropolitan Housing Dev. Corp., 429 U.S. 252 (1977)

[8]

Barocas, S., & Selbst, A. D. (2016). Big data's disparate impact. Calif. L. Rev., 104, 671.

[9]

Buolamwini, J., & Gebru, T. (2018). Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency (pp. 77--91).

[10]

Casey, B., Farhangi, A., & Vogl, R. (2018). Rethinking explainable machines: The GDPR's "right to explanation" debate and the rise of algorithmic audits in enterprise.

[11]

Chouldechova, A., Benavides-Prado, D., Fialko, O., & Vaithianathan, R. (2018). A case study of algorithm-assisted decision making in child maltreatment hotline screening decisions. In Conference on Fairness, Accountability and Transparency (pp. 134--148).

[12]

Civil Rights Act of 1964 Title VI, 78 Stat. 252, 42 U. S. C. § 2000d.

[13]

Civil Rights Act of 1964 Title VII, 42 U.S.C. §2000e et seq.

[14]

Corbett-Davies, S., & Goel, S. (2018). The measure and mismeasure of fairness: A critical review of fair machine learning. arXiv preprint arXiv:1808.00023.

[15]

Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., & Huq, A. (2017). Algorithmic decision making and the cost of fairness. In Proceedings of the 23rd acm sigkdd international conference on knowledge discovery and data mining (pp. 797--806).

Digital Library

[16]

Craig v. Boren, 429 U.S. 190 (1976))

[17]

Desmarais, S. L., & Lowder, E. M. (2019). Pre-trial risk assessment tools: A primer for judges, prosecutors, and defense attorneys. MacArthur Foundation Safety and Justice Challenge.

[18]

Draft Title VI Guidance for EPA Assistance Recipients Administering Environmental Permitting Programs (Draft Recipient Guidance) and Draft Revised Guidance for Investigating Title VI Administrative Complaints Challenging Permits (Draft Revised Investigation Guidance); Notice, 65 Fed. Reg. 124 (June 27, 2000). Federal Register: The Daily Journal of the United States.

[19]

Eubanks, V. (2018). Automating inequality How high-tech tools profile, police, and punish the poor. St. Martin's Press.

Digital Library

[20]

Fishel, S., Flack, D., & DeMatteo, D. (2018). Computer risk algorithms and judicial decision-making. Monitor on Psychology.

[21]

Gajane, P., & Pechenizkiy, M. (2017). On formalizing fairness in prediction with machine learning. arXiv preprint arXiv:1710.03184.

[22]

Giammarise, K. (2017). Allegheny County DHS using algorithm to assist in child welfare screening. https://www.post-gazette.com/local/region/2017/04/09/Allegheny-County-using-algorithm-to-assist-in-child-welfare-screening/stories/201701290002. (Online; accessed May 18, 2019)

[23]

Goodman, B., & Flaxman, S. (2017). European Union regulations on algorithmic decision-making and a "right to explanation". AI Magazine, 38(3), 50--57.

[24]

Griggs v. Duke Power Co., 401 U.S. 424 (1971)

[25]

Hawkins, D. (2019). The cybersecurity 202 Why a privacy law like gdpr would be a tough sell in the U.S. https://www.washingtonpost.com/news/powerpost/paloma/the-cybersecurity-202/2018/05/25/the-cybersecurity-202-why-a-privacy-law-like-gdpr-would-be-a-tough-sell-in-the-u-s/5b07038b1b326b492dd07e83/?utm_term=.1cc41e57f9cf. (Online; accessed May 18, 2019)

[26]

Hurley, D. (2018). Can an algorithm tell when kids are in danger. New York Times, 2.

[27]

Jung, J., Corbett-Davies, S., Shroff, R., & Goel, S. (2018). Omitted and included variable bias in tests for disparate impact. arXiv preprint arXiv:1809.05651.

[28]

Kleinberg, J., Mullainathan, S., & Raghavan, M. (2016). Inherent trade-offs in the fair determination of risk scores. arXiv preprint arXiv:1609.05807.

[29]

Kroll, J. A., Barocas, S., Felten, E. W., Reidenberg, J. R., Robinson, D. G., & Yu, H. (2016). Accountable algorithms. U. Pa. L. Rev., 165, 633.

[30]

K.W. v. Armstrong, No. 14--35296 (9th Cir. 2015)

[31]

Latessa, E., Smith, P., Lemke, R., Makarios, M., & Lowenkamp, C. (2009). Creation and validation of the Ohio risk assessment system: Final report. Cincinati, OH: University of Cincinnati.

[32]

Lau v. Nichols, 414 U.S. 563 (1974)

[33]

Lawson, R. P. (2019). GDPR enforcement actions, fines pile up. https://www.manatt.com/Insights/Newsletters/Advertising-Law/GDPR-Enforcement-Actions-Fines-Pile-Up. (Online; accessed May 18, 2019)

[34]

Lecher, C. (2018). What happens when an algorithm cuts your health care. The Verge.

[35]

Legal Information Institute. (2018a). Equal protection. https://www.law.cornell.edu/wex/equal_protection. (Online; accessed May 18, 2019)

[36]

Legal Information Institute. (2018b). Intermediate scrutiny. https://www.law.cornell.edu/wex/intermediate.scrutiny. (Online; accessed June 1, 2019)

[37]

Lewis, N. (2018). Will AI remove hiring bias? https://www.shrm.org/resourcesandtools/hr-topics/talent-acquisition/pages/will-ai-remove-hiring-bias-hr-technology.aspx. (Online; accessed May 18, 2019)

[38]

Mank, B. C. (2000). The draft recipient guidance and the draft revised investigation guidance: Too much discretion for epa and a more difficult standard for complainants? Environmental Law Reporter, 30.

[39]

Misra, T. (2018). When criminalizing the poor goes high-tech. https://www.citylab.com/equity/2018/02/the-rise-of-digital-poorhouses/552161/?platform=hootsuite. (Online; accessed May 18, 2019)

[40]

NJ Rev Stat § 2A:162-25 (2014)

[41]

O.J. (L 119) (2016). Reg (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Dir 95/46/EC (General Data Protection Regulation)

[42]

Personnel Adm'r of Massachusetts v. Feeney, 442 U.S. 256 (1979)

[43]

Reg (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Dir 95/46/EC (General Data Protection Regulation). (2016).

[44]

Ricci v. DeStefano, 557 U.S. 557 (2009)

[45]

Selmi, M. (2005). Was the disparate impact theory a mistake. Ucla L. Rev., 53, 701.

[46]

Skeem, J., Monahan, J., & Lowenkamp, C. (2016). Gender, risk assessment, and sanctioning: The cost of treating women like men. Law and human behavior, 40(5), 580.

[47]

Starr, S. B. (2014). Evidence-based sentencing and the scientific rationalization of discrimination. Stan. L. Rev., 66, 803.

[48]

State v. Loomis, 881 N.W.2d 749 (2016)

[49]

Steinhardt, E. (2018). European regulators are intensifying GDPR enforcement. https://www.insideprivacy.com/eu-data-protection/european-regulators-are-intensifying-gdpr-enforcement/. (Online; accessed May 18, 2019)

[50]

Summers, C., & Willis, T. (2010). Pre-trial risk assessment research summary. Washington, DC: Bureau of Justice Assistance.

[51]

United States. Department of Justice. (2019). Title VI Legal Manual (Updated)

[52]

U.S. Government Accountability Office. (2019). Our new science, technology assessment, and analytics team. https://blog.gao.gov/2019/01/29/our-new-science-technology-assessment-and-analytics-team/. (Online; accessed May 18, 2019)

[53]

U.S. Const. amend. V.

[54]

U.S. Const. amend. XIV.

[55]

VanNostrand, M., et al. (2009). Pretrial risk assessment in Virginia.

[56]

Weinzweig, M. J. (1983). Discriminatory impact and intent under the equal protection clause: The Supreme Court and the mind-body problem. Law & Ineq., 1, 277.

[57]

Zliobaite, I. (2015). On the relation between accuracy and fairness in binary classification. arXiv preprint arXiv:1505.05723.

Cited By

Eiermann M(2024)Algorithmic Risk Scoring and Welfare State Contact Among US ChildrenSociological Science10.15195/v11.a2611(707-742)Online publication date: 2024
https://doi.org/10.15195/v11.a26
Marjanovic OCecez-Kecmanovic DVidgen R(2021)Algorithmic pollution: Making the invisible visibleJournal of Information Technology10.1177/0268396221101035636:4(391-408)Online publication date: 28-May-2021
https://doi.org/10.1177/02683962211010356
G Johnson ICrivellaro CKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)Opening Research Commissioning To Civic Participation: Creating A Community Panel To Review The Social Impact of HCI Research ProposalsProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445113(1-17)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445113
Show More Cited By

Recommendations

The Bangladesh Justice Audit
ICEGOV '17: Proceedings of the 10th International Conference on Theory and Practice of Electronic Governance

The Bangladesh Constitution guarantees human rights, freedom, equality, justice and rule of law for all citizens. Article 33(1) ensures the rights for each citizen to legal support and article 35(3) affirms that every person accused of a criminal ...
Algorithmic Reproductive Justice
FAccT '24: Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency

Reproductive justice is an intersectional feminist framework and movement which argues all people have the right to have a child, to not have a child, to parent in safe and healthy environments, and to own their bodies and control their futures. We ...
Introduction to Criminal Justice Information Systems

Comments

Information & Contributors

Information

Published In

cover image AI Matters

AI Matters Volume 5, Issue 2

June 2019

44 pages

EISSN:2372-3483

DOI:10.1145/3340470

Editors:
Amy McGovern
U. Oklahoma
,
Iolanda Leite
KTH

Issue’s Table of Contents

Copyright © 2019 Authors.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 August 2019

Published in SIGAI-AIMATTERS Volume 5, Issue 2

Check for updates

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
188
Total Downloads

Downloads (Last 12 months)18
Downloads (Last 6 weeks)5

Reflects downloads up to 27 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Eiermann M(2024)Algorithmic Risk Scoring and Welfare State Contact Among US ChildrenSociological Science10.15195/v11.a2611(707-742)Online publication date: 2024
https://doi.org/10.15195/v11.a26
Marjanovic OCecez-Kecmanovic DVidgen R(2021)Algorithmic pollution: Making the invisible visibleJournal of Information Technology10.1177/0268396221101035636:4(391-408)Online publication date: 28-May-2021
https://doi.org/10.1177/02683962211010356
G Johnson ICrivellaro CKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)Opening Research Commissioning To Civic Participation: Creating A Community Panel To Review The Social Impact of HCI Research ProposalsProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445113(1-17)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445113
Marjanovic OCecez-Kecmanovic DVidgen R(2021)Theorising Algorithmic JusticeEuropean Journal of Information Systems10.1080/0960085X.2021.193413031:3(269-287)Online publication date: 21-Jun-2021
https://doi.org/10.1080/0960085X.2021.1934130
Rodolfa KSalomon EHaynes LMendieta ILarson JGhani RHildebrandt MCastillo CCelis ERuggieri STaylor LZanfir-Fortuna G(2020)Case studyProceedings of the 2020 Conference on Fairness, Accountability, and Transparency10.1145/3351095.3372863(142-153)Online publication date: 27-Jan-2020
https://dl.acm.org/doi/10.1145/3351095.3372863

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents