skip to main content
10.1145/3306618.3314279acmconferencesArticle/Chapter ViewAbstractPublication PagesaiesConference Proceedingsconference-collections
research-article

The Right To Confront Your Accusers: Opening the Black Box of Forensic DNA Software

Published: 27 January 2019 Publication History

Abstract

The results of forensic DNA software systems are regularly introduced as compelling evidence in criminal trials, but requests by defendants to evaluate how these results are generated are often denied. Furthermore, there is mounting evidence of problems such as failures to disclose substantial changes in methodology to oversight bodies and substantial differences in the results generated by different software systems. In a society that purports to guarantee defendants the right to face their accusers and confront the evidence against them, what then is the role of black-box forensic software systems in moral decision making in criminal justice? In this paper, we examine the case of the Forensic Statistical Tool (FST), a forensic DNA system developed in 2010 by New York City's Office of Chief Medical Examiner (OCME). For over 5 years, expert witness review requested by defense teams was denied, even under protective order, while the system was used in over 1300 criminal cases. When the first expert review was finally permitted in 2016, many problems were identified including an undisclosed function capable of dropping evidence that could be beneficial to the defense. Overall, the findings were so substantial that a motion to release the full source code of FST publicly was granted. In this paper, we quantify the impact of this undisclosed function on samples from OCME's own validation study and discuss the potential impact on individual defendants. Specifically, we find that 104 of the 439 samples (23.7%) triggered the undisclosed data-dropping behavior and that the change skewed results toward false inclusion for individuals whose DNA was not present in an evidence sample. Beyond this, we consider what changes in the criminal justice system could prevent problems like this from going unresolved in the future.

References

[1]
Federica Alessandrini, Monia Cecati, Mauro Pesaresi, Chiara Turchi, Flavia Carle, and Adriano Tagliabracci. 2003. Fingerprints as evidence for a genetic profile: morphological study on fingerprints and analysis of exogenous and individual factors affecting DNA typing. Journal of forensic sciences, Vol. 48, 3 (2003), 586--592.
[2]
J Angwin, J Larson, S Mattu, and L Kirchner. 2016. Machine Bias-ProPublica.{online} ProPublica.
[3]
Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on Fairness, Accountability and Transparency. 77--91.
[4]
John M Butler, Richard Schoske, Peter M Vallone, Janette W Redman, Margaret C Kline, et almbox. 2003. Allele frequencies for 15 autosomal STR loci on US Caucasian, African American, and Hispanic populations. Journal of forensic sciences, Vol. 48, 4 (2003), 908--911.
[5]
Alexandra Chouldechova. 2017. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big data, Vol. 5, 2 (2017), 153--163.
[6]
National Research Council et almbox. 1996. The evaluation of forensic DNA evidence .National Academies Press.
[7]
Paolo Garofano, Denise Caneparo, Giuseppina D'Amico, Marco Vincenti, and Eugenio Alladio. 2015. An alternative application of the consensus method to DNA typing interpretation for Low Template-DNA mixtures. Forensic Science International: Genetics Supplement Series, Vol. 5 (2015), e422--e424.
[8]
Hinda Haned, Klaas Slooten, and Peter Gill. 2012. Exploratory data analysis for the interpretation of low template DNA mixtures. Forensic Science International: Genetics, Vol. 6, 6 (2012), 762--774.
[9]
Lauren Kirchner. 2017. Thousands of Criminal Cases in New York Relied on Disputed DNA Testing Techniques.
[10]
Adele Mitchell. 2013. Testimony of Adele Mitchell in admissibility hearing in People v. Collins. 15 N.Y.S.3d 564 (Sup.Ct. Kings Co. 2015).
[11]
Adele A Mitchell, Jeannie Tamariz, Kathleen O'Connell, Nubia Ducasse, Zoran Budimlija, Mechthild Prinz, and Theresa Caragine. 2012. Validation of a DNA mixture statistics tool incorporating allelic drop-out and drop-in. Forensic Science International: Genetics, Vol. 6, 6 (2012), 749--761.
[12]
National Institute of Standards and Technology (NIST). 2017. NIST to Assess the Reliability of Forensic Methods for Analyzing DNA Mixtures. https://www.nist.gov/news-events/news/2017/10/nist-assess-reliability-forensic-methods-analyzing-dna-mixtures. (October 2017).
[13]
New York City Office of the Chief Medical Examiner (OCME). 2016. Forensic Biology Protocols for Forensic STR Analysis: Forensic Statistical Tool (FST). https://www1.nyc.gov/assets/ocme/downloads/pdf/technical-manuals/protocols-for-forensic-str-analysis/forensic-statistical-tool-fst.pdf .
[14]
New York City Office of the Chief Medical Examiner (OCME). 2018. Technical Manuals (OCME). https://www1.nyc.gov/site/ocme/services/technical-manuals.page .
[15]
David R Paoletti, Travis E Doom, Carissa M Krane, Michael L Raymer, and Dan E Krane. 2005. Empirical analysis of the STR profiles resulting from conceptual mixtures. Journal of Forensic Science, Vol. 50, 6 (2005), JFS2004475--6.
[16]
Christopher D Steele and David J Balding. 2014. Statistical evaluation of forensic DNA profile evidence. Annual Review of Statistics and Its Application, Vol. 1 (2014), 361--384.
[17]
J. Tashea. 2017. Defense lawyers want to peek behind the curtain of probabilistic genotyping. http://www.abajournal.com/magazine/article/code_of_science_defense_lawyers_want_to_peek_behind_the_curtain_of_probabil. Aba Journal (2017).
[18]
Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev., Vol. 70 (2018), 1343.
[19]
Katie Worth. 2018. Framed for Murder By His Own DNA. https://www.pbs.org/wgbh/frontline/article/framed-for-murder-by-his-own-dna/. (April 2018).

Cited By

View all
  • (2024)(Beyond) Reasonable Doubt: Challenges that Public Defenders Face in Scrutinizing AI in CourtProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641902(1-19)Online publication date: 11-May-2024
  • (2023)Unfairness Is Everywhere, so What to Do? An Interview With Jeanna MatthewsIEEE Software10.1109/MS.2023.330572240:6(135-138)Online publication date: 1-Nov-2023
  • (2023)Metamorphic Testing and Debugging of Tax Preparation SoftwareProceedings of the 45th International Conference on Software Engineering: Software Engineering in Society10.1109/ICSE-SEIS58686.2023.00019(138-149)Online publication date: 17-May-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
AIES '19: Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society
January 2019
577 pages
ISBN:9781450363242
DOI:10.1145/3306618
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 January 2019

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. algorithmic accountability
  2. criminal justice software
  3. forensic statistical tool (FST)
  4. probabilistic genotyping software

Qualifiers

  • Research-article

Funding Sources

  • Brown Institute

Conference

AIES '19
Sponsor:
AIES '19: AAAI/ACM Conference on AI, Ethics, and Society
January 27 - 28, 2019
HI, Honolulu, USA

Acceptance Rates

Overall Acceptance Rate 61 of 162 submissions, 38%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)26
  • Downloads (Last 6 weeks)5
Reflects downloads up to 22 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)(Beyond) Reasonable Doubt: Challenges that Public Defenders Face in Scrutinizing AI in CourtProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641902(1-19)Online publication date: 11-May-2024
  • (2023)Unfairness Is Everywhere, so What to Do? An Interview With Jeanna MatthewsIEEE Software10.1109/MS.2023.330572240:6(135-138)Online publication date: 1-Nov-2023
  • (2023)Metamorphic Testing and Debugging of Tax Preparation SoftwareProceedings of the 45th International Conference on Software Engineering: Software Engineering in Society10.1109/ICSE-SEIS58686.2023.00019(138-149)Online publication date: 17-May-2023
  • (2022)Adversarial Scrutiny of Evidentiary Statistical SoftwareProceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency10.1145/3531146.3533228(1733-1746)Online publication date: 21-Jun-2022
  • (2022)The Forgotten Margins of AI EthicsProceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency10.1145/3531146.3533157(948-958)Online publication date: 21-Jun-2022
  • (2021)Digital Privacy Perceptions of Asylum Seekers in Germany: An Empirical Study about Smartphone Usage during the FlightProceedings of the ACM on Human-Computer Interaction10.1145/34795265:CSCW2(1-24)Online publication date: 18-Oct-2021
  • (2021)Evaluating the Effectiveness of Deplatforming as a Moderation Strategy on TwitterProceedings of the ACM on Human-Computer Interaction10.1145/34795255:CSCW2(1-30)Online publication date: 18-Oct-2021
  • (2021)Learning to IgnoreProceedings of the ACM on Human-Computer Interaction10.1145/34491545:CSCW1(1-23)Online publication date: 22-Apr-2021
  • (2021)Anyone else have this experienceProceedings of the ACM on Human-Computer Interaction10.1145/34491535:CSCW1(1-30)Online publication date: 22-Apr-2021
  • (2021)Social-Emotional-Sensory Design Map for Affective Computing Informed by Neurodivergent ExperiencesProceedings of the ACM on Human-Computer Interaction10.1145/34491515:CSCW1(1-37)Online publication date: 22-Apr-2021
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media