research-article

Substantive Legal Software Quality: A Gathering Storm?

Authors:

Marc Lauritsen,

Quinten SteenhuisAuthors Info & Claims

ICAIL '19: Proceedings of the Seventeenth International Conference on Artificial Intelligence and Law

Pages 52 - 62

https://doi.org/10.1145/3322640.3326706

Published: 17 June 2019 Publication History

Abstract

Readily available interactive programs dispense substantive legal guidance, often including bespoke documents. These are found across a wide spectrum of commercial and non-commercial contexts. Consumers are coming to rely on them as alternatives to expensive lawyer services. Yet their quality is uneven and difficult to assess. We are in danger of serious harm being done to unwitting users. How can we avoid an epidemic of artificial misinformation, systematic inaccuracy, and mechanical malpractice? This paper reviews how those dangers play out in real-world application contexts and explores ways in which the AI & Law community might help address them.

References

[1]

Al-Abdulkarim, L., Atkinson, K. and Bench-Capon, T., 2017. Angelic Environment: Support for the Construction of Legal KBS. Technical Report ULCS-17-002, University of Liverpool.

Digital Library

[2]

Bennett, J. et al. Current State of Automated Legal Advice Tools, Networked Society Institute Discussion Paper. April 2018. https://networkedsociety.unimelb.edu.au/__data/assets/pdf_file/0020/2761013/2018-NSI-CurrentStateofALAT.pdf

[3]

Conrad, J.G. and Zeleznikow, J., 2015. The role of evaluation in AI and Law: an examination of its different forms in the AI and Law journal. In Proceedings of the 15th international conference on artificial intelligence and law (pp. 181--186). ACM.

Digital Library

[4]

Dimyadi, J., Bookman, S., Harvey, D. and Amor, R., 2019. Maintainable process model driven online legal expert systems. Artificial Intelligence and Law, 27(1), pp. 93--111.

Digital Library

[5]

Faciano, C., Mera, S., Schapachnik, F., Di Iorio, A.H., Clara, B.L., Uriarte, V., Giaccaglia, M.F., Ruffa, M.B. and Marcos, C., 2017. Performance improvement on legal model checking. In Proceedings of the 16th edition of the International Conference on Articial Intelligence and Law (pp. 59--68). ACM.

Digital Library

[6]

Hagan, M., Colarusso, D., 2018. Learned Hands: What is It. Retrieved from https://learnedhands.law.stanford.edu/, archived at https://perma.cc/2VXG-CYHT

[7]

Hokkanen, J. and Lauritsen, M., 2002. Knowledge tools for legal knowledge tool makers. Artificial Intelligence and Law, 10(4), pp. 295--302.

Digital Library

[8]

Hvitved, T. "Contract Formalisation and Modular Implementation of Domain-Specific Languages"---Ph.D. dissertation, The Faculty of Science, University of Copenhagen. https://drive.google.com/file/d/0BxOaYa8pqqSwbl9GMWtwVU5HSFU/view

[9]

Jacobson, I., 2019. What is Essence? Retrieved from https://www.ivarjacobson.com/services/what-essence

[10]

Lauritsen, M., 1992. Technology report: Building legal practice systems with today's commercial authoring tools. Artificial Intelligence and Law, 1(1), pp. 87--102.

Digital Library

[11]

Morris, J., 2018. Legal Expert Systems Just Got Smarter. (October 2018). Retrieved from https://medium.com/@jason90344/legal-expert-systems-just-got-smarter-e7e12b75e872, archived at https://perma.cc/URL5-HQSA

[12]

Muthuri, R., Boella, G., Hulstijn, J., Capecchi, S. and Humphreys, L., 2017. Compliance patterns: harnessing value modeling and legal interpretation to manage regulatory conversations. In Proceedings of the 16th edition of the International Conference on Articial Intelligence and Law (pp. 139--148). ACM.

Digital Library

[13]

Ramakrishna, S. et al., Legal Vocabulary and its Transformation Evaluation using Competency Questions, 2015. In Proceedings of the 15th International Conference on Artificial Intelligence and Law (pp. 211--215). ACM.

[14]

Steenhuis, Q., 2018. Separating Interview Logic from the Law. (June 2018). Retrieved from https://www.nonprofittechy.com/2018/06/28/separating-interview-logic-from-the-law/, archived at https://perma.cc/2M3S-S5X6

[15]

Von Rossum, G., Warsaw, B., Coghlan, N., 2013. PEP 8 -- Style Guide for Python Code. (August 2013). Retrieved on April 24 2019 from https://www.python.org/dev/peps/pep-0008/.

[16]

Wong, M. et al, 2019. L4: a domain-specific language (DSL) for law. Retrieved from https://legalese.com/#L4, archived at https://perma.cc/V8T7-3AZT

Cited By

Escher NBilik JBanovic NGreen B(2024)Code-ifying the Law: How Disciplinary Divides Afflict the Development of Legal SoftwareProceedings of the ACM on Human-Computer Interaction10.1145/36869378:CSCW2(1-37)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3686937
Steenhuis QWilley BColarusso DAndrade FGrabmair M(2023)Beyond Readability with RateMyPDFProceedings of the Nineteenth International Conference on Artificial Intelligence and Law10.1145/3594536.3595146(287-296)Online publication date: 19-Jun-2023
https://dl.acm.org/doi/10.1145/3594536.3595146
Linna D(2021)Evaluating Artificial Intelligence for Legal Services: Can “Soft Law” Lead to Enforceable Standards for Effectiveness?IEEE Technology and Society Magazine10.1109/MTS.2021.312373240:4(37-51)Online publication date: Dec-2021
https://doi.org/10.1109/MTS.2021.3123732
Show More Cited By

Index Terms

Substantive Legal Software Quality: A Gathering Storm?
1. Human-centered computing
  1. Interaction design
2. Software and its engineering
  1. Software organization and properties
    1. Software functional properties
      1. Correctness
      2. Formal methods
        Software verification

Recommendations

Improving legal quality: an application report
ICAIL '03: Proceedings of the 9th international conference on Artificial intelligence and law

Problems with legal quality will not only increase effort and costs of the law enforcement organisations, but also undermines the regulating power of the legislator. Unintended use or even abuse of the law may be the result. Governments therefore should ...
Acceptable Legal Standards for Software

The author discusses the legal issues that are involved in software protection. She discusses legal actions open to users who get a system that does not perform as expected and have exhausted all remedies that do not involve litigation. She examines how ...
The use of legal software by non-lawyers and the perils of unauthorised practice of law charges in the United States: a review of Jayson Reynoso decision

This paper critically reviews the judgment of the United States Court of Appeals for the Ninth Circuit In re: Jayson Reynoso: Frankfort Digital Services et al., v. Sara L. Kistler, United States Trustee et al. (2007) 447 F.3d 1117. The appellants, who ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICAIL '19: Proceedings of the Seventeenth International Conference on Artificial Intelligence and Law

June 2019

312 pages

ISBN:9781450367547

DOI:10.1145/3322640

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGAI: ACM Special Interest Group on Artificial Intelligence

In-Cooperation

Univ. of Montreal: University of Montreal
AAAI
IAAIL: Intl Asso for Artifical Intel & Law

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 June 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article
Research
Refereed limited

Conference

ICAIL '19

Sponsor:

SIGAI

ICAIL '19: Seventeenth International Conference on Artificial Intelligence and Law

June 17 - 21, 2019

QC, Montreal, Canada

Acceptance Rates

Overall Acceptance Rate 69 of 169 submissions, 41%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
335
Total Downloads

Downloads (Last 12 months)48
Downloads (Last 6 weeks)11

Reflects downloads up to 08 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Escher NBilik JBanovic NGreen B(2024)Code-ifying the Law: How Disciplinary Divides Afflict the Development of Legal SoftwareProceedings of the ACM on Human-Computer Interaction10.1145/36869378:CSCW2(1-37)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3686937
Steenhuis QWilley BColarusso DAndrade FGrabmair M(2023)Beyond Readability with RateMyPDFProceedings of the Nineteenth International Conference on Artificial Intelligence and Law10.1145/3594536.3595146(287-296)Online publication date: 19-Jun-2023
https://dl.acm.org/doi/10.1145/3594536.3595146
Linna D(2021)Evaluating Artificial Intelligence for Legal Services: Can “Soft Law” Lead to Enforceable Standards for Effectiveness?IEEE Technology and Society Magazine10.1109/MTS.2021.312373240:4(37-51)Online publication date: Dec-2021
https://doi.org/10.1109/MTS.2021.3123732
Branting LPfeifer CBrown BFerro LAberdeen JWeiss BPfaff MLiao B(2020)Scalable and explainable legal predictionArtificial Intelligence and Law10.1007/s10506-020-09273-1Online publication date: 24-Jun-2020
https://doi.org/10.1007/s10506-020-09273-1

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten