Skip to main content
Springer Nature Link
Log in

A Large Language Model Agent Based Legal Assistant for Governance Applications

  • Conference paper
  • First Online:
Electronic Government (EGOV 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14841))

Included in the following conference series:

Abstract

Large Language Models (LLMs) have gained significant traction, primarily due to their potential disruptive influence across industries reliant on natural language processing . Governance stands out as one such sector. Notably, there has been a surge in research activity surrounding the implications of LLMs in deciphering complex legal corpora. This research offers substantial assistance to various stakeholders, including decision-makers, administrators, and citizens. This article focuses on the design and implementation of an LLM-based legal assistant tailored for interacting with legal resources. To achieve this, a real-world scenario has been chosen, incorporating models GPT3.5 and GPT4 as the LLMs, a well-defined legal corpus comprising European Union (EU) legislation and case law concerning the General Data Protection Regulation (GDPR), alongside a series of reference legal queries of varying complexity. Retrieval Augmented Generation (RAG) as well as agent methodologies are employed to seamlessly integrate the LLMs’ functionalities with the customized dataset. The results appear to be promising, as the system managed to correctly address the majority of the legal queries, though with variable precision. Expectantly, the complexity of the queries severely impacted the quality of the outcome.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://www.parla.berlin.

  2. 2.

    https://eur-lex.europa.eu.

References

  1. Achiam, J., et al.: GPT-4 technical report. arXiv preprint arXiv:2303.08774 (2023)

  2. Brown, T., et al.: Language models are few-shot learners. In: Advances in Neural Information Processing Systems, vol. 33, pp. 1877–1901 (2020)

    Google Scholar 

  3. Colombo, P., et al.: SaulLM-7B: a pioneering large language model for law. arXiv preprint arXiv:2403.03883 (2024)

  4. Dahl, M., Magesh, V., Suzgun, M., Ho, D.E.: Large legal fictions: profiling legal hallucinations in large language models. arXiv preprint arXiv:2401.01301 (2024)

  5. Office of the Federal Register, National Archives and Records Administration: 88 FR 75191 - safe, secure, and trustworthy development and use of artificial intelligence. [government]. Federal Register (2023). https://www.govinfo.gov/app/details/FR-2023-11-01/2023-24283

  6. Fitsilis, F.: Artificial Intelligence (AI) in parliaments - preliminary analysis of the Eduskunta experiment. J. Legislative Stud. 27(4), 621–633 (2021). https://doi.org/10.1080/13572334.2021.1976947

    Article  Google Scholar 

  7. Fitsilis, F., Theodorakopoulos, G.: Better regulation and its evolution in the Hellenic legislative and parliamentary system. Statute Law Rev. 45(1), hmae003 (2024)

    Google Scholar 

  8. Green, S., Hurst, L., Nangle, B., Cunningham, P., Somers, F., Evans, R.: Software agents: a review. Technical report TCS-CS-1997-06, Department of Computer Science, Trinity College Dublin (1997)

    Google Scholar 

  9. Gruske, C.: Alberta courts caution against using unverified citations generated by AI or large language models (2023). https://www.canadianlawyermag.com/practice-areas/litigation/alberta-courts-caution-against-using-unverified-citations-generated-by-ai-or-large-language-models/380383

  10. Harris, M., Wilson, A.: Representative bodies in the AI era: insights for legislatures. https://www.popvox.org/ai-vol1

  11. Helberger, N., Diakopoulos, N.: ChatGPT and the AI act. Internet Policy Rev. 12(1) (2023). https://doi.org/10.14763/2023.1.1682

  12. Jiang, A.Q., et al.: Mistral 7B. arXiv preprint arXiv:2310.06825 (2023)

  13. Jiang, A.Q., et al.: Mixtral of experts. arXiv preprint arXiv:2401.04088 (2024)

  14. Kalampokis, E., Karacapilidis, N., Tsakalidis, D., Tarabanis, K.: Understanding the use of emerging technologies in the public sector: a review of horizon 2020 projects. Digit. Gov. Res. Pract. 4(1), 1–28 (2023)

    Article  Google Scholar 

  15. Ke, Y.H., et al.: Enhancing diagnostic accuracy through multi-agent conversations: using large language models to mitigate cognitive bias. arXiv preprint arXiv:2401.14589 (2024)

  16. Kojima, T., Gu, S.S., Reid, M., Matsuo, Y., Iwasawa, Y.: Large language models are zero-shot reasoners. In: Advances in Neural Information Processing Systems, vol. 35, pp. 22199–22213 (2022)

    Google Scholar 

  17. Lai, J., Gan, W., Wu, J., Qi, Z., Yu, P.S.: Large language models in law: a survey. arXiv preprint arXiv:2312.03718 (2023)

  18. Laios, A., et al.: RoBERTa-assisted outcome prediction in ovarian cancer cytoreductive surgery using operative notes. Cancer Control 30, 10732748231209892 (2023)

    Article  Google Scholar 

  19. Laios, A., Theophilou, G., De Jong, D., Kalampokis, E.: The future of AI in ovarian cancer research: the large language models perspective (2023)

    Google Scholar 

  20. Lewis, P., et al.: Retrieval-augmented generation for knowledge-intensive NLP tasks. In: Advances in Neural Information Processing Systems, vol. 33, pp. 9459–9474 (2020)

    Google Scholar 

  21. Listorti, G., Basyte-Ferrari, E., Acs, S., Smits, P.: Towards an evidence-based and integrated policy cycle in the EU: a review of the debate on the better regulation agenda. JCMS: J. Common Market Stud. 58(6), 1558–1577 (2020)

    Google Scholar 

  22. von Lucke, J., Fitsilis, F., Etscheid, J.: Research and development agenda for the use of AI in parliaments. In: Proceedings of the 24th Annual International Conference on Digital Government Research, pp. 423–433 (2023)

    Google Scholar 

  23. Mamalis, M.E., Kalampokis, E., Karamanou, A., Brimos, P., Tarabanis, K.: Can large language models revolutionalize open government data portals? A case of using ChatGPT in statistics.gov.scot. In: Proceedings of the 27th Pan-Hellenic Conference on Progress in Computing and Informatics, pp. 53–59 (2023)

    Google Scholar 

  24. Cabinet Office and Central Digital and Data Office: Generative AI framework for HM government (2024). https://www.gov.uk/government/publications/generative-ai-framework-for-hmg

  25. Cabinet Office’s Central Digital and Data Office, Department for Science, Innovation & Technology (2024). https://www.nao.org.uk/reports/use-of-artificial-intelligence-in-government/

  26. Petroni, F., et al.: Language models as knowledge bases? arXiv preprint arXiv:1909.01066 (2019)

  27. Radford, A., et al.: Language models are unsupervised multitask learners. OpenAI Blog 1(8), 9 (2019)

    Google Scholar 

  28. Solove, D.J., Schwartz, P.M.: EU Data Protection and the GDPR. Aspen Publishing (2023)

    Google Scholar 

  29. Tonmoy, S., et al.: A comprehensive survey of hallucination mitigation techniques in large language models. arXiv preprint arXiv:2401.01313 (2024)

  30. Touvron, H., et al.: LLaMA: open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023)

  31. Touvron, H., et al.: LLaMA 2: open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 (2023)

  32. Van Noordt, C., Misuraca, G.: Exploratory insights on artificial intelligence for government in Europe. Soc. Sci. Comput. Rev. 40(2), 426–444 (2022)

    Article  Google Scholar 

  33. Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)

    Google Scholar 

  34. Webb, T., Holyoak, K.J., Lu, H.: Emergent analogical reasoning in large language models. Nat. Hum. Behav. 7(9), 1526–1541 (2023)

    Article  Google Scholar 

  35. Wei, J., et al.: Emergent abilities of large language models. arXiv preprint arXiv:2206.07682 (2022)

  36. Wei, J., et al.: Chain-of-thought prompting elicits reasoning in large language models. In: Advances in Neural Information Processing Systems, vol. 35, pp. 24824–24837 (2022)

    Google Scholar 

  37. Wooldridge, M., Jennings, N.R.: Intelligent agents: theory and practice. Knowl. Eng. Rev. 10(2), 115–152 (1995)

    Article  Google Scholar 

  38. Xi, Z., et al.: The rise and potential of large language model based agents: a survey. arXiv preprint arXiv:2309.07864 (2023)

  39. Xiao, C., Hu, X., Liu, Z., Tu, C., Sun, M.: Lawformer: a pre-trained language model for Chinese legal long documents. AI Open 2, 79–84 (2021)

    Article  Google Scholar 

  40. Yao, S., et al.: Tree of thoughts: deliberate problem solving with large language models. In: Advances in Neural Information Processing Systems, vol. 36 (2024)

    Google Scholar 

  41. Yin, R.K.: Case Study Research: Design and Methods, vol. 5. SAGE (2009)

    Google Scholar 

  42. Zhang, Y., et al.: Siren’s song in the AI ocean: a survey on hallucination in large language models. arXiv preprint arXiv:2309.01219 (2023)

  43. Zhou, Y., et al.: Large language models are human-level prompt engineers. arXiv preprint arXiv:2211.01910 (2022)

Download references

Author information

Authors and Affiliations

  1. Information Systems Laboratory, University of Macedonia, 54636, Thessaloniki, Greece

    Marios Evangelos Mamalis, Evangelos Kalampokis & Konstantinos Tarabanis

  2. Department for Scientific Documentation and Supervision, Hellenic Parliament, Athens, Greece

    Fotios Fitsilis

  3. Hellenic State Legal Council, Athens, Greece

    Georgios Theodorakopoulos

Authors
  1. Marios Evangelos Mamalis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Evangelos Kalampokis
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fotios Fitsilis
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Georgios Theodorakopoulos
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Konstantinos Tarabanis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marios Evangelos Mamalis .

Editor information

Editors and Affiliations

  1. Faculty of Technology, Policy and Management, Delft University of Technology, Delft, The Netherlands

    Marijn Janssen

  2. KU Leuven, Leuven, Belgium

    Joep Crompvoets

  3. University at Albany, State University of New York, Albany, NY, USA

    J. Ramon Gil-Garcia

  4. Brunel University London, Uxbridge, UK

    Habin Lee

  5. Linköping University, Linköping, Sweden

    Ida Lindgren

  6. Institute of Computer Science, University of Tartu, Tartu, Estonia

    Anastasija Nikiforova

  7. Krems Center for E-Governance, University for Continuing Education, Krems, Niederösterreich, Austria

    Gabriela Viale Pereira

Rights and permissions

Reprints and permissions

Copyright information

© 2024 IFIP International Federation for Information Processing

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Mamalis, M.E., Kalampokis, E., Fitsilis, F., Theodorakopoulos, G., Tarabanis, K. (2024). A Large Language Model Agent Based Legal Assistant for Governance Applications. In: Janssen, M., et al. Electronic Government. EGOV 2024. Lecture Notes in Computer Science, vol 14841. Springer, Cham. https://doi.org/10.1007/978-3-031-70274-7_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-70274-7_18

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-70273-0

  • Online ISBN: 978-3-031-70274-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships