Abstract
Based on a real world use case, we developed and evaluated a hybrid AI system that aims to extract key elements from legal permits by combining methods from the Semantic Web and Machine Learning. Specifically, we modelled the available background knowledge in a custom Knowledge Graph, which we exploited together with the usage of different language- and text-embedding-models in order to extract different information from official Austrian permits, including the Issuing Authority, the Operator of the facility in question, the Reference Number, and the Issuing Date. Additionally, we implemented mechanisms to capture automatically auditable traces of the system to ensure the transparency of the processes. Our quantitative evaluation showed overall promising results, while the in-depth qualitative analysis revealed concrete error types, providing guidance on how to improve the current prototype.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
The link to the online version will be made available upon acceptance.
- 3.
- 4.
- 5.
Please note, that version 2.1 has not yet officially been released yet, but is the latest develop branch of the ontology.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
References
Breit, A., Revenko, A., Rezaee, K., Pilehvar, M.T., Camacho-Collados, J.: WiC-TSV: an evaluation benchmark for target sense verification of words in context. In: Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pp. 1635–1645. Association for Computational Linguistics (2021). https://doi.org/10.18653/v1/2021.eacl-main.140, https://aclanthology.org/2021.eacl-main.140
Eckart de Castilho, R., Gurevych, I.: A broad-coverage collection of portable NLP components for building shareable analysis pipelines. In: Proceedings of the Workshop on Open Infrastructures and Analysis Frameworks for HLT, pp. 1–11. Association for Computational Linguistics and Dublin City University, Dublin (2014). https://doi.org/10.3115/v1/W14-5201, https://aclanthology.org/W14-5201
Eiband, M., Schneider, H., Bilandzic, M., Fazekas-Con, J., Haug, M., Hussmann, H.: Bringing transparency design into practice. In: 23rd International Conference on Intelligent User Interfaces, pp. 211–223 (2018)
Ekaputra, F.J., et al.: Semantic-enabled architecture for auditable privacy-preserving data analysis. Semant. Web (Preprint), 1–34 (2021)
Hellmann, S., Lehmann, J., Auer, S., Brümmer, M.: Integrating NLP using linked data. In: Alani, H., Kagal, L., Fokoue, A., Groth, P., Biemann, C., Parreira, J.X., Aroyo, L., Noy, N., Welty, C., Janowicz, K. (eds.) ISWC 2013. LNCS, vol. 8219, pp. 98–113. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-41338-4_7
Janowicz, K., et al.: UnifiedViews: an ETL tool for rdf data management. Semant. Web 9(5), 661–676 (2018). https://doi.org/10.3233/SW-180291
Leitner, E., Rehm, G., Moreno-Schneider, J.: Fine-grained named entity recognition in legal documents. In: Acosta, M., Cudré-Mauroux, P., Maleshkova, M., Pellegrini, T., Sack, H., Sure-Vetter, Y. (eds.) SEMANTiCS 2019. LNCS, vol. 11702, pp. 272–287. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-33220-4_20
Liao, Q.V., Gruen, D., Miller, S.: Questioning the AI: informing design practices for explainable AI user experiences. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pp. 1–15 (2020)
Longley, D., Sporny, M., Kellogg, G., Lanthaler, M., Lindström, N.: JSON-LD 1.1 framing (2020). https://www.w3.org/TR/json-ld-framing/
Miles, S., Groth, P., Munroe, S., Moreau, L.: PrIMe: a methodology for developing provenance-aware applications. ACM Trans. Softw. Eng. Methodol. (TOSEM) 20(3), 1–42 (2011)
Mitchell, M., et al.: Model cards for model reporting. In: Proceedings of the Conference on Fairness, Accountability, and Transparency, pp. 220–229 (2019)
Moreno-Schneider, J., et al.: Orchestrating NLP services for the legal domain. In: Proceedings of the Twelfth Language Resources and Evaluation Conference, pp. 2332–2340. European Language Resources Association, Marseille (2020). https://aclanthology.org/2020.lrec-1.284
Naja, I., Markovic, M., Edwards, P., Cottrill, C.: A semantic framework to support AI system accountability and audit. In: Verborgh, R., et al. (eds.) ESWC 2021. LNCS, vol. 12731, pp. 160–176. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-77385-4_10
Ostendorff, M., Blume, T., Ostendorff, S.: Towards an open platform for legal information. In: Proceedings of the ACM/IEEE Joint Conference on Digital Libraries in 2020, JCDL 2020, pp. 385–388. Association for Computing Machinery, New York (2020). https://doi.org/10.1145/3383583.3398616
Raji, I.D., et al.: Closing the AI accountability gap: Defining an end-to-end framework for internal algorithmic auditing. In: Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pp. 33–44 (2020)
Acknowledgements
This work has been supported by OBARIS (https://www.obaris.org/), a project funded by the Austrian Research Promotion Agency (FFG) under grant 877389.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Breit, A., Waltersdorfer, L., Ekaputra, F.J., Karampatakis, S., Miksa, T., Käfer, G. (2023). Combining Semantic Web and Machine Learning for Auditable Legal Key Element Extraction. In: Pesquita, C., et al. The Semantic Web. ESWC 2023. Lecture Notes in Computer Science, vol 13870. Springer, Cham. https://doi.org/10.1007/978-3-031-33455-9_36
Download citation
DOI: https://doi.org/10.1007/978-3-031-33455-9_36
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-33454-2
Online ISBN: 978-3-031-33455-9
eBook Packages: Computer ScienceComputer Science (R0)