Abstract
Artificial Intelligence applied to Law is getting attention in the community, both from the Judiciary and the lawyers, due to possible gains in procedural celerity and the automation of repetitive tasks, among other use cases. Many of its benefits can be derived from Natural Language Processing since legal proceedings are textual document-based. Named Entity Recognition is the NLP task of identifying and classifying named entities in unstructured text to help achieve these goals. In recent years, transformers emerged as the new state-of-the-art architecture for some tasks in NLP systems. NLP resources in Portuguese, such as models and datasets, are needed to allow Portuguese speaking countries to benefit from this new NLP level. This paper presents the first fine-tuned BERT models trained exclusively on Brazilian Portuguese for Legal NER. The models achieved new state-of-the-art on LeNER-Br dataset, a Portuguese NER corpus for the legal domain. We also built a prototype application for Judiciary users to evaluate how the model performed with authentic law documents. The results showed that the models were able to extract information with good quality. Both the models and the prototype application are publicly available for the community.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
References
Bonifacio, L.H., Vilela, P.A., Lobato, G.R., Fernandes, E.R.: A study on the impact of intradomain finetuning of deep language models for legal named entity recognition in Portuguese. In: Cerri, R., Prati, R.C. (eds.) BRACIS 2020. LNCS (LNAI), vol. 12319, pp. 648–662. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-61377-8_46
Ciurlino, V.H.: BertBR : a pretrained language model for law texts. Trabalho de Conclusão de Curso (Bacharelado em Engenharia Eletrônica). Universidade de Brasília (2021)
Devlin, J., et al.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Vol. 1 (Long and Short Papers), pp. 4171–4186. Association for Computational Linguistics, Minneapolis, Minnesota (2019). https://doi.org/10.18653/v1/N19-1423
Dozier, C., Kondadadi, R., Light, M., Vachher, A., Veeramachaneni, S., Wudali, R.: Named entity recognition and resolution in legal text. In: Francesconi, E., Montemagni, S., Peters, W., Tiscornia, D. (eds.) Semantic Processing of Legal Texts. LNCS (LNAI), vol. 6036, pp. 27–43. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-12837-0_2
Freitas, C., et al.: Second HAREM: advancing the state of the art of named entity recognition in Portuguese. In: Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC’10). European Language Resources Association (ELRA), Valletta, Malta (2010)
Howard, J., Ruder, S.: Universal language model fine-tuning for text classification. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 328–339. Association for Computational Linguistics, Melbourne, Australia (2018). https://doi.org/10.18653/v1/P18-1031
Luz de Araujo, P.H., de Campos, T.E., de Oliveira, R.R.R., Stauffer, M., Couto, S., Bermejo, P.: LeNER-Br: a dataset for named entity recognition in brazilian legal text. In: Villavicencio, A., et al. (eds.) PROPOR 2018. LNCS (LNAI), vol. 11122, pp. 313–323. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-99722-3_32
Marangunić, N., Granić, A.: Technology acceptance model: a literature review from 1986 to 2013. Univ. Access Inf. Soc. 14(1), 81–95 (2014). https://doi.org/10.1007/s10209-014-0348-1
Mendonça Júnior, C., et al.: Paramopama: a Brazilian-Portuguese corpus for named entity recognition. Presented at the XII Encontro Nacional de Inteligência Artificial e Computacional (ENIAC) (2015)
Menezes, D.S., et al.: Building a Massive Corpus for Named Entity Recognition using Free Open Data Sources. arXiv:190805758Cs (2019)
Mikolov, T., et al.: Efficient Estimation of Word Representations in Vector Space. arXiv:13013781Cs (2013)
Mohit, B.: Named entity recognition. In: Zitouni, I. (ed.) Natural Language Processing of Semitic Languages. TANLP, pp. 221–245. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-642-45358-8_7
Nothman, J., et al.: Learning multilingual named entity recognition from Wikipedia. Artif. Intell. 194, 151–175 (2013). https://doi.org/10.1016/j.artint.2012.03.006
Pennington, J., et al.: GloVe: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). pp. 1532–1543. Association for Computational Linguistics, Doha, Qatar (2014). https://doi.org/10.3115/v1/D14-1162
Peters, M.E., et al.: Deep contextualized word representations. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). pp. 2227–2237. Association for Computational Linguistics, New Orleans, Louisiana (2018). https://doi.org/10.18653/v1/N18-1202
Santos, D., Cardoso, N.: A golden resource for named entity recognition in Portuguese. In: Vieira, R., Quaresma, P., Nunes, M.D.G.V., Mamede, N.J., Oliveira, C., Dias, M.C. (eds.) PROPOR 2006. LNCS (LNAI), vol. 3960, pp. 69–79. Springer, Heidelberg (2006). https://doi.org/10.1007/11751984_8
Souza, F., Nogueira, R., Lotufo, R.: BERTimbau: pretrained BERT models for Brazilian Portuguese. In: Cerri, R., Prati, R.C. (eds.) BRACIS 2020. LNCS (LNAI), vol. 12319, pp. 403–417. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-61377-8_28
Vaswani, A., et al.: Attention is all you need. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. pp. 6000–6010. Curran Associates Inc., Red Hook, NY, USA (2017)
Wagner Filho, J.A., et al.: The brWaC corpus: a new open resource for Brazilian Portuguese. In: Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018). European Language Resources Association (ELRA), Miyazaki, Japan (2018)
Willian Sousa, A., Fabro, M.: Iudicium textum dataset uma base de textos jurídicos para NLP. In: 2nd Dataset Showcase Workshop at SBBD (Brazilian Symposium on Databases). Fortaleza, Brazil (2019)
Wolf, T., et al.: Transformers: state-of-the-art natural language processing. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. pp. 38–45. Association for Computational Linguistics, Online (2020). https://doi.org/10.18653/v1/2020.emnlp-demos.6
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Zanuz, L., Rigo, S.J. (2022). Fostering Judiciary Applications with New Fine-Tuned Models for Legal Named Entity Recognition in Portuguese. In: Pinheiro, V., et al. Computational Processing of the Portuguese Language. PROPOR 2022. Lecture Notes in Computer Science(), vol 13208. Springer, Cham. https://doi.org/10.1007/978-3-030-98305-5_21
Download citation
DOI: https://doi.org/10.1007/978-3-030-98305-5_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-98304-8
Online ISBN: 978-3-030-98305-5
eBook Packages: Computer ScienceComputer Science (R0)