Abstract
Currently, there is a vast amount of archival finding aids in Portuguese archives, however, these documents lack structure (are not annotated) making them hard to process and work with. In this way, we intend to extract and classify entities of interest, like geographical locations, people’s names, dates, etc. For this, we will use an architecture that has been revolutionizing several NLP tasks, Transformers, presenting several models in order to achieve high results. It is also intended to understand what will be the degree of improvement that this new mechanism will present in comparison with previous architectures. Can Transformer-based models replace the LSTMs in NER? We intend to answer this question along this paper.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Alammar, J.: The illustrated transformer. http://jalammar.github.io/illustrated-transformer. Accessed 18 July 2021
Alvi, A., Kharya, P.: Using deepspeed and megatron to train megatron-turing NLG 530b, the world’s largest and most powerful generative language model (2021). https://www.microsoft.com/en-us/research/blog/using-deepspeed-and-megatron-to-train-megatron-turing-nlg-530b-the-worlds-largest-and-most-powerful-generative-language-model/. Accessed 15 Oct 2021
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate (2016)
Cunha, L.F.C., Ramalho, J.C.: http://ner.epl.di.uminho.pt/
Cunha, L.F.C., Ramalho, J.C.: NER in Archival Finding Aids (2021). https://doi.org/10.4230/OASIcs.SLATE.2021.8
Devlin, J., Chang, M., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. CoRR abs/1810.04805 (2018). http://arxiv.org/abs/1810.04805
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding (2019)
Howard, J., Ruder, S.: Universal language model fine-tuning for text classification (2018). https://doi.org/10.18653/v1/p18-1031
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space (2013)
Pennington, J., Socher, R., Manning, C.: GloVe: Global vectors for word representation (2014). https://doi.org/10.3115/v1/D14-1162
Radford, A., Narasimhan, K.: Improving language understanding by generative pre-training (2018)
Ruder, S.: NLP’s ImageNet moment has arrived. https://ruder.io/nlp-imagenet/ (2018). Accessed 07 Oct 2021
Russakovsky, O., et al.: ImageNet large scale visual recognition challenge (2015)
Souza, F., Nogueira, R., Lotufo, R.: BERTimbau: pretrained BERT models for Brazilian Portuguese. In: 9th Brazilian Conference on Intelligent Systems, BRACIS, Rio Grande do Sul, Brazil, October 20-23 (2020). (to appear)
Vaswani, A., et al.: Attention is all you need (2017)
Wagner, J., Wilkens, R., Idiart, M., Villavicencio, A.: The brWaC corpus: a new open resource for Brazilian Portuguese (2018)
Wolf, T., et al.: Huggingface’s transformers: state-of-the-art natural language processing (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
da Costa Cunha, L.F., Ramalho, J.C. (2022). NER in Archival Finding Aids: Next Level. In: Rocha, A., Adeli, H., Dzemyda, G., Moreira, F. (eds) Information Systems and Technologies. WorldCIST 2022. Lecture Notes in Networks and Systems, vol 469. Springer, Cham. https://doi.org/10.1007/978-3-031-04819-7_33
Download citation
DOI: https://doi.org/10.1007/978-3-031-04819-7_33
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-04818-0
Online ISBN: 978-3-031-04819-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)