Abstract
Active Self-Learning algorithms reduce the labeled data required to train a Machine Learning model through supervised training. This paper explores various Active Self-Learning algorithms for named entity recognition tasks. Firstly, we investigate the impact of different self-training techniques on Active Self-Learning algorithms. Secondly, we propose a novel token-level Active Self-Learning algorithm that achieves near-peak performance using fewer hand-annotated tokens compared to existing works. Through numerous experiments, we found that the sentence-level Active Self-Learning algorithm did not consistently yield significant results compared to pure active learning. However, our proposed token-level Active Self-Learning algorithm showed promising performance, training a neural model to nearly peak accuracy with fewer human-annotated tokens compared to state-of-the-art active learning baseline algorithms. The experimental results are presented and discussed, demonstrating the superior performance of the token-level Active Self-Learning algorithm
J. R. C. S. A. V. S. Neto—Research performed during the author’s masters undertaking at the University of Brasilia (UnB).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Neto, J.R.C.S.A.V.S.: Deep active learning approaches to the task of named entity recognition. Masters Dissertation [University of Brasilia] (2021)
Neto, J.R.C.S.A.V.S., Faleiros, T.P.: Deep active-self learning applied to named entity recognition. In: Britto, A., Valdivia Delgado, K. (eds.) BRACIS 2021. LNCS (LNAI), vol. 13074, pp. 405–418. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-91699-2_28
Clark, K., Luong, M.T., Manning, C.D., Le, Q.: Semi-supervised sequence modeling with cross-view training. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 1914–1925. Association for Computational Linguistics, Brussels, Belgium, October–November 2018
Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples (2015)
Hartmann, N.S., Fonseca, E.R., Shulby, C.D., Treviso, M.V., Rodrigues, J.S., Aluísio, S.M.: Portuguese word embeddings: evaluating on word analogies and natural language tasks. In: Anais do XI Simpósio Brasileiro de Tecnologia da Informação e da Linguagem Humana, pp. 122–131. SBC, Porto Alegre, RS, Brasil (2017)
Houlsby, N., Huszár, F., Ghahramani, Z., Lengyel, M.: Bayesian active learning for classification and preference learning (2011)
Kobayashi, K., Wakabayashi, K.: Named entity recognition using point prediction and active learning. In: Proceedings of the 21st International Conference on Information Integration and Web-Based Applications and Services, iiWAS2019, pp. 287–293. Association for Computing Machinery, New York, NY, USA (2019)
Lakshmi Narayan, P., Nagesh, A., Surdeanu, M.: Exploration of noise strategies in semi-supervised named entity classification. In: Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (*SEM 2019), pp. 186–191. Association for Computational Linguistics, Minneapolis, Minnesota, June 2019
Ma, X., Hovy, E.: End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1064–1074. Association for Computational Linguistics, Berlin, Germany, August 2016
Miyato, T., Dai, A.M., Goodfellow, I.: Adversarial training methods for semi-supervised text classification. In: International Conference on Learning Representations (ICLR) (2017)
Nair, V., Hinton, G.E.: Rectified linear units improve restricted Boltzmann machines. In: Proceedings of the 27th International Conference on International Conference on Machine Learning, ICML 2010, pp. 807–814. Omnipress, Madison, WI, USA (2010)
Neubig, G., Nakata, Y., Mori, S.: Pointwise prediction for robust, adaptable Japanese morphological analysis. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pp. 529–533. Association for Computational Linguistics, Portland, Oregon, USA, June 2011
Park, J., Kim, G., Kang, J.: Consistency training with virtual adversarial discrete perturbation (2021)
Pennington, J., Socher, R., Manning, C.: GloVe: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543. Association for Computational Linguistics, Doha, Qatar, October 2014. https://doi.org/10.3115/v1/D14-1162
Pradhan, S., et al.: Towards robust linguistic analysis using OntoNotes. In: Proceedings of the Seventeenth Conference on Computational Natural Language Learning, pp. 143–152. Association for Computational Linguistics, Sofia, Bulgaria, August 2013
Radmard, P., Fathullah, Y., Lipani, A.: Subsequence based deep active learning for named entity recognition. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 4310–4321. Association for Computational Linguistics, Online, August 2021
Sang, E.F.T.K., Meulder, F.D.: Introduction to the CoNLL-2003 shared task: language-independent named entity recognition. In: Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003, pp. 142–147 (2003)
Shen, Y., Yun, H., Lipton, Z., Kronrod, Y., Anandkumar, A.: Deep active learning for named entity recognition. In: Proceedings of the 2nd Workshop on Representation Learning for NLP, pp. 252–256. Association for Computational Linguistics, Vancouver, Canada, August 2017. https://doi.org/10.18653/v1/W17-2630
Siddhant, A., Lipton, Z.C.: Deep Bayesian active learning for natural language processing: results of a large-scale empirical study. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 2904–2909. Association for Computational Linguistics, Brussels, Belgium, October–November 2018. https://doi.org/10.18653/v1/D18-1318
Srivastava, N., Hinton, G.E., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)
Tran, V.C., Nguyen, N.T., Fujita, H., Hoang, D.T., Hwang, D.: A combination of active learning and self-learning for named entity recognition on Twitter using conditional random fields. Knowl.-Based Syst. 132, 179–187 (2017)
Acknowledgements
The authors were supported by the Fundação de Apoio a Pesquisa do Distritio Federal (FAP-DF) as members of the Knowledge Extraction from Documents of Legal content (KnEDLe) project from the University of Brasilia.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Cunha Santos A. V. Silva Neto, J.R., de Paulo Faleiros, T. (2023). Investigation of Deep Active Self-learning Algorithms Applied to Named Entity Recognition. In: Naldi, M.C., Bianchi, R.A.C. (eds) Intelligent Systems. BRACIS 2023. Lecture Notes in Computer Science(), vol 14197. Springer, Cham. https://doi.org/10.1007/978-3-031-45392-2_31
Download citation
DOI: https://doi.org/10.1007/978-3-031-45392-2_31
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-45391-5
Online ISBN: 978-3-031-45392-2
eBook Packages: Computer ScienceComputer Science (R0)