Abstract
Nested named entity recognition attracts increasingly attentions due to their pervasiveness in general domain as well as in other specific domains. This paper proposes a multi-layer joint learning model for Chinese named entities recognition based on self-attention aggregation mechanism where a series of multi-layered sequence labeling sub-models are joined to recognize named entities in a bottom-up fashion. In order to capture entity semantic information in a lower layer, hidden units in an entity are aggregated using self-attention mechanism and further fed into the higher layer. We conduct extensive experiments using various entity aggregation methods. The results on the Chinese nested entity corpus transformed from the People’s Daily show that our model performs best among other competitive methods, implying that self-attention mechanism can effectively aggregate important semantic information in an entity.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Lample, G., Ballesteros, M., Subramanian, S., Kawakami, K., Dyer, C.: Neural architectures for named entity recognition. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 260–270 (2016)
Ma, X., Hovy, E.: End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Long Papers), vol. 1, pp. 1064–1074 (2016)
Gridach, M.: Character-level neural network for biomedical named entity recognition. J. Biomed. Inform. 70, 85–91 (2017)
Strubell, E., Verga, P., Belanger, D., McCallum, A.: Fast and accurate entity recognition with iterated dilated convolutions. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 2670–2680 (2017)
Muis, A.O., Lu, W.: Labeling gaps between words: recognizing overlapping mentions with mention separators. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 2608–2618 (2017)
Lu, W., Roth, D.: Joint mention extraction and classification with mention hypergraphs. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 857–867 (2015)
Xu, M.B., Jiang, H., Watcharawittayakul, S.: A local detection approach for named entity recognition and mention detection. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Long Papers), vol. 1, pp. 1237–1247 (2017)
Zhou, G.D., Zhang, J., Su, J., Shen, D., Tan, C.L., et al.: Recognizing names in biomedical texts: a machine learning approach. Bioinformatics 20, 1178–1190 (2004)
Zhou, G.D.: Recognizing names in biomedical texts using mutual information independence model and SVM plus sigmoid. Int. J. Med. Inform. 456–467 (2006)
Alex, B., Haddow, B., Grover, C.: Recognising nested named entities in biomedical text. In: Proceedings of the Workshop on BioNLP 2007, pp. 65–72 (2007)
Fu, C.Y.: Research on Chinese Nested Named Entity Recognition Method. Heilongjiang University, Harbin (2011)
Byrne, K.: Nested named entity recognition in historical archive text. In: Proceedings of International Conference on Semantic Computing, pp. 589–596 (2007)
Sohrab, M.G., Miwa, M.: Deep exhaustive model for nested named entity recognition. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 2843–2849 (2018)
Ju, M., Miwa, M., Ananiadou, S.: A neural layered model for nested named entity recognition. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Long Papers), vol. 1, pp. 1446–1459 (2018)
Finkel, J.R., Manning, C.D.: Nested named entity recognition. In: Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, vol, pp. 141–150 (2009)
Ohta, T., Tateisi, Y., Kim, J.D.: The GENIA corpus: an annotated research abstract corpus in molecular biology domain. In: Proceedings of the Second International Conference on Human Language Technology Research, pp. 82–86. Morgan Kaufmann Publishers Inc. (2002)
Walker, C., Strassel, S., Medero, J., Maeda, K.: ACE 2005 Multilingual Training Corpus. Linguistic Data Consortium, Philadelphia (2006)
Hu, J.H.: Statistics and analysis of data in people’s daily 1998. In: Proceedings of the First Symposium on Computational Linguistics for Students, pp. 323–329 (2002)
Li, Y.Q., He, Y.Q., Qian, L.H., Zhou, G.D.: Chinese nested named entity recognition corpus construction. J. Chin. Inform. Process. 19–26 (2018)
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Proceedings of the 26th International Conference on Neural Information Processing Systems, Advances in neural information processing systems, vol. 2, pp. 3111–3119 (2013)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 1735–1780 (1997)
Elman, J.L.: Distributed representations, simple recurrent networks, and grammatical structure. Mach. Learn. 7, 195–225 (1991)
Feng, H.: Research on visual attention mechanism and its application. North China Electric Power University (Beijing) (2011)
Vaswani, A., et al.: Attention is all you need. In: Proceedings of the 31st International Conference on Neural Information Processing, pp. 6000–6010 (2017)
Zhu, Y.Y., Wang, G.X.: CAN-NER: convolutional attention network for chinese named entity recognition. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Long and Short Papers), vol. 1, pp. 3384–3393 (2019)
Devlin, J., Chang, M.W., Lee, K., Toutanova, k.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Long and Short Papers), vol. 1, pp. 4171–4186 (2019)
Acknowledgments
Sincere appreciation to anonymous reviewers for their helpful and insightful comments that greatly improve the manuscript.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Ethics declarations
Publication of this article was sponsored by National Natural Science Foundation of China [61976147; 2017YFB1002101; 61373096].
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Li, H., Xu, H., Qian, L., Zhou, G. (2020). Multi-layer Joint Learning of Chinese Nested Named Entity Recognition Based on Self-attention Mechanism. In: Zhu, X., Zhang, M., Hong, Y., He, R. (eds) Natural Language Processing and Chinese Computing. NLPCC 2020. Lecture Notes in Computer Science(), vol 12431. Springer, Cham. https://doi.org/10.1007/978-3-030-60457-8_12
Download citation
DOI: https://doi.org/10.1007/978-3-030-60457-8_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-60456-1
Online ISBN: 978-3-030-60457-8
eBook Packages: Computer ScienceComputer Science (R0)
