Abstract
As one of the essential tasks in the context of natural language understanding, Few-shot Named Entity Recognition (NER) aims to identify and classify entities against limited samples. Recently, many works have attempted to enhance semantic representations by constructing prompt templates with text and label names. These methods, however, not only distract attention from the text, but also cause unnecessary enumerations. Furthermore, ambiguous label names always fail in delivering the intended meaning. To address the above issues, we present a Label-Description Enhanced Network (LaDEN) for few-shot named entity recognition, under which we propose a BERT-based Siamese network to incorporate fine-grained label descriptions as knowledge augmentation. The designed semantic attention mechanism captures label-specific textual representations, and the distance function matches similar token and label representations based on the nearest-neighbor criterion. Experimental results demonstrate that our model outperforms previous works in both few-shot and resource-rich settings, achieving state-of-the-art performance on five benchmarks. Our method is particularly efficient in low-resource scenarios, especially for cross-domain applications.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
We count at the entity level. For instance, both words in “Stephen Frater” are tagged as “I-PERSON”. We consider “I-PERSON” to occur once.
- 2.
The BERT-base-cased is available in Hugging-Face: https://huggingface.co/bert-base-cased.
- 3.
We use the pre-trained 200-dimensional word vectors from Stanford University: https://nlp.stanford.edu/projects/glove/.
References
Aly, R., Vlachos, A., McDonald, R., et al.: Leveraging type descriptions for zero-shot named entity recognition and classification. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 1516–1528 (2021)
Bromley, J., et al.: Signature verification using a “Siamese” time delay neural network. In: Advances in Neural Information Processing Systems, vol. 6 (1993)
Cui, L., et al.: Template-based named entity recognition using BART. arXiv preprint arXiv:2106.01760 (2021)
Derczynski, L., et al.: Results of the WNUT2017 shared task on novel and emerging entity recognition. In: Proceedings of the 3rd Workshop on Noisy User-generated Text, pp. 140–147 (2017)
Devlin, J., et al.: Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Ding et al. "Few-nerd: a few-shot named entity recognition dataset. arXiv preprint arXiv:2105.07464 (2021)
Ding, N., et al.: Prompt-learning for fine-grained entity typing. arXiv preprint arXiv:2108.10604 (2021)
Doğan, R.I., Leaman, R., Lu, Z.: NCBI disease corpus: a resource for disease name recognition and concept normalization. J. Biomed. Inf. 47, 1–10 (2014)
Fritzler, A., Logacheva, V., Kretov, M.: Few-shot classification in named entity recognition task. In: Proceedings of the 34th ACM/SIGAPP Symposium on Applied Computing, pp. 993–1000 (2019)
Halder, K., et al.: Task-aware representation of sentences for generic text classification. In: Proceedings of the 28th International Conference on Computational Linguistics, pp. 3202–3213 (2020)
Hou, Y., et al.: Few-shot slot tagging with collapsed dependency transfer and label-enhanced task-adaptive projection network. arXiv preprint arXiv:2006.05702 (2020)
Kim, J.-D., et al.: Introduction to the bio-entity recognition task at JNLPBA. In: Proceedings of the International Joint Workshop on Natural Language Processing in Biomedicine and its Applications. Citeseer. pp. 70–75 (2004)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Lafferty, J., McCallum, A., Pereira, F.C.N.: Conditional random fields: probabilistic models for segmenting and labeling sequence data (2001)
Lewis, M., et al. Bart: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461 (2019)
Li, X., et al.: A unified MRC framework for named entity recognition. arXiv preprint arXiv:1910.11476 (2019)
Liu, A.T., et al.: QaNER: prompting question answering models for few-shot named entity recognition. arXiv preprint arXiv:2203.01543 (2022)
Haitao, L., et al.: TFM: a triple fusion module for integrating lexicon information in Chinese named entity recognition. In: Neural Processing Letters, pp. 1–18 (2022)
Ma, J., et al.: Label semantics for few shot named entity recognition. arXiv preprint arXiv:2203.08985 (2022)
Pennington, J., Socher, R., Manning, C.D.: Glove: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543 (2014)
Sang, E.F., De Meulder, F.: Introduction to the CoNLL-2003 shared task: language-independent named entity recognition. arXiv preprint cs/0306050 (2003)
Snell, J., Swersky, K., Zemel, R.: Prototypical networks for few-shot learning. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Stubbs, A., Uzuner, Ö.: Annotating longitudinal clinical narratives for de-identification: The 2014 i2b2/UTHealth corpus. J. Biomed. Inf. 58, S20–S29 (2015)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Vinyals, O., et al.: Matching networks for one shot learning. In: Advances in Neural Information Processing Systems, vol. 29 (2016)
Weischedel, R., et al.: Ontonotes release 5.0 ldc2013t19. In: Linguistic Data Consortium, Philadelphia, PA, vol. 23 (2013)
Yang, Y., Katiyar, A.: Simple and effective few-shot named entity recognition with structured nearest neighbor learning. arXiv preprint arXiv:2010.02405 (2020)
Yin, W., et al.: Universal natural language processing with limited annotations: try few-shot textual entailment as a start. arXiv preprint arXiv:2010.02584 (2020)
Acknowledgements
Our work was supported by Sichuan Science and Technology Program (No.2023YFG0021, No.2022YFG0038 and No.2021YFG0018), and by Xinjiang Science and Technology Program (No. 2022D01B185).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, X., Gao, H. (2023). Label-Description Enhanced Network for Few-Shot Named Entity Recognition. In: Iliadis, L., Papaleonidas, A., Angelov, P., Jayne, C. (eds) Artificial Neural Networks and Machine Learning – ICANN 2023. ICANN 2023. Lecture Notes in Computer Science, vol 14261. Springer, Cham. https://doi.org/10.1007/978-3-031-44198-1_37
Download citation
DOI: https://doi.org/10.1007/978-3-031-44198-1_37
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-44197-4
Online ISBN: 978-3-031-44198-1
eBook Packages: Computer ScienceComputer Science (R0)