Abstract
Recent advancements in Machine Reading Comprehension (MRC) models have sparked interest in the field of Named Entity Recognition (NER), where entities are extracted as answers of given queries. Yet, existing MRC-based models face several challenges, including high computational costs, limited consideration of entity content information, and the tendency to generate sharp boundaries, that hinder their generalizability. To alleviate these issues, this paper introduces CREAM, an enhanced model leveraging Concise query and REgion-Aware Minimization. First, we propose a simple yet effective strategy of generating concise queries based primarily on entity categories. Second, we propose to go beyond existing methods by identifying entire entities, instead of just their boundaries (start and end positions), with an efficient continuous cross-entropy loss. An in-depth analysis is further provided to reveal their benefit. The proposed method is evaluated on six well-known NER benchmarks. Experimental results demonstrate its remarkable effectiveness by surpassing the current state-of-the-art models, with the substantial averaged improvement of 2.74, 1.12, and 2.38 absolute percentage points in Precision, Recall, and F1 metrics, respectively.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Collier, N., Ohta, T., Tsuruoka, Y., Tateisi, Y., Kim, J.D.: Introduction to the bio-entity recognition task at JNLPBA. In: Proceedings of the International Joint Workshop on Natural Language Processing in Biomedicine and its Applications (NLPBA/BioNLP), pp. 73–78. COLING, Geneva, Switzerland (2004)
Doddington, G., Mitchell, A., Przybocki, M., Ramshaw, L., Strassel, S., Weischedel, R.: The automatic content extraction (ACE) program - tasks, data, and evaluation. In: Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC’04). European Language Resources Association (ELRA), Lisbon, Portugal (2004)
Fu, J., Huang, X., Liu, P.: SpanNER: named entity re-/recognition as span prediction. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics, pp. 7183–7195. Association for Computational Linguistics (2021)
Huang, P., Zhao, X., Hu, M., Fang, Y., Li, X., Xiao, W.: Extract-select: a span selection framework for nested named entity recognition with generative adversarial training. In: Findings of the Association for Computational Linguistics: ACL 2022, pp. 85–96. Association for Computational Linguistics, Dublin, Ireland (2022)
Japkowicz, N., Stephen, S.: The class imbalance problem: a systematic study. Intell. Data Anal. 6(5), 429–449 (2002)
Katiyar, A., Cardie, C.: Nested named entity recognition revisited. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pp. 861–871. Association for Computational Linguistics, New Orleans, Louisiana (2018)
Li, F., Lin, Z., Zhang, M., Ji, D.: A span-based model for joint overlapped and discontinuous named entity recognition. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 4814–4828. Association for Computational Linguistics (2021)
Li, F., et al.: Modularized interaction network for named entity recognition. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 200–209. Association for Computational Linguistics (2021)
Li, X., Feng, J., Meng, Y., Han, Q., Wu, F., Li, J.: A unified MRC framework for named entity recognition. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 5849–5859. Association for Computational Linguistics (2020)
Liu, J., Mei, S., Hu, X., Yao, X., Yang, J., Guo, Y.: Seeing the wood for the trees: a contrastive regularization method for the low-resource knowledge base question answering. In: Findings of the Association for Computational Linguistics: NAACL 2022, pp. 1085–1094. Association for Computational Linguistics, Seattle, United States (2022)
Liu, Y., et al.: RoBERTa: a robustly optimized BERT pretraining approach, vol. abs/1907.11692 (2019)
Long, X., Niu, S., Li, Y.: Hierarchical region learning for nested named entity recognition. In: Findings of the Association for Computational Linguistics: EMNLP 2020, pp. 4788–4793. Association for Computational Linguistics (2020)
Lou, C., Yang, S., Tu, K.: Nested named entity recognition as latent lexicalized constituency parsing. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 6183–6198. Association for Computational Linguistics, Dublin, Ireland (2022)
Ma, X., Hovy, E.: End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1064–1074. Association for Computational Linguistics, Berlin, Germany (2016)
Medero, J., Maeda, K., Strassel, S., Walker, C.: An efficient approach to gold-standard annotation: decision points for complex tasks. In: Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC’06). European Language Resources Association (ELRA), Genoa, Italy (2006)
Muis, A.O., Lu, W.: Labeling gaps between words: recognizing overlapping mentions with mention separators. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 2608–2618. Association for Computational Linguistics, Copenhagen, Denmark (2017)
Ohta, T., Tateisi, Y., Kim, J.D.: The GENIA corpus: an annotated research abstract corpus in molecular biology domain. In: International Conference on Human Language Technology Research (2002)
Shen, Y., Ma, X., Tan, Z., Zhang, S., Wang, W., Lu, W.: Locate and label: A two-stage identifier for nested named entity recognition. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 2782–2794. Association for Computational Linguistics (2021)
Shrimal, A., Jain, A., Mehta, K., Yenigalla, P.: NER-MQMRC: formulating named entity recognition as multi question machine reading comprehension. In: Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Industry Track, pp. 230–238. Association for Computational Linguistics, Hybrid: Seattle, Washington + Online (2022)
Tan, Z., Shen, Y., Zhang, S., Lu, W., Zhuang, Y.: A sequence-to-set network for nested named entity recognition. In: Proceedings of the 30th International Joint Conference on Artificial Intelligence, IJCAI-21 (2021)
Tjong Kim Sang, E.F., De Meulder, F.: Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition. In: Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003, pp. 142–147 (2003)
Verlinden, S., Zaporojets, K., Deleu, J., Demeester, T., Develder, C.: Injecting knowledge base information into end-to-end joint entity and relation extraction and coreference resolution. In: Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp. 1952–1957. Association for Computational Linguistics (2021)
Wan, J., Ru, D., Zhang, W., Yu, Y.: Nested named entity recognition with span-level graphs. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 892–903. Association for Computational Linguistics, Dublin, Ireland (2022)
Wang, X., et al.: MINER: improving out-of-vocabulary named entity recognition from an information theoretic perspective. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 5590–5600. Association for Computational Linguistics, Dublin, Ireland (2022)
Yan, H., Gui, T., Dai, J., Guo, Q., Zhang, Z., Qiu, X.: A unified generative framework for various NER subtasks. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 5808–5822. Association for Computational Linguistics (2021)
Yun, C., Bhojanapalli, S., Rawat, A.S., Reddi, S.J., Kumar, S.: Are transformers universal approximators of sequence-to-sequence functions? CoRR abs/1912.10077 (2019)
Zhang, Q., Fu, J., Liu, X., Huang, X.: Adaptive co-attention network for named entity recognition in tweets. Proc, AAAI Conf. Artif. Intell. 32(1) (2018)
Zhu, E., Li, J.: Boundary smoothing for named entity recognition. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 7096–7108. Association for Computational Linguistics, Dublin, Ireland (2022)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Yao, X., Yang, Q., Hu, X., Yang, J., Guo, Y. (2023). CREAM: Named Entity Recognition with Concise query and REgion-Aware Minimization. In: Zhang, F., Wang, H., Barhamgi, M., Chen, L., Zhou, R. (eds) Web Information Systems Engineering – WISE 2023. WISE 2023. Lecture Notes in Computer Science, vol 14306. Springer, Singapore. https://doi.org/10.1007/978-981-99-7254-8_59
Download citation
DOI: https://doi.org/10.1007/978-981-99-7254-8_59
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-7253-1
Online ISBN: 978-981-99-7254-8
eBook Packages: Computer ScienceComputer Science (R0)