skip to main content
10.1145/3587716.3587729acmotherconferencesArticle/Chapter ViewAbstractPublication PagesicmlcConference Proceedingsconference-collections
research-article

Nested Named Entity Recognition based on Star-Transformer and BiLSTM

Published: 07 September 2023 Publication History

Abstract

Nested named entity recognition is a challenging task due to the complex boundary structure of coarse-grained entities containing other fine-grained entities. To obtain the boundary position information and the dependencies between words to capture more boundary information of nested named entities, this paper proposed a novel method that fused the Star-Transformer with BiLSTM which will improve the performance of the model in nested named entity recognition and entity boundary detection, the star topology structure of the Star-Transformer can also reduce the complexity of attention models such as Transformer. Meanwhile, we utilized Entity Boundary Classification Layer (EBCL) and Entity Type Classification Layer (ETCL) to perceive entity boundary structure for the classification of detected boundaries and entities within boundaries. We conducted experiments on two public datasets and our own built-in industrial domain dataset. The experimental results show that our model performs better than state-of-the-art boundary-aware models.

References

[1]
Beatrice Alex, Barry Haddow, and Claire Grover. 2007. Recognising Nested Named Entities in Biomedical Text. In Meeting of the Association for Computational Linguistics.
[2]
Z. Dai, Z. Yang, Y. Yang, J. Carbonell, and R. Salakhutdinov. 2019. Transformer-XL: Attentive Language Models beyond a Fixed-Length Context. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.
[3]
Yao Fu, Chuanqi Tan, Mosha Chen, Songfang Huang, and Fei Huang. 2021. Nested Named Entity Recognition with Partially-Observed TreeCRFs. ArXiv abs/2012.08478 (2021).
[4]
Q. Guo, X. Qiu, P. Liu, Y. Shao, X. Xue, and Z. Zhang. 2019. Star-Transformer. In Proceedings of the 2019 Conference of the North.
[5]
M. Ju, M. Miwa, and S. Ananiadou. 2018. A Neural Layered Model for Nested Named Entity Recognition. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers).
[6]
Bing Li, Shifeng Liu, Yifang Sun, Wei Wang, and Xiang Zhao. 2020. Recursively Binary Modification Model for Nested Named Entity Recognition. In Proceedings of the AAAI Conference on Artificial Intelligence.
[7]
Fei Li, Z. Wang, Siu Cheung Hui, Lejian Liao, Xin hua Zhu, and Heyan Huang. 2021. A segment enhanced span-based model for nested named entity recognition. Neurocomputing 465 (2021), 26–37.
[8]
Xinwei Long, Shuzi Niu, and Yucheng Li. 2020. Hierarchical Region Learning for Nested Named Entity Recognition. In Findings of the Association for Computational Linguistics: EMNLP 2020.
[9]
W. Lu and R. Dan. 2015. Joint Mention Extraction and Classification with Mention Hypergraphs. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.
[10]
Y. Luo and H. Zhao. 2020. Bipartite Flat-Graph Network for Nested Named Entity Recognition. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.
[11]
Aldrian Obaja Muis and Wei Lu. 2017. Labeling Gaps Between Words: Recognizing Overlapping Mentions with Mention Separators. In Conference on Empirical Methods in Natural Language Processing.
[12]
Dan Shen, Jie Zhang, Guodong Zhou, Jian Su, and Chew Lim Tan. 2003. Effective Adaptation of Hidden Markov Model-based Named Entity Recognizer for Biomedical Domain. In Meeting of the Association for Computational Linguistics.
[13]
Yongliang Shen, Xinyin Ma, Zeqi Tan, Shuai Zhang, Wen Wang, and Weiming Lu. 2021. Locate and Label: A Two-stage Identifier for Nested Named Entity Recognition. ArXiv abs/2105.06804 (2021).
[14]
Mohammad Golam Sohrab and Makoto Miwa. 2018. Deep Exhaustive Model for Nested Named Entity Recognition. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.
[15]
Hagen Soltau, Izhak Shafran, Mingqiu Wang, and Laurent El Shafey. 2022. RNN Transducers for Nested Named Entity Recognition with constraints on alignment for long sequences. ArXiv abs/2203.03543 (2022).
[16]
J Strakova, M. Straka, and J. Haji. 2019. Neural Architectures for Nested NER through Linearization. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.
[17]
Chuanqi Tan, Weijian Qiu, Mosha Chen, Rui Wang, and Fei Huang. 2020. Boundary Enhanced Neural Span Classification for Nested Named Entity Recognition. In Proceedings of the AAAI Conference on Artificial Intelligence.
[18]
Zeqi Tan, Yongliang Shen, Shuai Zhang, Weiming Lu, and Yueting Zhuang. 2021. A Sequence-to-Set Network for Nested Named Entity Recognition. In IJCAI.
[19]
Ashish Vaswani, Noam M. Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In NIPS.
[20]
Bailin Wang and Wei Lu. 2018. Neural Segmental Hypergraphs for Overlapping Mention Recognition. In Conference on Empirical Methods in Natural Language Processing.
[21]
Jue Wang, Lidan Shou, Ke Chen, and Gang Chen. 2020. Pyramid: A Layered Model for Nested Named Entity Recognition. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.
[22]
Yu Wang, Yun Li, Hanghang Tong, and Ziye Zhu. 2020. HIT: Nested Named Entity Recognition via Head-Tail Pair and Token Interaction. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing.
[23]
Lei Xu, Shuang Li, Yuchen Wang, and Lizhen Xu. 2021. Named Entity Recognition of BERT-BiLSTM-CRF Combined with Self-attention. In International Conference on Web Information Systems and Applications.
[24]
Yongxiu Xu, Heyan Huang, Chong Feng, and Yue Hu. 2021. A Supervised Multi-Head Self-Attention Network for Nested Named Entity Recognition. In Proceedings of the AAAI Conference on Artificial Intelligence.
[25]
Zenan Zhai, Dat Quoc Nguyen, and Karin M. Verspoor. 2018. Comparing CNN and LSTM character-level embeddings in BiLSTM-CRF models for chemical and disease named entity recognition. In Proceedings of the Ninth International Workshop on Health Text Mining and Information Analysis.
[26]
Changmeng Zheng, Y. Cai, Jingyun Xu, Ho fung Leung, and Guandong Xu. 2019. A Boundary-aware Neural Model for Nested Named Entity Recognition. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing.

Index Terms

  1. Nested Named Entity Recognition based on Star-Transformer and BiLSTM

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    ICMLC '23: Proceedings of the 2023 15th International Conference on Machine Learning and Computing
    February 2023
    619 pages
    ISBN:9781450398411
    DOI:10.1145/3587716
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 07 September 2023

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. BiLSTM
    2. Entity boundary detection
    3. Nested named entity recognition
    4. Star-Transformer

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Conference

    ICMLC 2023

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 49
      Total Downloads
    • Downloads (Last 12 months)21
    • Downloads (Last 6 weeks)1
    Reflects downloads up to 20 Jan 2025

    Other Metrics

    Citations

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media