research-article

Token Relation Aware Chinese Named Entity Recognition

Authors:

Xiaofeng Zhang,

Yuanxin Ouyang,

Zhang XiongAuthors Info & Claims

ACM Transactions on Asian and Low-Resource Language Information Processing, Volume 22, Issue 1

Article No.: 24, Pages 1 - 21

https://doi.org/10.1145/3531534

Published: 25 November 2022 Publication History

Abstract

Due to the lack of natural delimiters, most Chinese Named Entity Recognition (NER) approaches are character-based and utilize an external lexicon to leverage the word-level information. Although they have achieved promising results, the latent words they introduced are still non-contextualized. In this paper, we investigate three relations, i.e., adjacent relation between characters, character co-occurrence relation between latent words, and dependency relation among tokens, to address this issue. Specifically, we first establish the local context for latent words and then propose a masked self-attention mechanism to incorporate such local contextual information. Besides, since introducing external knowledge such as lexicon and dependency relation inevitably brings in some noises, we propose a gated information controller to handle this problem. Extensive experimental results show that the proposed approach surpasses most similar methods on public datasets and demonstrates its promising potential.

References

[1]

Adam L. Berger and John D. Lafferty. 2017. Information retrieval as statistical translation. SIGIR Forum 51, 2 (2017), 219–226.

Digital Library

[2]

Roi Blanco, Giuseppe Ottaviano, and Edgar Meij. 2015. Fast and space-efficient entity linking for queries. In Proceedings of the 8th ACM International Conference on Web Search and Data Mining. 179–188.

Digital Library

[3]

Xiaoling Cai, Shoubin Dong, and Jinlong Hu. 2019. A deep learning model incorporating part of speech and self-matching attention for named entity recognition of Chinese electronic medical records. BMC Medical Informatics & Decision Making 19-S, 2 (2019), 101–109.

[4]

Miao Chen, Ganhui Lan, Fang Du, and Victor S. Lobanov. 2020. Joint learning with pre-trained transformer on named entity recognition and relation extraction tasks for clinical analytics. In Proceedings of the 3rd Clinical Natural Language Processing Workshop. 234–242.

[5]

Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel P. Kuksa. 2011. Natural language processing (almost) from scratch. Journal of Machine Learning Research 12 (2011), 2493–2537.

Digital Library

[6]

Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Ziqing Yang, Shijin Wang, and Guoping Hu. 2019. Pre-training with whole word masking for Chinese BERT. CoRR abs/1906.08101 (2019).

[7]

Zihang Dai, Zhilin Yang, Yiming Yang, Jaime G. Carbonell, Quoc Viet Le, and Ruslan Salakhutdinov. 2019. Transformer-XL: Attentive language models beyond a fixed-length context. In Proceedings of the 57th Conference of the Association for Computational Linguistics. 2978–2988.

[8]

Thanh Hai Dang, Hoang-Quynh Le, Trang M. Nguyen, and Sinh T. Vu. 2018. D3NER: Biomedical named entity recognition using CRF-biLSTM improved with fine-tuned embeddings of various linguistic information. Bioinformatics. 34, 20 (2018), 3539–3546.

[9]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 4171–4186.

[10]

Shizhe Diao, Jiaxin Bai, Yan Song, Tong Zhang, and Yonggang Wang. 2020. ZEN: Pre-training Chinese text encoder enhanced by N-gram representations. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings. 4729–4740.

[11]

Ruixue Ding, Pengjun Xie, Xiaoyan Zhang, Wei Lu, Linlin Li, and Luo Si. 2019. A neural multi-digraph model for Chinese NER with gazetteers. In Proceedings of the 57th Conference of the Association for Computational Linguistics. 1462–1467.

[12]

Cícero Nogueira dos Santos and Victor Guimarães. 2015. Boosting named entity recognition with neural character embeddings. In Proceedings of the 5th Named Entity Workshop. 25–33.

[13]

Sean R. Eddy. 1996. Hidden Markov models. Current Opinion in Structural Biology 6, 3 (1996), 361–365.

[14]

Chen Gong, Saihao Huang, Houquan Zhou, Zhenghua Li, Min Zhang, Zhefeng Wang, Baoxing Huai, and Nicholas Jing Yuan. 2021. An in-depth study on internal structure of Chinese words. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. 5823–5833.

[15]

Tao Gui, Ruotian Ma, Qi Zhang, Lujun Zhao, Yu-Gang Jiang, and Xuanjing Huang. 2019. CNN-based Chinese NER with lexicon rethinking. In Proceedings of the 28th International Joint Conference on Artificial Intelligence. 4982–4988.

[16]

Tao Gui, Yicheng Zou, Qi Zhang, Minlong Peng, Jinlan Fu, Zhongyu Wei, and Xuanjing Huang. 2019. A lexicon-based graph neural network for Chinese NER. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 1040–1050.

[17]

Qipeng Guo, Xipeng Qiu, Pengfei Liu, Yunfan Shao, Xiangyang Xue, and Zheng Zhang. 2019. Star-transformer. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Human Language Technologies. 1315–1325.

[18]

Dou Hu and Lingwei Wei. 2020. SLK-NER: Exploiting second-order lexicon knowledge for Chinese NER. In Proceedings of the 32nd International Conference on Software Engineering and Knowledge Engineering. 413–417.

[19]

Zhiheng Huang, Wei Xu, and Kai Yu. 2015. Bidirectional LSTM-CRF models for sequence tagging. CoRR abs/1508.01991 (2015).

[20]

Zhanming Jie and Wei Lu. 2019. Dependency-guided LSTM-CRF for named entity recognition. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 3860–3870.

[21]

Yanliang Jin, Jinfei Xie, Weisi Guo, Can Luo, Dijia Wu, and Rui Wang. 2019. LSTM-CRF neural network with gated self attention for Chinese NER. IEEE Access 7 (2019), 136694–136703.

[22]

Saravanakumar Kandasamy and Aswani Kumar Cherukuri. 2020. Query expansion using named entity disambiguation for a question-answering system. Concurrency and Computation: Practice and Experience 32, 4 (2020).

[23]

John D. Lafferty, Andrew McCallum, and Fernando C. N. Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the 18th International Conference on Machine Learning. 282–289.

Digital Library

[24]

Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016. Neural architectures for named entity recognition. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 260–270.

[25]

Gina-Anne Levow. 2006. The third international Chinese language processing bakeoff: Word segmentation and named entity recognition. In Proceedings of the 5th SIGHAN Workshop on Chinese Language Processing. 108–117.

[26]

Jing Li, Aixin Sun, Jianglei Han, and Chenliang Li. 2018. A survey on deep learning for named entity recognition. CoRR abs/1812.09449 (2018).

[27]

Peng-Hsuan Li, Tsu-Jui Fu, and Wei-Yun Ma. 2020. Why attention? Analyze BiLSTM deficiency and its remedies in the case of NER. In Proceedings of the 34th AAAI Conference on Artificial Intelligence. 8236–8244.

[28]

Xiaonan Li, Hang Yan, Xipeng Qiu, and Xuanjing Huang. 2020. FLAT: Chinese NER using flat-lattice transformer. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 6836–6842.

[29]

Yuan Li, Guodong Du, Yan Xiang, Shaozi Li, Lei Ma, Dangguo Shao, Xiongbin Wang, and Haoyu Chen. 2020. Towards Chinese clinical named entity recognition by dynamic embedding using domain-specific knowledge. Journal of Biomedical Informatics 106 (2020), 103435.

[30]

Yankai Lin, Shiqi Shen, Zhiyuan Liu, Huanbo Luan, and Maosong Sun. 2016. Neural relation extraction with selective attention over instances. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2124–2133.

[31]

Wei Liu, Xiyan Fu, Yue Zhang, and Wenming Xiao. 2021. Lexicon enhanced Chinese sequence labeling using BERT adapter. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. 5847–5858.

[32]

Wei Liu, Tongge Xu, QingHua Xu, Jiayu Song, and Yueran Zu. 2019. An encoding strategy based word-character LSTM for Chinese NER. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2379–2389.

[33]

Zhangxun Liu, Conghui Zhu, and Tiejun Zhao. 2010. Chinese named entity recognition with a sequence labeling approach: Based on characters, or based on words? In Proceedings of 6th International Conference on Intelligent Computing. 634–640.

[34]

Ruotian Ma, Minlong Peng, Qi Zhang, Zhongyu Wei, and Xuanjing Huang. 2020. Simplify the usage of lexicon in Chinese NER. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 5951–5960.

[35]

Xindian Ma, Peng Zhang, Shuai Zhang, Nan Duan, Yuexian Hou, Ming Zhou, and Dawei Song. 2019. A tensorized transformer for language modeling. In Proceedings of 2019 Annual Conference on Neural Information Processing Systems. 2229–2239.

[36]

Pedro Henrique Martins, Zita Marinho, and André F. T. Martins. 2019. Joint learning of named entity recognition and entity linking. In Proceedings of the 57th Conference of the Association for Computational Linguistics. 190–196.

[37]

Sewon Min, Victor Zhong, Richard Socher, and Caiming Xiong. 2018. Efficient and robust question answering from minimal context over documents. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 1725–1735.

[38]

Yuyang Nie, Yuanhe Tian, Yan Song, Xiang Ao, and Xiang Wan. 2020. Improving named entity recognition with attentive ensemble of syntactic information. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings. 4231–4245.

[39]

Yuyang Nie, Yuanhe Tian, Xiang Wan, Yan Song, and Bo Dai. 2020. Named entity recognition for social media texts with semantic augmentation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. 1383–1391.

[40]

Nanyun Peng and Mark Dredze. 2015. Named entity recognition for Chinese social media with jointly trained embeddings. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 548–554.

[41]

Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Human Language Technologies. 2227–2237.

[42]

Ying Qin, Taozheng Zhang, and Xiaojie Wang. 2008. Chinese named entity recognition with new contextual features. In Proceedings of the 4th International Conference on Natural Language Processing and Knowledge Engineering. 1–6.

[43]

Sunita Sarawagi and William W. Cohen. 2004. Semi-Markov conditional random fields for information extraction. In Proceedings of 2004 Annual Conference on Neural Information Processing Systems. 1185–1192.

[44]

Cijian Song, Yan Xiong, Wenchao Huang, and Lu Ma. 2020. Joint self-attention and multi-embeddings for Chinese named entity recognition. In Proceedings of 6th International Conference on Big Data Computing and Communications. 76–80.

[45]

Emma Strubell, Patrick Verga, David Belanger, and Andrew McCallum. 2017. Fast and accurate entity recognition with iterated dilated convolutions. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 2670–2680.

[46]

Dianbo Sui, Yubo Chen, Kang Liu, Jun Zhao, and Shengping Liu. 2019. Leverage lexical knowledge for Chinese named entity recognition via collaborative graph network. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 3828–3838.

[47]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of 2017 Annual Conference on Neural Information Processing Systems. 5998–6008.

[48]

Caiyu Wang, Hong Wang, Hui Zhuang, Wei Li, Shu Han, Hui Zhang, and Luhe Zhuang. 2020. Chinese medical named entity recognition based on multi-granularity semantic dictionary and multimodal tree. Journal of Biomedical Informatics 111 (2020), 103583.

Digital Library

[49]

Qi Wang, Yangming Zhou, Tong Ruan, Daqi Gao, Yuhang Xia, and Ping He. 2019. Incorporating dictionaries into deep neural networks for the Chinese clinical named entity recognition. Journal of Biomedical Informatics 92 (2019).

Digital Library

[50]

Rui Wang, Xin Xin, Wei Chang, Kun Ming, Biao Li, and Xin Fan. 2019. Chinese NER with height-limited constituent parsing. In Proceedings of the 33rd AAAI Conference on Artificial Intelligence. 7160–7167.

Digital Library

[51]

Ralph Weischedel, Sameer Pradhan, Lance Ramshaw, Martha Palmer, Nianwen Xue, Mitchell Marcus, Ann Taylor, Craig Greenberg, Eduard Hovy, Robert Belvin, and Ann Houston. 2011. Ontonotes Release 4.0. https://catalog.ldc.upenn.edu/docs/LDC2011T03/OntoNotes-Release-4.0.pdf.

[52]

Shuang Wu, Xiaoning Song, and Zhen-Hua Feng. 2021. MECT: Multi-metadata embedding based cross-transformer for Chinese named entity recognition. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. 1529–1539.

[53]

Mengge Xue, Weiming Cai, Jinsong Su, Linfeng Song, Yubin Ge, Yubao Liu, and Bin Wang. 2019. Neural collective entity linking based on recurrent random walk network learning. In Proceedings of the 28th International Joint Conference on Artificial Intelligence. 5327–5333.

[54]

Mengge Xue, Bowen Yu, Tingwen Liu, Yue Zhang, Erli Meng, and Bin Wang. 2020. Porous lattice transformer encoder for Chinese NER. In Proceedings of the 28th International Conference on Computational Linguistics. 3831–3841.

[55]

Chengxi Yan, Qi Su, and Jun Wang. 2020. MoGCN: Mixture of gated convolutional neural network for named entity recognition of Chinese historical texts. IEEE Access 8 (2020), 181629–181639.

[56]

Hang Yan, Bocao Deng, Xiaonan Li, and Xipeng Qiu. 2019. TENER: Adapting transformer encoder for named entity recognition. CoRR abs/1911.04474 (2019).

[57]

Shuo Yan, Jianping Chai, and Liyun Wu. 2020. Bidirectional GRU with multi-head attention for Chinese NER. In Proceedings of 5th IEEE Information Technology and Mechatronics Engineering Conference. 1160–1164.

[58]

Naixin Zhang, Feng Li, Guangluan Xu, Wenkai Zhang, and Hongfeng Yu. 2019. Chinese NER using dynamic meta-embeddings. IEEE Access 7 (2019), 64450–64459.

[59]

Naixin Zhang, Guangluan Xu, Zequn Zhang, and Feng Li. 2019. MIFM: Multi-granularity information fusion model for Chinese named entity recognition. IEEE Access 7 (2019), 181648–181655.

[60]

Yue Zhang and Jie Yang. 2018. Chinese NER using lattice LSTM. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 1554–1564.

[61]

Dandan Zhao, Jingxiang Cao, Degen Huang, Jiana Meng, and Pan Zhang. 2021. Dual neural network fusion model for Chinese named entity recognition. International Journal of Computational Intelligence Systems 14, 1 (2021), 471–481.

[62]

Yuying Zhu and Guoxin Wang. 2019. CAN-NER: Convolutional attention network for Chinese named entity recognition. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 3384–3393.

Cited By

Hou SQian YChen JZhao JLv HZhang JLeng HMa M(2025)HiNER: Hierarchical feature fusion for Chinese named entity recognitionNeurocomputing10.1016/j.neucom.2024.128667611(128667)Online publication date: Jan-2025
https://doi.org/10.1016/j.neucom.2024.128667
Zhang ZLiu SJian ZYin H(2024)Research on Chinese Named Entity Recognition Based on Lexical Information and Spatial FeaturesApplied Sciences10.3390/app1406224214:6(2242)Online publication date: 7-Mar-2024
https://doi.org/10.3390/app14062242
Mai CChen YGong ZWang HQiu MYuan CHuang Y(2024)PromptCNER: A Segmentation-based Method for Few-shot Chinese NER with Prompt-tuningACM Transactions on Asian and Low-Resource Language Information Processing10.1145/3705314Online publication date: 20-Nov-2024
https://doi.org/10.1145/3705314

Index Terms

Token Relation Aware Chinese Named Entity Recognition
1. Information systems
  1. World Wide Web
    1. Web mining

Recommendations

Attention in Character-Based BiLSTM-CRF for Chinese Named Entity Recognition
ICMAI '19: Proceedings of the 2019 4th International Conference on Mathematics and Artificial Intelligence

Named Entity Recognition (NER) is a crucial step in natural language processing (NLP). Recently, some researches work on enhancing the word representations by character-level extensions in English and have achieve excellent performance. The same method ...
A Lexicon Enhanced Chinese Long Named Entity Recognition Using Word-Aware Attention
MLNLP '23: Proceedings of the 2023 6th International Conference on Machine Learning and Natural Language Processing

In recent years, due to the rapid growth of space science and utilization research of China Manned Space Engineering, a considerable amount of technical documents and web data have been produced. Named entity recognition (NER) plays a vital role in ...
Attention-Based Bidirectional Long Short-Term Memory Networks for Chinese Named Entity Recognition
ICMLT '19: Proceedings of the 2019 4th International Conference on Machine Learning Technologies

Named entity recognition is an important task in natural language processing and has been carefully studied in recent decades. In this paper, we investigate the problem of Chinese named entity recognition. Using attention mechanisms based on BiLSTM-CRF ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Asian and Low-Resource Language Information Processing

ACM Transactions on Asian and Low-Resource Language Information Processing Volume 22, Issue 1

January 2023

340 pages

ISSN:2375-4699

EISSN:2375-4702

DOI:10.1145/3572718

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 November 2022

Online AM: 29 April 2022

Accepted: 03 April 2022

Revised: 25 November 2021

Received: 22 February 2021

Published in TALLIP Volume 22, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Refereed

Funding Sources

State Key Laboratory of Software Development Environment of China
National Natural Science Foundation of China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
745
Total Downloads

Downloads (Last 12 months)155
Downloads (Last 6 weeks)11

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Hou SQian YChen JZhao JLv HZhang JLeng HMa M(2025)HiNER: Hierarchical feature fusion for Chinese named entity recognitionNeurocomputing10.1016/j.neucom.2024.128667611(128667)Online publication date: Jan-2025
https://doi.org/10.1016/j.neucom.2024.128667
Zhang ZLiu SJian ZYin H(2024)Research on Chinese Named Entity Recognition Based on Lexical Information and Spatial FeaturesApplied Sciences10.3390/app1406224214:6(2242)Online publication date: 7-Mar-2024
https://doi.org/10.3390/app14062242
Mai CChen YGong ZWang HQiu MYuan CHuang Y(2024)PromptCNER: A Segmentation-based Method for Few-shot Chinese NER with Prompt-tuningACM Transactions on Asian and Low-Resource Language Information Processing10.1145/3705314Online publication date: 20-Nov-2024
https://doi.org/10.1145/3705314

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View full text|Download PDF

View Issue’s Table of Contents