Menglin Zhou
ABSTRACT
To solve the problem that existing named entity recognition models lacked of ability to deal with unseen classes, the zero-shot learning was proposed to be used in the task of Chinese medical named entity recognition. Zero-shot learning utilizes the description information of the entity's class to establish the connection between the entity and the class, and transfers information from the observed classes to the unseen target classes. The model proposed by this paper was mainly based on BERT, which was used to model the relationship between the entity and the description. Moreover, the static word embedding, which is as a supplementary information, is concatenated with the features obtained from the BERT to solve the problem that BERT is not suitable for a specific field. At the same time, Correlation Searchers are added between the transformers of BERT to search for the word information most relevant to the character, so as to solve the problem that the model cannot obtain complete word information with characters as the input unit. Experiments show that the model's recognition performance has been significantly improved after adding the static word embedding and word information.
- GRISHMAN R, SUNDHEIM B. Message Understanding Conference- 6: A Brief History[C/OL]//COLING 1996 Volume 1: The 16th International Conference on Computational Linguistics. 1996[2022-10-24]. https://aclanthology.org/C96-1079.Google Scholar
- JI B, LIU R, LI S, A BILSTM-CRF method to Chinese electronic medical record named entity recognition[C]//Proceedings of the 2018 International Conference on Algorithms, Computing and Artificial Intelligence. 2018: 1-6.Google Scholar
- LAMPLE G, BALLESTEROS M, SUBRAMANIAN S, Neural Architectures for Named Entity Recognition: arXiv:1603.01360[R/OL]. arXiv, 2016[2022-05-27]. http://arxiv.org/abs/1603.01360. DOI:10.48550/arXiv.1603.01360.Google ScholarCross Ref
- VASWANI A, SHAZEER N, PARMAR N, Attention is all you need[J]. Advances in neural information processing systems, 2017, 30.Google Scholar
- LAMPERT C H, NICKISCH H, HARMELING S. Learning to detect unseen object classes by between-class attribute transfer[C]//2009 IEEE conference on computer vision and pattern recognition. IEEE, 2009: 951-958.Google Scholar
- FU Y, HOSPEDALES T M, XIANG T, Transductive multi-view zero-shot learning[J]. IEEE transactions on pattern analysis and machine intelligence, 2015, 37(11): 2332-2345.Google ScholarDigital Library
- XIAN Y, LAMPERT C H, SCHIELE B, Zero-shot learning—a comprehensive evaluation of the good, the bad and the ugly[J]. IEEE transactions on pattern analysis and machine intelligence, 2018, 41(9): 2251-2265.Google Scholar
- LOGESWARAN L, CHANG M W, LEE K, Zero-shot entity linking by reading entity descriptions[J]. arXiv preprint arXiv:1906.07348, 2019.Google Scholar
- WU L, PETRONI F, JOSIFOSKI M, Scalable zero-shot entity linking with dense entity retrieval[J]. arXiv preprint arXiv:1911.03814, 2019.Google Scholar
- DEVLIN J, CHANG M W, LEE K, Bert: Pre-training of deep bidirectional transformers for language understanding[J]. arXiv preprint arXiv:1810.04805, 2018.Google Scholar
- MIKOLOV T, CHEN K, CORRADO G, Efficient estimation of word representations in vector space[J]. arXiv preprint arXiv:1301.3781, 2013.Google Scholar
- WANG R, TANG D, DUAN N, K-adapter: Infusing knowledge into pre-trained models with adapters[J]. arXiv preprint arXiv:2002.01808, 2020.Google Scholar
- HOULSBY N, GIURGIU A, JASTRZEBSKI S, Parameter-efficient transfer learning for NLP[C]//International Conference on Machine Learning. PMLR, 2019: 2790-2799.Google Scholar
- LIU W, FU X, ZHANG Y, Lexicon enhanced chinese sequence labeling using bert adapter[J]. arXiv preprint arXiv:2105.07148, 2021.Google Scholar
- JAWAHAR G, SAGOT B, SEDDAH D. What does BERT learn about the structure of language?[C]//ACL 2019-57th Annual Meeting of the Association for Computational Linguistics. 2019.Google Scholar
- VAN DER MAATEN L, HINTON G. Visualizing data using t-SNE.[J]. Journal of machine learning research, 2008, 9(11).Google Scholar
- SONG Y, SHI S, LI J, Directional skip-gram: Explicitly distinguishing left and right context for word embeddings[C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers). 2018: 175-180.Google Scholar
- PETERS M E, NEUMANN M, IYYER M, Deep contextualized word representations: arXiv:1802.05365[R/OL]. arXiv, 2018[2022-05-27]. http://arxiv.org/abs/1802.05365. DOI:10.48550/arXiv.1802.05365.Google ScholarCross Ref
Index Terms
- Chinese medical named entity recognition based on zero-shot learning
Recommendations
Hippocampus-heuristic character recognition network for zero-shot learning in Chinese character recognition
Highlights- A novel hippocampus-heuristic character recognition network (HCRN) is proposed for zero/few-shot learning.
AbstractThe recognition of Chinese characters has always been a challenging task due to their huge variety and complex structures. The current radical-based methods fail to recognize Chinese characters without learning all of their radicals in ...
Towards zero-shot cross-lingual named entity disambiguation
Highlights- Novel zero-shot cross-lingual Named Entity Disambiguation approach.
- Robust ...
AbstractIn cross-Lingual Named Entity Disambiguation (XNED) the task is to link Named Entity mentions in text in some native language to English entities in a knowledge graph. XNED systems usually require training data for each native language,...
Overview of NLPCC 2023 Shared Task 6: Chinese Few-Shot and Zero-Shot Entity Linking
Natural Language Processing and Chinese ComputingAbstractEntity Linking (EL) is the task of grounding a textual mention in context to a corresponding entity in a knowledge base. However, current EL systems demonstrate a popularity bias, significantly underperforming on tail and emerging entities. To ...
Comments