skip to main content
10.1145/3436369.3436390acmotherconferencesArticle/Chapter ViewAbstractPublication PagesiccprConference Proceedingsconference-collections
research-article

A BERT-Based Named Entity Recognition in Chinese Electronic Medical Record

Published: 11 January 2021 Publication History

Abstract

Named entity recognition, aiming at identifying and classifying named entity mentioned in the structured or unstructured text, is a fundamental subtask for information extraction in natural language processing (NLP). With the development of electronic medical records, obtaining the key and effective information in electronic document through named entity identification has become an increasingly popular research direction. In this article, we adapt a recently introduced pre-trained language model BERT for named entity recognition in electronic medical records to solve the problem of missing context information and we add an extra mechanism to capture the relationship between words. Based on this, (1) the entities can be represented by sentence-level vector, with the forward as well as backward information of the sentence, which can be directly used by downstream tasks; (2) the model acquires the representation of word in context and learn the potential relation between words to decrease the influence of inconsistent entity markup problem of a text. We conduct experiments an electronic medical record dataset proposed by China Conference on Knowledge Graph and Semantic Computing in 2019. The experimental result shows that our proposed method has an improvement compared with the traditional methods.

References

[1]
Zhang, J., Shang, H., Gao, X. & Ernst, E. (2010). Acupuncture-related adverse events: a systematic review of the Chinese literature. Bulletin of the World Health Organisation. 88(12):915--921. DOI= 10.1590 / S0042-96862010001200012.
[2]
Mikolov, T., Chen, K., Corrado, G., et al. (2013). Efficient Estimation of Word Representations in Vector Space. Proceedings of Workshop at ICLR. 2013. DOI=https://arxiv.org/abs/1301.3781.
[3]
Peters, M., Neumann, M., Iyyer, M., et al. (2018). Deep contextualized word representations. DOI=https://arxiv.org/abs/1802.05365v2.
[4]
Devlin, J., Chang, M. W., Lee, K. et al. (2018). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. DOI= https://arxiv.org/abs/1810.04805.
[5]
Li, L., Nie, Y., Han, W., Huang, J. (2017) A Multi-attention-Based Bidirectional Long Short-Term Memory Network for Relation Extraction. In: Liu D., Xie S., Li Y., Zhao D., El-Alfy ES. (eds) Neural Information Processing. ICONIP 2017. Lecture Notes in Computer Science, vol 10638. Springer, Cham. DOI= https://doi.org/10.1007/978-3-319-70139-4_22.
[6]
Grishman, R., Sundheim, B. (1996). Message Understanding Conference 6: A Brief History. Proc COLING. 96. 466--471. DOI= http://doi.org/10.3115/992628.992709.
[7]
Ding, W., and Marchionini, G. (1997). A Study on Video Browsing Strategies. Technical Report. University of Maryland at College Park. DOI=https://dl.acm.org/doi/book/10.5555/270653.
[8]
Hanisch, D., Fundel, K., Mevissen, H. et al. ProMiner: rule-based protein and gene entity recognition. BMC Bioinformatics 6, S14 (2005). DOI=https://doi.org/10.1186/1471-2105-6-S1-S14.
[9]
Quimbaya, P., Múnera, A. S., Rivera, R. A. G., Rodriguez, J. C. D., Velandia, O. M. M., Peña, A. A. G., and Labbé, C. (2016). Named Entity Recognition Over Electronic Health Records Through a Combined Dictionary-based Approach. Procedia Computer Science. 100. 55--61. DOI=https://doi.org/10.1016/j.procs.2016.09.123.
[10]
Zhang, S., Elhadad, N. Unsupervised biomedical named entity recognition: experiments with clinical and biological texts. J Biomed Inform. 2013;46(6): 1088--1098. DOI=https://doi.org/10.1016/j.jbi.2013.08.004.
[11]
Settles, B. (2004). Biomedical Named Entity Recognition Using Conditional Random Fields and Rich Feature Sets. Proceedings of the Joint Workshop on Natural Language Processing in Biomedicine and its Applications. DOI=https://dl.acm.org/doi/10.5555/1567594.1567618.
[12]
Dong, C., Zhang, J., Zong, C., Hattori M., Di H. (2016) Character-Based LSTM-CRF with Radical-Level Features for Chinese Named Entity Recognition. In: Lin CY., Xue N., Zhao D., Huang X., Feng Y. (eds) Natural Language Understanding and Intelligent Applications. ICCPOL 2016, NLPCC 2016. Lecture Notes in Computer Science, vol 10102. Springer, Cham. DOI= https://doi.org/10.1007/978-3-319-50496-4_20.
[13]
Xiong, Y., Wang, Z., Jiang, D. et al. A fine-grained Chinese word segmentation and part-of-speech tagging corpus for clinical text. BMC Med Inform Decis Mak 19, 66 (2019). DOI=https://doi.org/10.1186/s12911-019-0770-7.
[14]
Vaswani, A., Shazeer, N., Parmar, N., et al. (2017). Attention Is All You Need. DOI=https://arxiv.org/abs/1706.03762v5.
[15]
Cai, Q. "Research on Chinese Naming Recognition Model Based on BERT Embedding," 2019 IEEE 10th International Conference on Software Engineering and Service Science (ICSESS), Beijing, China, 2019, pp. 1--4. DOI=https://doi.org/10.1109/ICSESS47205.2019.9040736.
[16]
Yu, X., Hu, W., Lu, S., Sun, X., and Yuan, Z. "BioBERT Based Named Entity Recognition in Electronic Medical Record," 2019 10th International Conference on Information Technology in Medicine and Education (ITME), Qingdao, sChina, 2019, pp. 49--52. DOI=https://doi.org/10.1109/ICCTEC.2017.00174.
[17]
Hochreiter, S. and Schmidhuber, J. (1997). Long Short-Term Memory. Neural Computation, 9, 1735--1780. DOI=https://doi.org/10.1162/neco.1997.9.8.1735.
[18]
Lafferty, J., McCallum, A. & Pereira, F. (2001). Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data. Proc. 18th International Conf. on Machine Learning, pp. 282--289. DOI=https://dl.acm.org/doi/10.5555/645530.655813.
[19]
Li, L., Ding, Z., Huang, D., and Zhou, H., "A Hybrid Model Based on CRFs for Chinese Named Entity Recognition," 2008 International Conference on Advanced Language Processing and Web Information Technology, Dalian Liaoning, 2008, pp. 127--132. DOI=https://doi.org/10.1007/978-3-642-14932-0_78.
[20]
Zhang, Y., Yang, J. (2018). Chinese NER Using Lattice LSTM. DOI= https://arxiv.org/abs/1805.02023.

Cited By

View all
  • (2024)Joint Extraction of Entity Relationships in Walnut Disease and Pest Based on Chinese NLP Models2024 IEEE 3rd International Conference on Electrical Engineering, Big Data and Algorithms (EEBDA)10.1109/EEBDA60612.2024.10485759(1027-1035)Online publication date: 27-Feb-2024
  • (2024)Transformer-based dual path cross fusion for pansharpening remote sensing imagesInternational Journal of Remote Sensing10.1080/01431161.2024.230615345:4(1170-1200)Online publication date: 2-Feb-2024
  • (2023)Fusion of SoftLexicon and RoBERTa for Purpose-Driven Electronic Medical Record Named Entity RecognitionApplied Sciences10.3390/app13241329613:24(13296)Online publication date: 15-Dec-2023
  • Show More Cited By

Index Terms

  1. A BERT-Based Named Entity Recognition in Chinese Electronic Medical Record

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    ICCPR '20: Proceedings of the 2020 9th International Conference on Computing and Pattern Recognition
    October 2020
    552 pages
    ISBN:9781450387835
    DOI:10.1145/3436369
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    In-Cooperation

    • Beijing University of Technology

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 11 January 2021

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. BERT
    2. Named entity recognition
    3. attention mechanism
    4. electronic medical records

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Conference

    ICCPR 2020

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)33
    • Downloads (Last 6 weeks)4
    Reflects downloads up to 20 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Joint Extraction of Entity Relationships in Walnut Disease and Pest Based on Chinese NLP Models2024 IEEE 3rd International Conference on Electrical Engineering, Big Data and Algorithms (EEBDA)10.1109/EEBDA60612.2024.10485759(1027-1035)Online publication date: 27-Feb-2024
    • (2024)Transformer-based dual path cross fusion for pansharpening remote sensing imagesInternational Journal of Remote Sensing10.1080/01431161.2024.230615345:4(1170-1200)Online publication date: 2-Feb-2024
    • (2023)Fusion of SoftLexicon and RoBERTa for Purpose-Driven Electronic Medical Record Named Entity RecognitionApplied Sciences10.3390/app13241329613:24(13296)Online publication date: 15-Dec-2023
    • (2023)UAV-FDN: Forest-fire detection network for unmanned aerial vehicle perspectiveJournal of Intelligent & Fuzzy Systems10.3233/JIFS-23155045:4(5821-5836)Online publication date: 4-Oct-2023
    • (2023)ECA-YOLOv5:Multi scale infrared salient target detection algorithm based on anchor free network9th Symposium on Novel Photoelectronic Detection Technology and Applications (NDTA2022)10.1117/12.2663112(24)Online publication date: 20-Mar-2023
    • (2023)Named Entity Recognition in Electronic Medical Records Based on RoBERTa Embedding and BiLSTM-CRF2023 26th ACIS International Winter Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD-Winter)10.1109/SNPD-Winter57765.2023.10223876(9-14)Online publication date: 5-Jul-2023
    • (2023)Casting defect detection based on improved DETR2023 IEEE 7th Information Technology and Mechatronics Engineering Conference (ITOEC)10.1109/ITOEC57671.2023.10291950(1621-1625)Online publication date: 15-Sep-2023
    • (2023)Joint Multi-Level Feature Network for Lightweight Person Re-IdentificationICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP49357.2023.10096139(1-5)Online publication date: 4-Jun-2023
    • (2023)Substation helmet detection based on improved YOLOX-S algorithm2023 IEEE 12th Data Driven Control and Learning Systems Conference (DDCLS)10.1109/DDCLS58216.2023.10167037(129-134)Online publication date: 12-May-2023
    • (2023)Network security situation assessment based on dual attention mechanism and HHO-ResNeXtConnection Science10.1080/09540091.2023.217408035:1Online publication date: 20-Feb-2023
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media