Skip to main content
Springer Nature Link
Log in

Named Entity Recognition with Context-Aware Dictionary Knowledge

  • Conference paper
  • First Online:
Chinese Computational Linguistics (CCL 2020)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 12522))

Included in the following conference series:

Abstract

Named entity recognition (NER) is an important task in the natural language processing field. Existing NER methods heavily rely on labeled data for model training, and their performance on rare entities is usually unsatisfactory. Entity dictionaries can cover many entities including both popular ones and rare ones, and are useful for NER. However, many entity names are context-dependent and it is not optimal to directly apply dictionaries without considering the context. In this paper, we propose a neural NER approach which can exploit dictionary knowledge with contextual information. We propose to learn context-aware dictionary knowledge by modeling the interactions between the entities in dictionaries and their contexts via context-dictionary attention. In addition, we propose an auxiliary term classification task to predict the types of the matched entity names, and jointly train it with the NER model to fuse both contexts and dictionary knowledge into NER. Extensive experiments on the CoNLL-2003 benchmark dataset validate the effectiveness of our approach in exploiting entity dictionaries to improve the performance of various NER models.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://www.geonames.org.

  2. 2.

    The performance of BERT is surprisingly unsatisfactory though we used the officially released model and carefully tuned hyperparameters.

References

  1. Akbik, A., Blythe, D., Vollgraf, R.: Contextual string embeddings for sequence labeling. In: COLING, pp. 1638–1649 (2018)

    Google Scholar 

  2. Arora, R., Tsai, C.T., Tsereteli, K., Kambadur, P., Yang, Y.: A semi-Markov structured support vector machine model for high-precision named entity recognition. In: ACL (2019)

    Google Scholar 

  3. Chen, H., Lin, Z., Ding, G., Lou, J., Zhang, Y., Karlsson, B.: GRN: gated relation network to enhance convolutional neural network for named entity recognition. In: AAAI (2019)

    Google Scholar 

  4. Chiu, J., Nichols, E.: Named entity recognition with bidirectional LSTM-CNNs. TACL 4(1), 357–370 (2016)

    Article  Google Scholar 

  5. Clark, K., Luong, M.T., Manning, C.D., Le, Q.: Semi-supervised sequence modeling with cross-view training. In: EMNLP, pp. 1914–1925 (2018)

    Google Scholar 

  6. Cohen, W.W., Sarawagi, S.: Exploiting dictionaries in named entity extraction: combining semi-Markov extraction processes and data integration methods. In: KDD, pp. 89–98. ACM (2004)

    Google Scholar 

  7. Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P.: Natural language processing (almost) from scratch. JMLR 12(Aug), 2493–2537 (2011)

    MATH  Google Scholar 

  8. Derczynski, L., et al.: Analysis of named entity recognition and linking for tweets. Inf. Process. Manag. 51(2), 32–49 (2015)

    Article  Google Scholar 

  9. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: NAACL-HLT, pp. 4171–4186 (2019)

    Google Scholar 

  10. Gui, T., et al.: A lexicon-based graph neural network for Chinese NER. In: EMNLP-IJCNLP, pp. 1039–1049 (2019)

    Google Scholar 

  11. Higashinaka, R., Sadamitsu, K., Saito, K., Makino, T., Matsuo, Y.: Creating an extended named entity dictionary from Wikipedia, In: COLING, pp. 1163–1178 (2012)

    Google Scholar 

  12. Huang, Z., Xu, W., Yu, K.: Bidirectional LSTM-CRF models for sequence tagging. arXiv preprint arXiv:1508.01991 (2015)

  13. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  14. Lample, G., Ballesteros, M., Subramanian, S., Kawakami, K., Dyer, C.: Neural architectures for named entity recognition. In: NAACL-HLT, pp. 260–270 (2016)

    Google Scholar 

  15. Levow, G.A.: The third international Chinese language processing bakeoff: word segmentation and named entity recognition. In: Proceedings of the Fifth SIGHAN Workshop on Chinese Language Processing, pp. 108–117 (2006)

    Google Scholar 

  16. Li, P.H., Dong, R.P., Wang, Y.S., Chou, J.C., Ma, W.Y.: Leveraging linguistic structures for named entity recognition with bidirectional recursive neural networks. In: EMNLP, pp. 2664–2669 (2017)

    Google Scholar 

  17. Lin, B.Y., Lu, W.: Neural adaptation layers for cross-domain named entity recognition. In: EMNLP, pp. 2012–2022 (2018)

    Google Scholar 

  18. Lin, H., Li, Y., Yang, Z.: Incorporating dictionary features into conditional random fields for gene/protein named entity recognition. In: Washio, T., et al. (eds.) PAKDD 2007. LNCS (LNAI), vol. 4819, pp. 162–173. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-77018-3_18

    Chapter  Google Scholar 

  19. Lin, Y., Shen, S., Liu, Z., Luan, H., Sun, M.: Neural relation extraction with selective attention over instances. In: ACL, pp. 2124–2133 (2016)

    Google Scholar 

  20. Liu, T., Yao, J.g., Lin, C.Y.: Towards improving neural named entity recognition with gazetteers. In: ACL, pp. 5301–5307 (2019)

    Google Scholar 

  21. Luo, X., Zhou, W., Wang, W., Zhu, Y., Deng, J.: Attention-based relation extraction with bidirectional gated recurrent unit and highway network in the analysis of geological data. IEEE Access 6, 5705–5715 (2018)

    Article  Google Scholar 

  22. Ma, X., Hovy, E.: End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF. In: ACL, vol. 1, pp. 1064–1074 (2016)

    Google Scholar 

  23. Magnolini, S., Piccioni, V., Balaraman, V., Guerini, M., Magnini, B.: How to use gazetteers for entity recognition with neural models. In: Proceedings of the 5th Workshop on Semantic Deep Learning, pp. 40–49 (2019)

    Google Scholar 

  24. Neelakantan, A., Collins, M.: Learning dictionaries for named entity recognition using minimal supervision. In: EACL, pp. 452–461 (2014)

    Google Scholar 

  25. Passos, A., Kumar, V., McCallum, A.: Lexicon infused phrase embeddings for named entity resolution. CoNLL-2014, p. 78 (2014)

    Google Scholar 

  26. Peng, M., Xing, X., Zhang, Q., Fu, J., Huang, X.: Distantly supervised named entity recognition using positive-unlabeled learning. In: ACL, pp. 2409–2419 (2019)

    Google Scholar 

  27. Peters, M., Ammar, W., Bhagavatula, C., Power, R.: Semi-supervised sequence tagging with bidirectional language models. In: ACL, vol. 1, pp. 1756–1765 (2017)

    Google Scholar 

  28. Peters, M., et al.: Deep contextualized word representations. In: NAACL-HLT, pp. 2227–2237 (2018)

    Google Scholar 

  29. Ratinov, L., Roth, D.: Design challenges and misconceptions in named entity recognition. In: CoNLL, pp. 147–155 (2009)

    Google Scholar 

  30. Rei, M.: Semi-supervised multitask learning for sequence labeling. In: ACL, pp. 2121–2130 (2017)

    Google Scholar 

  31. Rocktäschel, T., Huber, T., Weidlich, M., Leser, U.: WBI-NER: the impact of domain-specific features on the performance of identifying and classifying mentions of drugs. In: SemEval 2013, vol. 2, pp. 356–363 (2013)

    Google Scholar 

  32. Song, M., Yu, H., Han, W.S.: Developing a hybrid dictionary-based bio-entity recognition technique. BMC Med. Inform. Decis. Mak. 15(1), S9 (2015)

    Article  Google Scholar 

  33. Sui, D., Chen, Y., Liu, K., Zhao, J., Liu, S.: Leverage lexical knowledge for Chinese named entity recognition via collaborative graph network. In: EMNLP-IJCNLP, pp. 3821–3831 (2019)

    Google Scholar 

  34. Tjong Kim Sang, E.F., De Meulder, F.: Introduction to the CoNLL-2003 shared task: language-independent named entity recognition. In: NAACL-HLT, pp. 142–147 (2003)

    Google Scholar 

  35. Vaswani, A., et al.: Attention is all you need. In: NIPS, pp. 5998–6008 (2017)

    Google Scholar 

  36. Wan, X., et al.: Named entity recognition in Chinese news comments on the web. In: IJCNLP, pp. 856–864 (2011)

    Google Scholar 

  37. Wang, Q., Zhou, Y., Ruan, T., Gao, D., Xia, Y., He, P.: Incorporating dictionaries into deep neural networks for the chinese clinical named entity recognition. J. Biomed. Inform. 92, 103133 (2019)

    Article  Google Scholar 

  38. Yu, X., Lam, W., Chan, S.K., Wu, Y.K., Chen, B.: Chinese NER using CRFs and logic for the fourth SIGHAN bakeoff. In: Proceedings of the Sixth SIGHAN Workshop on Chinese Language Processing (2008)

    Google Scholar 

  39. Zeng, W., Tang, J., Zhao, X.: Entity linking on Chinese microblogs via deep neural network. IEEE Access 6, 25908–25920 (2018)

    Article  Google Scholar 

  40. Zhu, Y., Wang, G.: CAN-NER: convolutional attention network for Chinese named entity recognition. In: NAACL-HLT, pp. 3384–3393 (2019)

    Google Scholar 

Download references

Acknowledgments

Supported by the National Key Research and Development Program of China under Grant No. 2018YFC1604002, the National Natural Science Foundation of China under Grant Nos. U1936208, U1936216, U1836204 and U1705261.

Author information

Authors and Affiliations

  1. Department of Electronic Engineering and BNRist, Tsinghua University, Beijing, 100084, China

    Chuhan Wu, Tao Qi & Yongfeng Huang

  2. Microsoft Research Asia, Beijing, 100080, China

    Fangzhao Wu

Authors
  1. Chuhan Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fangzhao Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tao Qi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yongfeng Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chuhan Wu .

Editor information

Editors and Affiliations

  1. Tsinghua University, Beijing, China

    Maosong Sun

  2. Peking University, Beijing, China

    Sujian Li

  3. Westlake University, Hangzhou, China

    Yue Zhang

  4. Tsinghua University, Beijing, China

    Yang Liu

  5. Chinese Academy of Sciences, Beijing, China

    Shizhu He

  6. Beijing Language and Culture University, Beijing, China

    Gaoqi Rao

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wu, C., Wu, F., Qi, T., Huang, Y. (2020). Named Entity Recognition with Context-Aware Dictionary Knowledge. In: Sun, M., Li, S., Zhang, Y., Liu, Y., He, S., Rao, G. (eds) Chinese Computational Linguistics. CCL 2020. Lecture Notes in Computer Science(), vol 12522. Springer, Cham. https://doi.org/10.1007/978-3-030-63031-7_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-63031-7_10

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-63030-0

  • Online ISBN: 978-3-030-63031-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics