Skip to main content
SpringerLink
Log in

A joint model for entity and relation extraction based on BERT

  • Special Issue on Multi-modal Information Learning and Analytics on Big Data
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

In recent years, as the knowledge graph has attained significant achievements in many specific fields, which has become one of the core driving forces for the development of the internet and artificial intelligence. However, there is no mature knowledge graph in the field of agriculture, so it is a great significance study on the construction technology of agricultural knowledge graph. Named entity recognition and relation extraction are key steps in the construction of knowledge graph. In this paper, based on the joint extraction model LSTM-LSTM-Bias brought in BERT pre-training language model to proposed a agricultural entity relationship joint extraction model BERT-BILSTM-LSTM which is applied to the standard data set NYT and self-built agricultural data set AgriRelation. Experimental results showed that the model can effectively extracted the relationship between agricultural entities and entities.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Goldberg Y, Levy O (2014) word2vec explained: deriving mikolov et al.’s negative-sampling word-embedding method. arXiv:1402.3722

  2. Zheng S, Wang F, Bao H, Hao Y, Zhou P, Xu B (2017) Joint extraction of entities and relations based on a novel tagging scheme. arXiv:1706.05075

  3. Zhou Z, Shin J, Zhang L, Gurudu S, Gotway M, Liang J (2017) Fine-tuning convolutional neural networks for biomedical image analysis: actively and incrementally. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7340–7351

  4. Devlin J, Chang M-W, Lee K, Toutanova K (2018) Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805

  5. Bikel DM, Schwartz R, Weischedel RM (1999) An algorithm that learns what’s in a name. Mach Learn 34(1–3):211–231

    Article  Google Scholar 

  6. Fu G, Luke K-K (2005) Chinese named entity recognition using lexicalized hmms. ACM SIGKDD Explor Newsl 7(1):19–25

    Article  Google Scholar 

  7. Chieu HL, Ng HT (2002) Named entity recognition: a maximum entropy approach using global information. In: COLING 2002: the 19th international conference on computational linguistics

  8. Uchimoto K, Ma Q, Murata M, Ozaku H, Isahara H (2000),“amed entity extraction based on a maximum entropy model and transformation rules. In: Proceedings of the 38th annual meeting of the association for computational linguistics, pp 326–335

  9. Isozaki H, Kazawa H (2002) Efficient support vector classifiers for named entity recognition. In: COLING 2002: the 19th international conference on computational linguistics

  10. Chiu JP, Nichols E (2015) Named entity recognition with bidirectional lstm-cnns. arXiv:1511.08308

  11. Huang Z, Xu W, Yu K (2015) Bidirectional LSTM-CRF models for sequence tagging. arXiv:1508.01991

  12. Ma X, Hovy E (2016) End-to-end sequence labeling via bi-directional LSTM-CNNS-CRF. arXiv:1603.01354

  13. Lample G, Ballesteros M, Subramanian S, Kawakami K, Dyer C (2016) Neural architectures for named entity recognition. arXiv:1603.01360

  14. Wu H, Lu L, Yu B (2019) Chinese named entity recognition based on transfer learning and bilstm-crf. Small Micro Comput Syst 40:1142–1147

    Google Scholar 

  15. Mintz M, Bills S, Snow R, Jurafsky D (2009) Distant supervision for relation extraction without labeled data. In: Proceedings of the joint conference of the 47th annual meeting of the ACL and the 4th international joint conference on natural language processing of the AFNLP: volume 2-volume 2. Association for Computational Linguistics, pp 1003–1011

  16. Zelenko D, Aone C, Richardella A (2003) Kernel methods for relation extraction. J Mach Learn Res 3(Feb):1083–1106

    MathSciNet  MATH  Google Scholar 

  17. Zhou G, Zhang M, Ji D, Zhu Q (2007) Tree kernel-based relation extraction with context-sensitive structured parse tree information. In: Proceedings of the 2007 joint conference on empirical methods in natural language processing and computational natural language learning (EMNLP-CoNLL), pp 728–736

  18. Yao L, Riedel S, McCallum A (2010) Collective cross-document relation extraction without labelled data. In: Proceedings of the 2010 conference on empirical methods in natural language processing. Association for Computational Linguistics, pp 1013–1023

  19. Zeng D, Liu K, Lai S, Zhou G, Zhao J (2014) Relation classification via convolutional deep neural network. In: Proceedings of COLING 2014, the 25th international conference on computational linguistics: technical papers, pp 2335–2344

  20. Nguyen TH, Grishman R (2015) Relation extraction: perspective from convolutional neural networks. In: Proceedings of the 1st workshop on vector space modeling for natural language processing, pp 39–48

  21. dos Santos CN, Xiang B, Zhou B (2015) Classifying relations by ranking with convolutional neural networks. arXiv:1504.06580

  22. Socher R, Huval B, Manning CD, Ng AY (2012) Semantic compositionality through recursive matrix-vector spaces. In: Proceedings of the 2012 joint conference on empirical methods in natural language processing and computational natural language learning, pp 1201–1211

  23. Zhou P, Shi W, Tian J, Qi Z, Li B, Hao H, Xu B (2016) Attention-based bidirectional long short-term memory networks for relation classification. In: Proceedings of the 54th annual meeting of the association for computational linguistics (volume 2: short papers), pp 207–212

  24. Lin C, Miller T, Dligach D, Amiri H, Bethard S, Savova G (2018) Self-training improves recurrent neural networks performance for temporal relation extraction. In: Proceedings of the ninth international workshop on health text mining and information analysis, pp 165–176

  25. Zhang Y, Qi P, Manning CD (2018) Graph convolution over pruned dependency trees improves relation extraction. arXiv:1809.10185

  26. Zhu H, Lin Y, Liu Z, Fu J, Chua T-S, Sun M (2019) Graph neural networks with generated parameters for relation extraction. arXiv:1902.00756

  27. Shi P, Lin J (2019) Simple bert models for relation extraction and semantic role labeling. arXiv:1904.05255

  28. Shen T, Wang D, Feng S, Zhang Y (2019) Bert-based denoising and reconstructing data of distant supervision for relation extraction. In: CCKS2019-shared task

  29. Li Q, Ji H (2014) Incremental joint extraction of entity mentions and relations. In: Proceedings of the 52nd annual meeting of the association for computational linguistics (volume 1: long papers), vol 1, pp 402–412

  30. Miwa M, Sasaki Y (2014) Modeling joint entity and relation extraction with table representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 1858–1869

  31. Miwa M, Bansal M (2016) End-to-end relation extraction using lstms on sequences and tree structures. arXiv:1601.00770

  32. Zheng S, Hao Y, Lu D, Bao H, Xu J, Hao H, Xu B (2017) Joint entity and relation extraction based on a hybrid neural network. Neurocomputing 257:59–66

    Article  Google Scholar 

  33. Li F, Zhang M, Fu G, Ji D (2017) A neural joint model for entity and relation extraction from biomedical text. BMC Bioinform 18(1):198

    Article  Google Scholar 

  34. Fang LM et al (1994) Agricultural thesaurus (the third volume). China Agriculture Press, Beijing, pp 191–192

    Google Scholar 

Download references

Acknowledgements

Our works have been achieved significant help and supporting from Natural Science Foundation of Hunan Province of China (Grant No. 2019JJ40133), Natural Science Foundation of Hunan Province of China (Grant No. 2019JJ50239), Scientific Research Fund of Hunan Provincial Education Department of China (Grant No. 20A249), as well as the Key Research and Development Program of Hunan Province of China (Grant No. 2020NK2033).

Author information

Authors and Affiliations

  1. College of Information and Intelligence, Hunan Agricultural University, Changsha, Hunan, China

    Bo Qiao, Zhuoyang Zou, Kui Fang, Xinghui Zhu & Yiming Chen

  2. College of Biological Science and Technology, Hunan Agricultural University, Changsha, Hunan, China

    Yu Huang

Authors
  1. Bo Qiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhuoyang Zou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yu Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kui Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xinghui Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yiming Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Kui Fang or Xinghui Zhu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Qiao, B., Zou, Z., Huang, Y. et al. A joint model for entity and relation extraction based on BERT. Neural Comput & Applic 34, 3471–3481 (2022). https://doi.org/10.1007/s00521-021-05815-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-05815-z

Keywords

Search

Navigation