Skip to main content
SpringerLink
Log in

CLGR-Net: a collaborative local-global reasoning network for document-level relation extraction

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Document-level relation extraction aims to model the reasoning information over multiple sentences of a document and capture complex dependency interactions between inter-sentence entities. However, modeling reasoning information effectively in the document remains a challenging task. In this paper, we propose a Collaborative Local-Global Reasoning Network (CLGR-Net) for the Document-Level Relation Extraction model to effectively predict such relations by integrating rich local and global information from the multi-granularity graph. Specifically, CLGR-Net first constructs a mention-level graph and a concept-level graph. The former aggregates complex local interactions underlying the same entities, the latter captures long-distance global interaction among different entities. Finally, it creates an entity-level graph, the nodes and edges of the entity graph are aggregated by Relational Graph Convolutional Networks (R-GCN) and enriched by probability Knowledge Graphs (KGs), based on which we design a novel hybrid reasoning mechanism to collaborate relevant global and local information for entities. In this way, our model can effectively model reasoning information from these three graphs. The mention-level graph and concept-level graph are used as auxiliary information for the entity-level graph in the form of independent heterogeneous graphs. Our CLGR-Net model achieves more competitive performance than state-of-the-art on three widely used benchmarks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data Availability

The datasets used in the experiments are publicly available in the online repository.

References

  1. Zhang Y, Qi P, Manning CD (2018) Graph convolution over pruned dependency trees improves relation extraction. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp 2205–2215. https://doi.org/10.18653/v1/d18-1244

  2. Soares LB, FitzGerald N, Ling J, Kwiatkowski T (2019) Matching the blanks: distributional similarity for relation learning. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL), pp 2895–2905. https://doi.org/10.18653/v1/p19-1279

  3. Alt C, Gabryszak A, Hennig L (2020) Probing linguistic features of sentence-level representations in neural relation extraction. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL), pp 2895–2905. https://doi.org/10.18653/v1/2020.acl-main.140

  4. Hu X, Zhang C, Ma F, Liu C, Wen L, Yu PS(2021) Semi-supervised relation extraction via incremental meta self-training. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp 1753–1762. https://doi.org/10.18653/v1/2021.findings-emnlp.44

  5. Christopoulou F, Miwa M, Ananiadou S (2019) Connecting the dots: document-level neural relation extraction with edge-oriented graphs. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp 4925–4936. https://doi.org/10.18653/v1/d19-1498

  6. Nan G, Guo Z, Sekulić I, Lu W (2020) Reasoning with latent structure refinement for document-level relation extraction. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL), pp 1546–1557. https://doi.org/10.18653/v1/2020.acl-main.141

  7. Xu W, Chen K, Zhao T (2021) Document-level relation extraction with reconstruction. In: Proceedings of the 35th AAAI Conference on Artificial Intelligence (AAAI), pp 14167–14175

  8. Li J, Xu K, Li F, Fei H, Ren Y, Ji D (2021) MRN: A locally and globally mention-based reasoning network for document-level relation extraction. In: Findings of the Association for Computational Linguistics (ACL-IJCNLP), pp 1359-1370. https://doi.org/10.18653/v1/2021.findings-acl.117

  9. Yuan C, Huang H, Feng C, Shi G, Wei X (2021) Document-level relation extraction with entity-selection attention. Inf Sci 568:163–174. https://doi.org/10.1016/j.ins.2021.04.007

    Article  Google Scholar 

  10. Yao Y, Ye D, Li P, Han X, Lin Y, Liu Z , Sun M (2019) DocRED: A large-scale document-level relation extraction dataset. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL), pp 764–777. https://doi.org/10.18653/v1/p19-1074

  11. Wu W, Li H, Wang H, Zhu KQ (2012) Probase: a probabilistic taxonomy for text understanding. In: Proceedings of the ACM SIGMOD International Conference on Management of Data (SIGMOD), pp 481–492. https://doi.org/10.1145/2213836.2213891

  12. Schlichtkrull M, Kipf TN, Bloem P, Berg RVD, Titov I, Welling M (2018) Modeling relational data with graph convolutional networks. In: European Semantic Web Conference (ESWC), pp 593-607. https://doi.org/10.1007/978-3-319-93417-4_38

  13. Speer R, Chin J, Havasi C (2017) Conceptnet 5.5: An open multilingual graph of general knowledge. In: Proceedings of the 31st AAAI Conference on Artificial Intelligence (AAAI), pp 4444–4451

  14. Zeng S, Xu R, Chang, B, Li L (2020) Double graph based reasoning for document-level relation extraction. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp 1630–1640. https://doi.org/10.18653/v1/2020.emnlp-main.127

  15. Tang H, Cao Y, Zhang Z, Cao J, Fang F, Wang S, Yin P (2020) HIN: hierarchical inference network for document-level relation extraction. In: Proceedings of the 2019 Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD), pp 197–209. https://doi.org/10.1007/978-3-030-47426-3_16

  16. Wang H, Qin K, Lu G, Yin J, Zakari RY, Owusu JW (2021) Document-level relation extraction using evidence reasoning on RST-GRAPH. Knowl-Based Syst 228:107274. https://doi.org/10.1016/j.knosys.2021.107274

    Article  Google Scholar 

  17. Xu B, Wang Q, Lyu Y, Zhu Y, Mao Z (2021) Entity structure within and throughout: modeling mention dependencies for document-level relation extraction. In: Proceedings of the 35th AAAI Conference on Artificial Intelligence (AAAI), pp 14149–14157

  18. Li B, Ye W, Huang C, Zhang S (2021) Multi-view inference for relation extraction with uncertain knowledge. In: Proceedings of the 35th AAAI Conference on Artificial Intelligence (AAAI), pp 13234-13242

  19. Hao J, Chen M, Yu W, Sun Y, Wang W (2019) Universal representation learning of knowledge bases by jointly embedding instances and ontological concepts. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp 1709–1719. https://doi.org/10.1145/3292500.3330838

  20. Bahdanau D, Cho K, Bengio Y (2015) Neural machine translation by jointly learning to align and translate. In: 3rd International Conference on Learning Representations (ICLR), https://doi.org/10.48550/arXiv.1409.0473

  21. Li J, Sun Y, Johnson RJ, Sciaky D, Wei CH, Leaman R, Lu Z (2016) BioCreative V CDR task corpus: a resource for chemical disease relation extraction. Database: J Biol Databases Curation, https://doi.org/10.1093/database/baw068

  22. Wu Y, Luo R, Leung H, Ting HF, Lam TW (2019) Renet: a deep learning approach for extracting gene-disease associations from literature. Res Comput Mol Biol 25:272–284. https://doi.org/10.1007/978-3-030-17083-7_17

    Article  MATH  Google Scholar 

  23. Zhou W, Huang K, Ma T, Huang J (2021) Document-level relation extraction with adaptive thresholding and localized context pooling. In: Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), pp 14612-14620

  24. Wang D, Hu W, Cao E, Sun W (2020) Global-to-local neural networks for document-level relation extraction. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp 3711–3721. https://doi.org/10.18653/v1/2020.emnlp-main.303

  25. Devlin J, Chang M, Lee K, Toutanova K (2019) BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), pp 4171–4186

  26. Wang H, Focke C, Sylvester R, Mishra N, Wang W (2019) Fine-tune Bert for DocRED with two-step process. http://arxiv.org/abs/1909.11898

  27. Beltagy I.; Lo, K.; and Cohan, A. 2019. SciBERT: A pretrained language model for scientific text. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp 3615–3620. https://doi.org/10.18653/v1/d19-1371

  28. Pennington J, Socher R, Manning CD (2014) Glove: Global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp 1532–1543. https://doi.org/10.3115/v1/d14-1162

  29. Loshchilov I, Hutter F (2019) Decoupled weight decay regularization. In: International Conference on Learning Representations (ICLR), https://doi.org/10.48550/arXiv:1711.05101

  30. Paszke A, Gross S, Chintala S, Chanan G, Yang E, DeVito Z, Lerer A (2017) Automatic differentiation in pytorch. In: Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS). https://doi.org/10.18653/v1/d18-1244

Download references

Acknowledgements

This work is supported by the Natural Science Foundation of Henan Province, China, under grant No. 222300420590. Also, we thank all the reviewers and editors for their feedback.

Funding

This study was financed in part by the Natural Science Foundation of Henan Province, China under grant No. 222300420590.

Author information

Authors and Affiliations

  1. PLA Strategic Support Force Information Engineering University, Zhengzhou, 450000, China

    Xiaoyao Ding, Gang Zhou, Jicang Lu & Taojie Zhu

Authors
  1. Xiaoyao Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gang Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jicang Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Taojie Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

XD Conceptualization, Design, Software, Validation, Formal analysis, Investigation, Data curation, Writing—original draft, Writing—review & editing, and Visualization. GZ Conceptualization, Writing—review & editing, and Supervision. JL Conceptualization, Writing—review & editing, and Supervision. TZ Conceptualization, Writing—review & editing, and Supervision.

Corresponding author

Correspondence to Gang Zhou.

Ethics declarations

Conflict of interest

There are no conflicts or competing interests.

Consent for publication

There is the consent of all authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ding, X., Zhou, G., Lu, J. et al. CLGR-Net: a collaborative local-global reasoning network for document-level relation extraction. J Supercomput 79, 5469–5485 (2023). https://doi.org/10.1007/s11227-022-04875-9

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-022-04875-9

Keywords

Search

Navigation