Skip to main content
Springer Nature Link
Log in

Graph neural networks with selective attention and path reasoning for document-level relation extraction

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Document-level Relation Extraction (DocRE) aims to extract relations from multiple sentences simultaneously. Existing graph-based methods adopt static graphs to represent the document structure, which is unable to capture complex interactions. Besides, they take all sentences in the document as the scope of relation extraction (RE) while introducing noise by irrelevant sentences. Furthermore, they do not explicitly model the reasoning chain, leading to a lack of explainability in the reasoning results. These limitations may significantly hinder their performance in practical applications. In this paper, we propose a model based on selective attention and path reasoning for DocRE. Firstly, we adopt hierarchical heterogeneous graph neural networks and recurrent neural networks to realize document modeling and capture complex interactions in the document. Secondly, we adopt selective attention to select sentences related to the entity pair to generate document subgraphs as the scope of RE. Lastly, we adopt path reasoning to explicitly model the reasoning chain between multiple entities in the document subgraph, infer the relations between entities and provide corresponding supporting evidence. Extensive experiment results on three benchmark datasets show that the proposed framework is effective and achieves superior performance compared to most methods. Further analysis demonstrates that selective attention and path reasoning can discover more accurate inter-sentence relations and supporting evidence.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Notes

  1. https://github.com/thunlp/DocRED

  2. https://biocreative.bioinformatics.udel.edu/media/store/files/2016/CDR_Data.zip

  3. https://bitbucket.org/alexwuhkucs/gda-extraction/get/fd4a7409365e.zip

  4. https://competitions.codalab.org/competitions/20717#results

  5. https://pytorch.org

References

  1. Distiawan B, Weikum G, Qi J, Zhang R (2019) Neural relation extraction for knowledge base enrichment. In: Proceedings of the 57th annual meeting of the association for computational linguistics. https://doi.org/10.18653/v1/p19-1023

  2. Yu M, Yin W, Hasan KS, dos Santos C, Xiang B, Zhou B (2017) Improved neural relation detection for knowledge base question answering. In: Proceedings of the 55th annual meeting of the association for computational linguistics. https://doi.org/10.18653/v1/P17-1053

  3. Lai T, Cheng L, Wang D, Ye H, Zhang W (2022) Rman: Relational multi-head attention neural network for joint extraction of entities and relations. Appl Intell 52(3):3132–3142. https://doi.org/10.1007/s10489-021-02600-2

  4. Li X, Li Y, Yang J, Liu H, Hu P (2022) A relation aware embedding mechanism for relation extraction. Appl Intell, pp 1–10. https://doi.org/10.1007/s10489-021-02699-3

  5. Christopoulou F, Miwa M, Ananiadou S (2018) A walk-based model on entity graphs for relation extraction. In: Proceedings of the 56th annual meeting of the association for computational linguistics. https://doi.org/10.18653/v1/P18-2014

  6. Zhu H, Lin Y, Liu Z, Fu J, Chua T-s, Sun M (2019) Graph neural networks with generated parameters for relation extraction. In: Proceedings of the 57th conference of the association for computational linguistics. https://doi.org/10.18653/v1/p19-1128

  7. Wang H, Qin K, Lu G, Luo G, Liu G (2020) Direction-sensitive relation extraction using bi-sdp attention model. Knowl Based Syst, pp 105928. https://doi.org/10.1016/j.knosys.2020.105928

  8. Hang T, Feng J, Wu Y, Yan L, Wang Y (2021) Joint extraction of entities and overlapping relations using source-target entity labeling. Expert Syst Appl 177:114853. https://doi.org/10.1016/j.eswa.2021.114853

  9. Hang T, Feng J, Yan L, Wang Y, Lu J (2022) Joint extraction of entities and relations using multi-label tagging and relational alignment. Neural Comput Appl 34(8):6397–6412. https://doi.org/10.1007/s00521-021-06685-1

  10. Zeng D, Liu K, Lai S, Zhou G, Zhao J (2014) Relation classification via convolutional deep neural network. In: Proceedings of the 4th international conference on learning representations

  11. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

  12. Cai R, Zhang X, Wang H (2016) Bidirectional recurrent convolutional neural network for relation classification. https://doi.org/10.18653/v1/p16-1072

  13. Sorokin D, Gurevych I (2017) Context-Aware representations for knowledge base relation extraction. In: Proceedings of the 2017 conference on empirical methods in natural language processing

  14. Sahu SK, Christopoulou F, Miwa M, Ananiadou S (2019) Inter-sentence relation extraction with document-level graph convolutional neural network. In: Proceedings of the 57th annual meeting of the association for computational linguistics. https://doi.org/10.18653/v1/p19-1423

  15. Christopoulou F, Miwa M, Ananiadou S (2019) Connecting the dots: document-level neural relation extraction with edge-oriented graphs. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing. https://doi.org/10.18653/v1/D19-1498

  16. Wang D, Hu W, Cao E, Sun W (2020) Global-to-local neural networks for document-level relation extraction. In: Proceedings of the 2020 conference on empirical methods in natural language processing. https://doi.org/10.18653/v1/2020.emnlp-main.303

  17. Guo Z, Zhang Y, Lu W (2019) Attention guided graph convolutional networks for relation extraction. In: Proceedings of the 57th annual meeting of the association for computational linguistics. https://doi.org/10.18653/v1/p19-1024

  18. Nan G, Guo Z, Sekulic I, Lu W (2020) Reasoning with latent structure refinement for document-Level relation extraction. In: Proceedings of the 58th annual meeting of the association for computational linguistics. https://doi.org/10.18653/v1/2020.acl-main.141

  19. Zeng S, Xu R, Chang B, Li L (2020) Double graph based reasoning for document-level relation extraction. In: Proceedings of the 2020 conference on empirical methods in natural language processing. https://doi.org/10.18653/v1/2020.emnlp-main.127

  20. Tang H, Cao Y, Zhang Z, Cao J, Fang F, Wang S, Yin P (2020) HIN: hierarchical inference network for document-level relation extraction. In: Pacific-Asia conference on knowledge discovery and data mining. https://doi.org/10.1007/978-3-030-47426-3_16

  21. Zhou W, Huang K, Ma T, Huang J (2021) Document-level relation extraction with adaptive thresholding and localized context pooling. In: Proceedings of the 35th AAAI conference on artificial intelligence

  22. Xu B, Wang Q, Lyu Y, Zhu Y, Mao Z (2021) Entity structure within and throughout: modeling mention dependencies for document-level relation extraction. In: Proceedings of the 35th AAAI conference on artificial intelligence

  23. Zhang N, Chen X, Xie X, Deng S, Tan C, Chen M, Huang F, Si L, Chen H, Center HI (2021) Document-level relation extraction as semantic segmentation. In: Proceedings of the 30th international joint conference on artificial intelligence. https://doi.org/10.24963/ijcai.2021/5517

  24. Xiao Y, Zhang Z, Mao Y, Yang C, Han, J (2022) SAIS: supervising and augmenting intermediate steps for document-level relation extraction. https://doi.org/10.18653/v1/2022.naacl-main.171

  25. Quirk C, Poon H (2017) Distant supervision for relation extraction beyond the sentence boundary. In: Proceedings of the 15th conference of the european chapter of the association for computational linguistics. https://doi.org/10.18653/v1/e17-1110

  26. Peng N, Poon H, Quirk C, Toutanova K, Yih W-t (2017) Cross-sentence n-ary relation extraction with graph lstms. Trans Assoc Comput Linguistics 5:101–115. https://doi.org/10.1162/tacl_a_00049

  27. Song L, Zhang Y, Wang Z, Gildea D (2018) N-ary relation extraction using graph state LSTMs. In: Proceedings of the 2018 conference on empirical methods in natural language processing. https://doi.org/10.18653/v1/d18-1246

  28. Zhang Z, Yu B, Shu X, Liu T, Tang H, Yubin W, Guo L (2020) Document-level relation extraction with dual-tier heterogeneous graph. In: Proceedings of the 28th international conference on computational linguistics. https://doi.org/10.18653/v1/2020.coling-main.143

  29. Pennington J, Socher R, Manning CD (2014) Glove: global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing. https://doi.org/10.3115/v1/d14-1162

  30. Chiu B, Crichton G, Korhonen A, Pyysalo S (2016) How to train good word embeddings for biomedical NLP. In: Proceedings of the 15th workshop on biomedical natural language processing. https://doi.org/10.18653/v1/W16-2922

  31. Li Y, Tarlow D, Brockschmidt M, Zemel R (2016) Gated graph sequence neural networks. In: Proceedings of the 4th international conference on learning representations

  32. Song L, Zhang Y, Wang Z, Gildea D (2018) A graph-to-sequence model for amr-to-text generation. In: Proceedings of the 56th annual meeting of the association for computational linguistics. https://doi.org/10.18653/v1/P18-1150

  33. Zhang Y, Liu Q, Song L (2018) Sentence-state LSTM for text representation. In: Proceedings of the 56th annual meeting of the association for computational linguistics. https://doi.org/10.18653/v1/P18-1030

  34. Jia R, Wong C, Poon H (2019) Document-Level N-ary relation extraction with multiscale representation learning. In: Proceedings of the 2019 conference of the north american chapter of the association for computational linguistics: human language technologies. https://doi.org/10.18653/v1/n19-1370

  35. Yao Y, Ye D, Li P, Han X, Lin Y, Liu Z, Liu Z, Huang L, Zhou J, Sun M (2019) DocRED: A large-scale document-level relation extraction dataset. In: Proceedings of the 57th annual meeting of the association for computational linguistics. https://doi.org/10.18653/v1/p19-1074

  36. Li J, Sun Y, Johnson RJ, Sciaky D, Wei C-H, Leaman R, Davis AP, Mattingly CJ, Wiegers TC, Lu Z (2016) Biocreative v cdr task corpus: a resource for chemical disease relation extraction. Database J Biol Databases Curation 2016. https://doi.org/10.1093/database/baw068

  37. Wu Y, Luo R, Leung HC, Ting H-F, Lam T-W (2019) Renet: A deep learning approach for extracting gene-disease associations from literature. In: Proceedings of the 23rd international conference on research in computational molecular biology. https://doi.org/10.1007/978-3-030-17083-7_17

  38. Tan Q, He R, Bing L, Ng HT (2022) Document-level relation extraction with adaptive focal loss and knowledge distillation. In: Findings of the association for computational linguistics: ACL 2022. https://doi.org/10.18653/v1/2022.findings-acl.132

  39. Paszke A, Gross S, Chintala S, Chanan G, Yang E, DeVito Z, Lin Z, Desmaison A, Antiga L, Lerer A (2017) Automatic differentiation in pytorch. In: Proceedings of NIPS 2017 workshop

  40. Kingma DP, Ba J(2015) Adam: a method for stochastic optimization. In: Proceedings of the 3th international conference on learning representations

  41. Caruana R, Lawrence S, Giles CL (2000) Overfitting in neural nets: backpropagation, conjugate gradient, and early stopping. In: Proceedings of the advances in neural information processing systems 13

  42. Devlin J, Chang M-W, Lee K, Toutanova K (2019) Bert: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 conference of the north american chapter of the association for computational linguistics: human language technologies. https://doi.org/10.18653/v1/n19-1423

  43. Liu Y, Ott M, Goyal N, Du J, Joshi M, Chen D, Levy O, Lewis M, Zettlemoyer L, Stoyanov V (2019) Roberta: A robustly optimized bert pretraining approach. arXiv:1907.11692

  44. Beltagy I, Lo K, Cohan A (2019) SciBERT: A pretrained language model for scientific text. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing . https://doi.org/10.18653/v1/D19-1371

  45. Huang K, Wang G, Ma T, Huang J (2020) Entity and evidence guided relation extraction for docred. arXiv:2008.12283

  46. Xie Y, Shen J, Li S, Mao Y, Han J (2021) Eider: Evidence-enhanced document-level relation extraction. arXiv:2106.08657

  47. Li B, Ye W, Sheng Z, Xie R, Xi X, Zhang S (2020) Graph enhanced dual attention network for document-level relation extraction. In: Proceedings of the 28th international conference on computational linguistics. https://doi.org/10.18653/v1/2020.coling-main.136

Download references

Acknowledgements

The authors would like to thank the anonymous reviewers for their encouragement and helpful comments. The paper is supported by the National Natural Science Foundation of China (Grant No. 62306007), the National Key R &D Program of China (Grant No. 2021YFB3900601), and the Natural Science Foundation of Anhui Province (Grant No. 2008085QF305).

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Anhui University of Technology, Ma’anshan, China

    Tingting Hang

  2. Key Laboratory of Water Big Data Technology of Ministry of Water Resource, Hohai University, Nanjing, China

    Jun Feng & Yunfeng Wang

  3. School of Computer and Information, Hohai University, Nanjing, China

    Jun Feng & Yunfeng Wang

  4. College of Information Engineering, Nanjing Xiaozhuang University, Nanjing, China

    Le Yan

Authors
  1. Tingting Hang
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Jun Feng
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Yunfeng Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Le Yan
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Tingting Hang.

Ethics declarations

Conflicts of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hang, T., Feng, J., Wang, Y. et al. Graph neural networks with selective attention and path reasoning for document-level relation extraction. Appl Intell 54, 5353–5372 (2024). https://doi.org/10.1007/s10489-024-05448-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-024-05448-4

Keywords