Skip to main content
SpringerLink
Log in

Multi-perspective context aggregation for document-level relation extraction

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

The task of document-level relation extraction (RE) involves integration of information within and across multiple sentences of a document and extraction of complex semantic relations between multiple named entities. However, effective aggregation of local and nonlocal contexts information in the document continues to be a challenging research question. This study proposes a novel document-level RE model, called the multi-perspective context aggregation (MPCA), that aggregates document context information from multi-perspective at different layers. Specifically, this aggregated context information not only comes from the pre-training stage but is also reflected during node construction and classification. Experimental results show that our model achieves desirable performance on two public datasets for document-level RE and is particularly effective in extracting relations between entities with multiple mentions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Yu M, Yin W, Hasan KS, Santos CD, Xiang B, Zhou B (2017) Improved neural relation detection for knowledge base question answering. In: Proceedings of the 55th annual meeting of the association for computational linguistics(ACL), pp 571–581

  2. Qiu L, Xiao Y, Qu Y, Zhou H, Li L, Zhang W, Yu Y (2019) Dynamically fused graph network for multihop reasoning. In: Proceedings of the 57th annual meeting of the association for computational linguistics(ACL), pp 6140–6150

  3. Li J, Sun Y, Johnson RJ, Sciaky D, Wei CH, Leaman R, Lu Z (2016) BioCreative V CDR task corpus: a resource for chemical disease relation extraction. Database, 2016

  4. Wu Y, Luo R, Leung H, Ting HF, Lam TW (2019) Renet: a deep learning approach for extracting gene-disease associations from literature. In: International conference on research in computational molecular biology (RE-COMB), pp 272–284

  5. Beltagy I, Lo K, Cohan A (2019) SciBERT: a pretrained language model for scientific text. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), pp 3615–3620

  6. Xiao M, Cong L (2016) Semantic relation classification via hierarchical recurrent neural network with attention. In: Proceedings of COLING 2016, the 26th international conference on computational linguistics(COLING), pp 1254–1263

  7. Zhang Y, Zhong V, Chen D, Angeli G, Manning CD (2017) Position-aware attention and supervised data improve slot filling. In: Proceedings of the 2017 conference on empirical methods in natural language processing(EMNLP), pp 35–45

  8. Zhang Y, Qi P, Manning CD (2018) Graph convolution over pruned dependency trees improves relation extraction. In: Proceedings of the 2018 conference on empirical methods in natural language processing(EMNLP), pp 2205–2215

  9. Soares LB, FitzGerald N, Ling J, Kwiatkowski T (2019) Matching the blanks: distributional similarity for relation learning. In: Proceedings of the 57th annual meeting of the association for computational linguistics(ACL), pp 2895–2905

  10. Verga P, Strubell E, McCallum A (2018) Simultaneously self-attending to all mentions for full-abstract biological relation extraction. In: Proceedings of the 2018 conference of the North American chapter of the association for computational linguistics (NAACL), pp 872–884

  11. Christopoulou F, Miwa M, Ananiadou S (2019) Connecting the dots: document-level neural relation extraction with edge-oriented graphs. In: Proceedings of the 2019 conference on empirical methods in natural language processing(EMNLP), pp 4925–4936

  12. Sahu SK, Christopoulou F, Miwa M, Ananiadou S (2019) Inter-sentence relation extraction with document-level graph convolutional neural network. In: Proceedings of the 57th annual meeting of the association for computational linguistics(ACL), pp 4309–4316

  13. Yao Y, Ye D, Li P, Han X, Lin Y, Liu Z, Sun M (2019) DocRED: a large-scale document-level relation extraction dataset. In: Proceedings of the 57th annual meeting of the association for computational linguistics(ACL), pp 764–777

  14. Nan G, Guo Z, Sekulić I, Lu W (2020) Reasoning with latent structure refinement for document-level relation extraction. In: Proceedings of the 58th annual meeting of the association for computational linguistics(ACL), pp 1546–1557

  15. Tang H, Cao Y, Zhang Z, Cao J, Fang F, Wang S, Yin P (2020) HIN:hierarchical inference networkfor document-level relation extraction. In: Proceedings of the 2019 Pacific-Asia conference on knowledge discovery and data mining(PAKDD), pp 197–209

  16. Wang D, Hu W, Cao E, Sun W (2020) Global-to-local neural networks for document-level relation extraction. In: Proceedings of the 2020 conference on empirical methods in natural language processing(EMNLP), pp 3711–3721

  17. Xu W, Chen K, Zhao T (2021) Document-level relation extraction with reconstruction. In: The 35th AAAI conference on artificial intelligence (AAAI)

  18. Liu Y, Lapata M (2018) Learning structured text representations. In: Transactions of the association for computational linguistics(TACL), pp 63–75

  19. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  Google Scholar 

  20. Chung J, Gulcehre C, Cho K, Bengio Y (2014) Empirical evaluation of gated recurrent neural networks on sequence modeling. In: Proceedings of the 2014 international conference and workshop on neural information processing systems(NIPS), pp 1–9

  21. Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J (2013) Distributed representations of words and phrases and their compositionality. Neur Inform, 3111–3119

  22. Pennington J, Socher R, Manning CD (2014) Glove: global vectors for word representation. In: proceedings of the 2014 conference on empirical methods in natural language processing(EMNLP), pp 1532–1543

  23. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Polosukhin I (2017) Attention is all you need. Neural information, 5998–6008

  24. Tenney I, Das D, Pavlick E (2019) BERT rediscovers the classical NLP pipeline. In: Proceedings of the 57th annual meeting of the association for computational linguistics(ACL), pp 4593–4601

  25. Devlin J, Chang M, Lee K, Toutanova K (2019) BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics(NAACL), pp 4171–4186

  26. Zhou W, Huang K, Ma T, Huang J (2021) Document-level relation extraction with adaptive thresholding and localized context pooling. In: Proceedings of the AAAI conference on artificial intelligence(AAAI), pp 14612–14620

  27. Wang H, Focke C, Sylvester R, Mishra N, Wang W (2019) Fine-tune bert for DocRED with two-step process. arXiv preprint arXiv:1909.11898

  28. Schlichtkrull M, Kipf TN, Bloem P, Berg RVD, Titov I, Welling M (2018) Modeling relational data with graph convolutional networks. In: European semantic web conference(ESWC), pp 593–607

  29. Sorokin D, Gurevych I (2017) Context-aware representations for knowledge base relation extraction. In: Proceedings of the 2017 conference on empirical methods in natural language processing(EMNLP), pp 1784–1789

  30. Jia R, Wong C, Poon H (2019) Document-level N-ary relation extraction with multiscale representation learning. In: Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics(NAACL), pp 3693–3704

  31. Mou L, Men R, Li G, Xu Y, Zhang L, Yan R, Jin Z (2016) Natural language inference by tree-based convolution and heuristic matching. In: Proceedings of the 54th annual meeting of the association for computational linguistics(ACL), pp 130–136

  32. Hendrickx I, Kim SN, Kozareva Z, Nakov P, Séaghdha DO, Padó S, Szpakowicz S (2009) Semeval-2010 task 8: multi-way classification of semantic relations between pairs of nominals. In: Proceedings of the 2009 conference of the North American chapter of the association for computational linguistics(NAACL), pp 33–38

  33. Wang H, Qin K, Lu G, Yin J, Zakari RY, Owusu JW (2021) Document-level relation extraction using evidence reasoning on RST-GRAPH, vol 228, p 107274

  34. Fukushima K (1980) Neocognitron: a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position, vol 36, pp 193–202

  35. Schuster M, Paliwal KK (1997) Bidirectional recurrent neural networks, vol 45, pp 2673–2681

  36. Veličković P, Cucurull G, Casanova A, Romero A, Lio P, Bengio Y (2018) Graph attention networks. In: International conference on learning representations(ILCR)

  37. Han X, Wang L (2020) A novel document-level relation extraction method based on BERT and entity information. IEEE Access 8:96912–96919

    Article  Google Scholar 

  38. Loshchilov I, Hutter F (2019) Decoupled weight decay regularization. In: International conference on learning representations(ILCR)

  39. Paszke A, Gross S, Chintala S, Chanan G, Yang E, DeVito Z, Lerer A (2017) Automatic differentiation in pytorch. In: NIPS-W

  40. Jiang F, Li P, Zhu Q (2021) Recognizing Chinese discourse relations based on multi-perspective and hierarchical modeling. In: 2021 International joint conference on neural networks (IJCNN), pp 1–8

  41. Allaway E, Wang S, Ballesteros M (2021) Sequential cross-document coreference resolution. arXiv preprint arXiv:2104.08413

  42. Lin JCW, Shao Y, Djenouri Y, Yun U (2021) ASRNN: a recurrent neural network with an attention model for sequence labeling, vol 212, p 106548

  43. Xiong Y, Feng Y, Wu H, Kamigaito H, Okumura M (2021) Fusing label embedding into BERT: an efficient improvement for text classification. In: Findings of the association for computational linguistics(ACL-IJCNLP), pp 1743–1750

  44. Xu B, Wang Q, Lyu Y, Zhu Y, Mao Z (2021) Entity structure within and throughout: modeling mention dependencies for document-level relation extraction. In: The 35th AAAI conference on artificial intelligence (AAAI)

Download references

Acknowledgments

This work is supported by the Science and Technology Research Project of Henan Province (No. 192102210129).

Author information

Authors and Affiliations

  1. PLA Strategic Support Force Information Engineering University, Zhengzhou, 450000, China

    Xiaoyao Ding, Gang Zhou & Taojie Zhu

Authors
  1. Xiaoyao Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gang Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Taojie Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gang Zhou.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ding, X., Zhou, G. & Zhu, T. Multi-perspective context aggregation for document-level relation extraction. Appl Intell 53, 6926–6935 (2023). https://doi.org/10.1007/s10489-022-03731-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-03731-w

Keywords

Search

Navigation