Skip to main content

Advertisement

Springer Nature Link
Log in

ETCGN: entity type-constrained graph networks for document-level relation extraction

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Document-level relation extraction aims at discerning semantic connections between entities within a given document. Compared with sentence-level relation extraction settings, the complexity of document-level relation extraction lies in necessitating models to exhibit the capability to infer semantic relations across multiple sentences. In this paper, we propose a novel model, named Entity Type-Constrained Graph Network (ETCGN). The proposed model utilizes a graph structure to capture intricate interactions among diverse mentions within the document. Moreover, it aggregates references to the same entity while integrating path-based reasoning mechanisms to deduce relations between entities. Furthermore, we present a novel constraint method that capitalizes on entity types to confine the scope of potential relations. Experimental results on two public dataset (DocRED and HacRED) show that our model outperforms a number of baselines and achieves state-of-the-art performance. Further analysis verifies the effectiveness of type-based constraints and path-based reasoning mechanisms. Our code is available at: https://github.com/yhx30/ETCGN.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

The datasets used in this study are publicly available. The DocRED dataset can be accessed at https://drive.google.com/drive/folders/1c5-0YwnoJx8NS6CV2f-NoTHR__BdkNqw, and the HacRED dataset can be accessed at https://drive.google.com/drive/folders/1T6QUfDV_ILAr6UJ_fROYQd4-NaFxIzqN.

References

  1. Cai R, Zhang X, Wang H (2016) Bidirectional recurrent convolutional neural network for relation classification. In: Proceedings of the 54th annual meeting of the association for computational linguistics (volume 1: long papers), pp 756–765

  2. Chen F, Wang X, Liu X, Peng D (2023) A trigger-free method enhanced by coreference information for document-level event extraction. In: International joint conference on neural networks, IJCNN 2023, Gold Coast, Australia, June 18–23, 2023. IEEE, pp 1–8. https://doi.org/10.1109/IJCNN54540.2023.10192046

  3. Cheng Q, Liu J, Qu X, Zhao J, Liang J, Wang Z, et al. (2021) Hacred: A large-scale relation extraction dataset toward hard cases in practical applications. In: Zong C, Xia F, Li W, Navigli R (eds) Findings of the association for computational linguistics: ACL/IJCNLP 2021, online event, August 1–6, 2021 (vol ACL/IJCNLP 2021). Association for Computational Linguistics, pp 2819–2831. https://doi.org/10.18653/v1/2021.findings-acl.249

  4. Christopoulou F, Miwa M, Ananiadou S (2019) Connecting the dots: Documentlevel neural relation extraction with edge-oriented graphs. In: Inui K, Jiang J, Ng V, Wan X (eds) Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3–7, 2019. Association for Computational Linguistics, pp 4924–4935. https://doi.org/10.18653/v1/D19-1498

  5. Devlin J, Chang M, Lee K, Toutanova K (2019) BERT: pre-training of deep bidirectional transformers for language understanding. In: Burstein J, Doran C, Solorio T (eds) Proceedings of the 2019 conference of the north American chapter of the association for computational linguistics: human language technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, vol 1 (long and short papers). Association for Computational Linguistics, pp 4171–4186. https://doi.org/10.18653/v1/n19-1423

  6. Fukushima K (1980) Neocognitron: a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biol Cybern 36(4):193–202

    Article  Google Scholar 

  7. Gupta P, Rajaram S, Schütze H, Runkler T (2019) Neural relation extraction within and across sentence boundaries. In: Proceedings of the AAAI conference on artificial intelligence, vol 33, pp 6513–6520

  8. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  Google Scholar 

  9. Hong Wang MY (2019) Deep graph library: towards efficient and scalable deep learning on graphs. In: International conference on learning representations workshop on representation learning on graphs and manifolds, ICLR, 2019

  10. Kipf TN, Welling M (2016) Semi-supervised classification with graph convolutional networks. CoRR arXiv:1609.02907v4

  11. Kullback S, Leibler RA (1951) On information and sufficiency. Ann Math Stat 22(1):79–86

    Article  MathSciNet  Google Scholar 

  12. Li X, Sun Q, Liu Y, Zhou Q, Zheng S, Chua T-S, Schiele B (2019) Learning to self-train for semi-supervised few-shot classification. Adv Neural Inf Process Syst 32:10276–10286

    Google Scholar 

  13. Lin Y, Shen S, Liu Z, Luan H, Sun M (2016) Neural relation extraction with selective attention over instances. In: Proceedings of the 54th annual meeting of the association for computational linguistics (vol 1: long papers), pp 2124–2133

  14. Loshchilov I, Hutter F (n.d.) Stochastic gradient descent with warm restarts. In: Proceedings of the 5th int. conf. learning representations, pp 1–16

  15. Mou L, Men R, Li G, Xu Y, Zhang L, Yan R, Jin Z (2016) Natural language inference by tree-based convolution and heuristic matching. In: Proceedings of the 54th annual meeting of the association for computational linguistics, ACL 2016, August 7–12, 2016, Berlin, Germany, vol 2: short papers. The Association for Computer Linguistics. https://doi.org/10.18653/v1/p16-2022

  16. Nan G, Guo Z, Sekulic I, Lu W (2020) Reasoning with latent structure refinement for document-level relation extraction. In: Jurafsky D, Chai J, Schluter N, Tetreault JR (eds) Proceedings of the 58th annual meeting of the association for computational linguistics, ACL 2020, online, July 5–10, 2020. Association for Computational Linguistics, pp 1546–1557. https://doi.org/10.18653/v1/2020.acl-main.141

  17. Nguyen DQ, Verspoor K (2018) Convolutional neural networks for chemicaldisease relation extraction are improved with character-based word embeddings. In: Demner-Fushman D, Cohen KB, Ananiadou S, Tsujii J (eds) Proceedings of the bionlp 2018 workshop, Melbourne, Australia, July 19, 2018. Association for Computational Linguistics, pp 129–136. https://doi.org/10.18653/v1/w18-2314

  18. Paszke A, Gross S, Chintala S, Chanan G, Yang E, DeVito Z, et al (2017) Automatic differentiation in pytorch, In: Advances in Neural Information Processing Systems Workshop, NeurIPS, October 28, 2017

  19. Peng N, Poon H, Quirk C, Toutanova K, Yih W-T (2017) Cross-sentence n-ary relation extraction with graph lstms. Trans Assoc Comput Linguistics 5:101–115

    Article  Google Scholar 

  20. Qiu L, Xiao Y, Qu Y, Zhou H, Li L, Zhang W, Yu Y (2019) Dynamically fused graph network for multi-hop reasoning. In: Proceedings of the 57th annual meeting of the association for computational linguistics, pp 6140–6150

  21. Quirk C, Poon H (2017) Distant supervision for relation extraction beyond the sentence boundary. In: Lapata M, Blunsom P, Koller A (eds) Proceedings of the 15th conference of the European chapter of the association for computational linguistics, EACL 2017, Valencia, Spain, April 3–7, 2017, vol 1: long papers. Association for Computational Linguistics, pp 1171–1182. https://doi.org/10.18653/v1/e17-1110

  22. Schuster M, Paliwal KK (1997) Bidirectional recurrent neural networks. IEEE Trans Signal Process 45(11):2673–2681

    Article  Google Scholar 

  23. Sorokin D, Gurevych I (2017) Context-aware representations for knowledge base relation extraction. In: Proceedings of the 2017 conference on empirical methods in natural language processing, pp 1784–1789

  24. Sun Q, Liu Y, Chua T-S, Schiele B (2019) Meta-transfer learning for few-shot learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 403–412

  25. Tang H, Cao Y, Zhang Z, Cao J, Fang F, Wang S, Yin P (2020) Hin: hierarchical inference network for document-level relation extraction. In: Advances in knowledge discovery and data mining: 24th pacific-Asia conference, PAKDD 2020, Singapore, May 11–14, 2020, proceedings, part I 24, pp 197–209

  26. Tran TT, Le P, Ananiadou S (2020) Revisiting unsupervised relation extraction. In: Jurafsky D, Chai J, Schluter N, Tetreault JR (eds) Proceedings of the 58th annual meeting of the association for computational linguistics, ACL 2020, online, July 5–10, 2020. Association for Computational Linguistics, pp 7498–7505. https://doi.org/10.18653/v1/2020.acl-main.669

  27. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN et al (2017) Attention is all you need. Adv Neural Inf Process Syst 30:5998–6008

    Google Scholar 

  28. Wang X, Li J, Yang X, Wang Y, Sang Y (2017) Chinese text sentiment analysis using bilinear character-word convolutional neural networks. In: Proceedings of international conference on computer science and application engineering, pp 36–43

  29. Wang H, Focke C, Sylvester R, Mishra N, Wang W (2019) Fine-tune bert for docred with two-step process. arXiv preprint arXiv:1909.11898

  30. Wang D, Hu W, Cao E, Sun W (2020) Global-to-local neural networks for document-level relation extraction. In: Webber B, Cohn T, He Y, Liu Y (eds) Proceedings of the 2020 conference on empirical methods in natural language processing, EMNLP 2020, online, November 16–20, 2020. Association for Computational Linguistics, pp 3711–3721. https://doi.org/10.18653/v1/2020.emnlp-main.303

  31. Xu B, Wang Q, Lyu Y, Zhu Y, Mao Z (2021) Entity structure within and throughout: Modeling mention dependencies for document-level relation extraction. In: AAAI conference on artificial intelligence. https://api.semanticscholar.org/CorpusID:231985811

  32. Yao Y, Ye D, Li P, Han X, Lin Y, Liu Z, et al (2019) Docred: a large-scale document-level relation extraction dataset. In: Korhonen A, Traum DR, Marquez L (eds) Proceedings of the 57th conference of the association for computational linguistics, ACL 2019, Florence, Italy, July 28–August 2, 2019, vol 1: long papers. Association for Computational Linguistics, pp 764–777. https://doi.org/10.18653/v1/p19-1074

  33. Yu M, Yin W, Hasan KS, dos Santos CN, Xiang B, Zhou B (2017) Improved neural relation detection for knowledge base question answering. In: Barzilay R, Kan M (eds) Proceedings of the 55th annual meeting of the association for computational linguistics, ACL 2017, Vancouver, Canada, July 30–August 4, vol 1: long papers. Association for Computational Linguistics, pp 571–581. https://doi.org/10.18653/v1/P17-1053

  34. Zeng D, Liu K, Lai S, Zhou G, Zhao J (2014) Relation classification via convolutional deep neural network. In: Proceedings of coling 2014, the 25th international conference on computational linguistics: technical papers, pp 2335–2344

  35. Zeng S, Xu R, Chang B, Li L (2020) Double graph based reasoning for document-level relation extraction. In: Webber B, Cohn T, He Y, Liu Y (eds) Proceedings of the 2020 conference on empirical methods in natural language processing, EMNLP 2020, online, November 16–20, 2020. Association for Computational Linguistics, pp 1630–1640. https://doi.org/10.18653/v1/2020.emnlp-main.127

  36. Zhang S, Zheng D, Hu X, Yang M (2015) Bidirectional long short-term memory networks for relation classification. In: Proceedings of the 29th pacific Asia conference on language, information and computation, pp 73–78

  37. Zhang Y, Zhong V, Chen D, Angeli G, Manning CD (2017) Position-aware attention and supervised data improve slot filling. In: Conference on empirical methods in natural language processing

  38. Zhang Y, Qi P, Manning CD (2018) Graph convolution over pruned dependency trees improves relation extraction. In: Riloff E, Chiang D, Hockenmaier J, Tsujii J (eds), Proceedings of the 2018 conference on empirical methods in natural language processing, Brussels, Belgium, October 31–November 4, 2018. Association for Computational Linguistics, pp 2205–2215. https://doi.org/10.18653/v1/d18-1244

  39. Zhang W, Shen H, Lei T, Wang Q, Peng D, Wang X (2023) GLQA: a generation-based method for legal question answering. In: International joint conference on neural networks, IJCNN 2023, Gold Coast, Australia, June 18-23, 2023. IEEE, pp 1–8. https://doi.org/10.1109/IJCNN54540.2023.10191483

  40. Zhou W, Huang K, Ma T, Huang J (2021). Document-level relation extraction with adaptive thresholding and localized context pooling. In: Proceedings of the AAAI conference on artificial intelligence vol 35, pp 14612–14620

Download references

Acknowledgements

This work is supported by Sichuan Science and Technology Planning Project (2023YFQ0020, 2023YFG0033, 2023ZHCG0016, 2022YFQ0014, 2022YFH0021), Chengdu Science and Technology Project (2023-XT00-00004-GX).

Author information

Authors and Affiliations

  1. College of Computer Science, Sichuan University, Chengdu, 610065, Sichuan, China

    Hangxiao Yang, Changpu Chen, Baiyang Chen & Chang Liu

  2. College of Electronics and Information Engineering, Sichuan University, Chengdu, 610065, Sichuan, China

    Shaokai Zhang

  3. Marketing Service Center of State Grid Sichuan Electric Power Corporation, Chengdu, 610045, Sichuan, China

    Qilin Li

Authors
  1. Hangxiao Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Changpu Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shaokai Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Baiyang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Chang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Qilin Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qilin Li.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix—Nomenclature

Appendix—Nomenclature

We use the nomenclature in Table 8 to denote the notations used in this paper:

Table 8 Nomenclature
Full size table

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, H., Chen, C., Zhang, S. et al. ETCGN: entity type-constrained graph networks for document-level relation extraction. Int. J. Mach. Learn. & Cyber. 15, 5949–5962 (2024). https://doi.org/10.1007/s13042-024-02293-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-024-02293-2

Keywords