Skip to main content
SpringerLink
Log in

A relation aware embedding mechanism for relation extraction

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Extracting possible relational triples from natural language text is a fundamental task of information extraction, which has attracted extensive attention. The embedding mechanism has a significant impact on the performance of relation extraction models, and the embedding vectors should contain rich semantic information that has close relevance to the relation extraction task. Driven by this motivation, we propose a Relation Aware Embedding Mechanism (RA) for relation extraction. In specific, this mechanism incorporates the relation label information into sentence embedding by leveraging the attention mechanism to distinguish the importance of different relation labels to each word of a sentence. We apply the proposed method to three state-of-the-art relation extraction models: CasRel, SMHSA and ETL-Span, and implement the corresponding models named RA-CasRel, RA-SMHSA and RA-ETL-Span. To evaluate the effectiveness of our method, we conduct extensive experiments on two widely-used open datasets: NYT and WebNLG, and compare RA-CasRel, RA-SMHSA and RA-ETL-Span with 12 state-of-the-art models. The experimental results show that our method can effectively improve the performance of relation extraction. For instance, RA-CasRel reaches 91.7% and 92.4% of F1-score on NYT and WebNLG, respectively, which is the best performance among all the compared models. We have open sourced the code of our proposed method in [1] to facilitate future research in relation extraction.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. Each word in a sentence is tokenized to fine-grained tokens.

  2. In this paper, all vectors and matrices are represented by bold symbols.

References

  1. Ra-casrel. https://github.com/lixiang20/RaCasRel

  2. Zelenko D, Aone C, Richardella A (2003) Kernel methods for relation extraction. J Mach Learn Res 3(3):1083–1106

    MathSciNet  MATH  Google Scholar 

  3. Zhou G, Su J, Zhang J, Zhang M (2005) Exploring various knowledge in relation extraction. In: Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL), pp 427–434

  4. Mintz M, Bills S, Snow R, Jurafsky D (2009) Distant supervision for relation extraction without labeled data. In: Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP (ACL-AFNLP), pp 1003–1011

  5. Chan Y S, Roth D (2011) Exploiting syntactico-semantic structures for relation extraction. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics (ACL), pp 551–560

  6. Gormley M R, Yu M, Dredze M (2015) Improved relation extraction with feature-rich compositional embedding models. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp 1774–1784

  7. Yu X, Lam W (2010) Jointly identifying entities and extracting relations in encyclopedia text via a graphical model approach. In: Proceedings of the 23rd International Conference on Computational Linguistics (COLING), pp 1399–1407

  8. Li Q, Ji H (2014) Incremental joint extraction of entity mentions and relations. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (ACL), pp 402–412

  9. Miwa M, Sasaki Y (2014) Modeling joint entity and relation extraction with table representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp 1858–1869

  10. Ren X, Wu Z, He W, Qu M, Voss C R, Ji H, Abdelzaher T F, Han J (2017) Cotype: Joint extraction of typed entities and relations with knowledge bases. In: Proceedings of the 26th International Conference on World Wide Web (WWW), pp 1015–1024

  11. Gupta P, Schütze H, Andrassy B (2016) Table filling multi-task recurrent neural network for joint entity and relation extraction. In: Proceedings of the 26th International Conference on Computational Linguistics (COLING), pp 2537–2547

  12. Miwa M, Bansal M (2016) End-to-end relation extraction using lstms on sequences and tree structures. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL), pp 1105–1116

  13. Katiyar A, Cardie C (2017) Going out on a limb: Joint extraction of entity mentions and relations without dependency trees. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL), pp 917–928

  14. Zheng S, Wang F, Bao H, Hao Y, Zhou P, Xu B (2017) Joint extraction of entities and relations based on a novel tagging scheme. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL), pp 1227–1236

  15. Zeng X, Zeng D, He S, Liu K, Zhao J (2018) Extracting relational facts by an end-to-end neural model with copy mechanism. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL), pp 506–514

  16. Zeng X, He S, Zeng D, Liu K, Liu S, Zhao J (2019) Learning the extraction order of multiple relational facts in a sentence with reinforcement learning. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp 367–377

  17. Zeng D, Zhang H, Liu Q (2020) Copymtl: Copy mechanism for joint extraction of entities and relations with multi-task learning.. In: Proceedings of the 34th AAAI Conference on Artificial Intelligence (AAAI), pp 9507–9514

  18. Fu T-J, Li P-H, Ma W-Y (2019) Graphrel: Modeling text as relational graphs for joint entity and relation extraction. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL), pp 1409–1418

  19. Nayak T, Ng H T (2020) Effective modeling of encoder-decoder architecture for joint entity and relation extraction. In: Proceedings of the 34th AAAI Conference on Artificial Intelligence (AAAI), pp 8528–8535

  20. Wei Z, Su J, Wang Y, Tian Y, Chang Y (2020) A novel cascade binary tagging framework for relational triple extraction. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL), pp 1476–1488

  21. Devlin J, Chang M-W, Lee K, Toutanova K (2019) Bert: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL), pp 4171–4186

  22. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A N, Kaiser L, Polosukhin I (2017) Attention is all you need. In: Advances in neural information processing systems, pp 5998–6008

  23. Liu J, Chen S, Wang B, Zhang J, Li N, Xu T (2020) Attention as relation: Learning supervised multi-head self-attention for relation extraction. In: Proceedings of the 29th International Joint Conference on Artificial Intelligence (IJCAI)

  24. Yu B, Zhang Z, Shu X, Wang Y, Liu T, Wang B, Li S (2020) Joint extraction of entities and relations based on a novel decomposition strategy. In: Proceedings of the 24th European Conference on Artificial Intelligence (ECAI)

  25. Yuan Y, Zhou X, Pan S, Zhu Q, Song Z, Guo L (2020) A relation-specific attention network for joint entity and relation extraction. In: Proceedings of the 29th International Joint Conference on Artificial Intelligence (IJCAI), pp 4054–4060

  26. Ma X, Hovy E (2016) End-to-end sequence labeling via bi-directional lstm-cnns-crf. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL), pp 1064–1074

  27. Seyler D, Dembelova T, Del Corro L, Hoffart J, Weikum G (2018) A study of the importance of external knowledge in the named entity recognition task. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL), pp 241–246

  28. Lin J C-W, Shao Y, Zhou Y, Pirouz M, Chen H-C (2019) A bi-lstm mention hypergraph model with encoding schema for mention extraction. Eng Appl Artif Intell 85:175–181

    Article  Google Scholar 

  29. Lin J C-W, Shao Y, Fournier-Viger P, Hamido F (2019) Bilu-nemh: A bilu neural-encoded mention hypergraph for mention extraction. Inf Sci 496:53–64

    Article  Google Scholar 

  30. Li X, Feng J, Meng Y, Han Q, Wu F, Li J (2020) A unified mrc framework for named entity recognition. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL), pp 5849–5859

  31. Lin Y, Yang S, Stoyanov V, Ji H (2018) A multi-lingual multi-task architecture for low-resource sequence labeling. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL), pp 799–809

  32. Liu Y, Meng F, Zhang J, Xu J, Chen Y, Zhou J (2019) Gcdt: A global context enhanced deep transition architecture for sequence labeling. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL), pp 2431–2441

  33. Lin J C-W, Shao Y, Zhang J, Yun U (2020) Enhanced sequence labeling based on latent variable conditional random fields. Neurocomputing 403:431–440

    Article  Google Scholar 

  34. Chen L, Ruan W, Liu X, Lu J (2020) Seqvat: Virtual adversarial training for semi-supervised sequence labeling. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL), pp 8801–8811

  35. Lin J C-W, Shao Y, Djenouri Y, Yun U (2021) Asrnn: a recurrent neural network with an attention model for sequence labeling. Knowl-Based Syst 212:106548

    Article  Google Scholar 

  36. Riedel S, Yao L, McCallum A (2010) Modeling relations and their mentions without labeled text. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases (ECML PKDD). Springer, pp 148–163

  37. Gardent C, Shimorina A, Narayan S, Perez-Beltrachini L (2017) Creating training corpora for nlg micro-planners. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL), pp 179–188

  38. [bert-based,cased]. https://storage.googleapis.com/bert_models/2018_10_18/cased_L-12_H-768_A-12.zip

Download references

Author information

Authors and Affiliations

  1. College of Electronic Engineering, National University of Defense Technology, Hefei, People’s Republic of China

    Xiang Li, Yuwei Li, Junan Yang, Hui Liu & Pengjiang Hu

Authors
  1. Xiang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuwei Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Junan Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hui Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Pengjiang Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Yuwei Li or Junan Yang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, X., Li, Y., Yang, J. et al. A relation aware embedding mechanism for relation extraction. Appl Intell 52, 10022–10031 (2022). https://doi.org/10.1007/s10489-021-02699-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02699-3

Keywords

Search

Navigation