Skip to main content

Advertisement

BHRAM: a knowledge graph embedding model based on bidirectional and heterogeneous relational attention mechanism

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Knowledge graph embedding (KGE) is a method designed to predict missing relations between entities in a knowledge graph (KG), which has garnered much attention in recent years due to the incompleteness of KGs. However, existing KGE models have limitations in dealing with heterogeneous KGs and relation direction prediction. To address this issue, a novel KGE model called BHRAM is proposed. It is based on a bidirectional and heterogeneous relational attention mechanism. Specifically, BHRAM comprises three primary components, namely entity aggregation, relation aggregation and triplet prediction. The entity aggregation module divides the adjacency matrix into original and reverse relation adjacency matrices, using graph convolution to aggregate node features and subsequently form entity embedding representations. The relation aggregation module leverages bidirectional relations for feature extraction, learns the weight information of diverse paths independently and generates embedding representations of relation paths through an aggregation function. Finally, the triplet prediction module utilizes a score function for probabilistic predictions. To validate the superiority of BHRAM, comprehensive experiments were conducted on four well-known datasets, including baseline comparisons, relation classification tasks and ablation study. The results demonstrate that BHRAM significantly outperforms the other baselines on the FB15k-237, Kinship and UMLS datasets, while achieving similar or better performance than the baselines on the WN18RR dataset. These findings indicate that BHRAM can serve as a robust and effective model for addressing the heterogeneity in KGs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data Availability

The data presented in this study are available upon reasonable request from the corresponding author.

References

  1. Berners-Lee T, Hendler J, Lassila O (2001, May 1). The semantic web. Online. Retrieved May 1, 2001

  2. Singhal (2012). Official google blog: introducing the knowledge graph: things, not strings. Google

  3. Miller GA (1995) WordNet: a lexical database for English. Commun ACM 38(11):39–41. https://doi.org/10.1145/219717.219748

    Article  MATH  Google Scholar 

  4. Auer S, Bizer C, Kobilarov G, Lehmann J, Cyganiak R, Lves Z (2007) DBpedia: a nucleus for a web of open data. Lect Note Comput Sci 4825:722–735. https://doi.org/10.1007/978-3-540-76298-052

    Article  Google Scholar 

  5. Suchanek, F. M., Kasneci, G., & Weikum, G. (2007). YAGO: a core of semantic knowledge unifying wordnet and Wikipedia. Proceedings of the 16th international conference on World Wide Web, (pp 697-706). https://doi.org/10.1145/1242572.1242667

  6. Bollacker K, Evans C, Paritosh P, Sturge T, Taylor J (2008). Freebase: a collaboratively created graph database for structuring human knowledge. Proceedings of the 2008 ACM SIGMOD international conference on Management of data, (pp 1247-1250). https://doi.org/10.1145/1376616.1376746

  7. Vrandecic D, Krtoetzsch M (2014) Wikidata: a free collaborative knowledgebase. Commun ACM 57(10):78–85. https://doi.org/10.1145/2629489

    Article  Google Scholar 

  8. Zhang JC, Zain AM, Zhou KQ, Chen X, Zhang RM (2024) A review of recommender systems based on knowledge graph embedding. Exp Syst Appl 250. https://doi.org/10.1016/j.eswa.2024.123876

  9. Kim S, Rawat A S, Zaheer M, Jayasumana S, Sadhanala V, Jitkrittum W, Menon A K, Fergus R, Kumar S (2023). EmbedDistill: a geometric knowledge distillation for information retrieval. Arxiv. https://doi.org/10.48550/arXiv.2301.12005

  10. Wang Y, Lipka N, Rossi RA, Siu A, Zhang R, Derr T (2024) Knowledge graph prompting for multi-document question answering. Proceed the AAAI Conf Art Intell 38(17):19206–19214. https://doi.org/10.1609/aaai.v38i17.29889

    Article  Google Scholar 

  11. Ji S, Pan S, Cambria E, Marttinen P, Yu PS (2022) A survey on knowledge graphs: representation, acquisition, and applications. IEEE Trans Neural Netw Learn Syst 33(2):494–514. https://doi.org/10.1109/TNNLS.2021.3070843

    Article  MathSciNet  Google Scholar 

  12. Qian W, Fu C, Zhu Y, Cai D, He X (2018). Translating embeddings for knowledge graph completion with relation attention mechanism. In: IJCAI’18 - Proceedings of the 27th International Joint Conference on Artificial Intelligence, (pp 4286-4292). https://doi.org/10.5555/3304222.3304366

  13. Shou Z, Chen Y, Wen H, Liu J, Mo J, Zhang H (2023) A knowledge concept recommendation model based on tensor decomposition and transformer reordering. Electron 12(7):1593. https://doi.org/10.3390/electronics12071593

    Article  MATH  Google Scholar 

  14. Zhang D, Feng W, Wu Z, Li G, Ning B (2024) CDRGN-SDE: Cross-dimensional recurrent graph network with neural stochastic differential equation for temporal knowledge graph embedding. Exp Syst Appl 247. https://doi.org/10.1016/j.eswa.2024.123295

  15. Scarselli F, Tsoi AC, Gori M, Hagenbuchner M (2004) Graphical-based learning environments for pattern recognition. Lect Note Comput Sci 3138:42–56. https://doi.org/10.1007/978-3-540-27868-9_4

    Article  MATH  Google Scholar 

  16. Scarselli F, Gori M, Tsoi AC, Hagenbuchner M, Monfardini G (2009) The graph neural network model. IEEE Trans Neural Netw 20(1):61–80. https://doi.org/10.1109/TNN.2008.2005605

    Article  MATH  Google Scholar 

  17. Ye R, Li X, Fang Y, Zang H, Wang M (2019). A vectorized relational graph convolutional network for multi-relational network alignment. In: IJCAI’19 - Proceedings of the 28th International Joint Conference on Artificial Intelligence, (pp 4135-4141). https://doi.org/10.5555/3367471.3367616

  18. Zhao Y, Qi J, Liu Q, Zhang R (2021). WGCN: graph convolutional networks with weighted structural features. In SIGIR ’21 - Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, (pp 624-633). https://doi.org/10.1145/3404835.3462834

  19. Schlichtkrull M, Kipf T N, Bloem P, Berg R, Titov I, Welling M (2018). Modeling relational data with graph convolutional networks. The Semantic Web: 15th International Conference, (pp 593-607). https://doi.org/10.1007/978-3-319-93417-438

  20. Veličković P, Cucurull G, Casanova A, Romero A, Lio P, Bengio Y (2017). Graph attention networks. Arxiv. https://doi.org/10.48550/arXiv.1710.10903

  21. Zhao Y, Zhou H, Zhang A, Xie R, Li Q, Zhuang F (2023) Connecting embeddings based on multiplex relational graph attention networks for knowledge graph entity typing. IEEE Trans Knowl Data Eng 35(5):4608–4620. https://doi.org/10.1109/TKDE.2022.3142056

    Article  MATH  Google Scholar 

  22. Wang W, Suo X, Wei X, Wang B, Wang H, Dai HN, Zhang X (2023) HGATE: Heterogeneous graph attention auto-encoders. IEEE Trans Knowl Data Eng 35(4):3938–3951. https://doi.org/10.1109/TKDE.2021.3138788

    Article  MATH  Google Scholar 

  23. Hou Y, Zhang J, Cheng J, Ma K, Ma R T B, Chen H (2020). Measuring and improving the use of graph information in graph neural networks. In: ICLR - International Conference on Learning Representations. https://doi.org/10.48550/arXiv.2206.13170

  24. Prakash SKA, Tucker CS (2021) Node classification using kernel propagation in graph neural networks. Exp Syst Appl 174. https://doi.org/10.1016/j.eswa.2021.114655

  25. Bruna J, Zaremba W, Szlam A, LeCun Y (2014). Spectral networks and deep locally connected networks on graphs. In: ICLR - International Conference on Learning Representations. https://doi.org/10.48550/arXiv.1312.6203

  26. Yu B, Xie H, Cai M, Ding W (2023) MG-GCN: multi-granularity graph convolutional neural network for multi-label classification in multi-label information system. IEEE Trans Emerg Topics in Comput Intell 8(1):1–12. https://doi.org/10.1109/TETCI.2023.3300303

    Article  MATH  Google Scholar 

  27. Feng F, He X, Zhang H, Chua TS (2023) Cross-GCN: enhancing graph convolutional network with kk-Order feature interactions. IEEE Trans Knowl Data Eng 35(1):225–236. https://doi.org/10.1109/TKDE.2021.3077524

    Article  Google Scholar 

  28. Zeb A, Haq AU, Zhang D, Chen J, Gong Z (2021) KGEL: a novel end-to-end embedding learning framework for knowledge graph completion. Exp Syst Appl 167. https://doi.org/10.1016/j.eswa.2020.114164

  29. Chen J, Ma T, Xiao C (2018). FastGCN: fast learning with graph convolutional networks via importance sampling. Arxiv. https://doi.org/10.48550/arXiv.1801.10247

  30. Wu L, Wang D, Song K, Feng S, Zhang Y, Yu G (2021) Dual-view hypergraph neural networks for attributed graph learning. Knowl-Based Syst 227. https://doi.org/10.1016/j.knosys.2021.107185

  31. Yang X, Yan M, Pan S, Ye X, Fan D (2023). Simple and efficient heterogeneous graph neural network. Proceed AAAI Conf Art Intell, (pp10816-10824). https://doi.org/10.48550/arXiv.2207.02547

  32. Xie B, Ma X, Wu J, Yang J, Fan H (2024) Knowledge graph enhanced heterogeneous graph neural network for fake news detection. IEEE Trans Consume Electron 70(1):2826–2837. https://doi.org/10.1109/TCE.2023.3324661

  33. Zhao L, Li J, Gao L, Rao Y, Song J, Shen H T (2023) Heterogeneous knowledge network for visual dialog. IEEE Trans Circ Syst Video Technol 33(2):861–871. https://doi.org/10.1109/TCSVT.2022.3207228

  34. Ma T, Huang L, Lu Q, Hu S (2023) KR-GCN: knowledge-aware reasoning with graph convolution network for explainable recommendation. ACM Trans Inf Syst 41(1):1–27. https://doi.org/10.1145/3511019

  35. Li Q, Wang Y, Dong J, Zhang C, Peng K (2024) Multi-node knowledge graph assisted distributed fault detection for large-scale industrial processes based on graph attention network and bidirectional LSTMs. Neural Netw 173. https://doi.org/10.1016/j.neunet.2024.106210

  36. Li Z, Liu X, Wang X, Liu P, Shen Y (2023) TransO: a knowledge-driven representation learning method with ontology information constraints. World Wide Web 26(1):297–319. https://doi.org/10.1007/s11280-022-01016-3

    Article  MATH  Google Scholar 

  37. Wang C, Wang X, Li Z, Chen Z, Li J (2023). HyConvE: a novel embedding model for knowledge hypergraph link prediction with convolutional neural networks," In: WWW ’23 - Proceedings of the ACM Web Conference 2023, (pp 188-198). https://doi.org/10.1145/3543507.3583256

  38. Mohamed HA, Pilutti D, James S, Bue AD, Pelillo M, Vascon S (2023) Locality-aware subgraphs for inductive link prediction in knowledge graphs. Pattern Recogn Lett 167:90–97. https://doi.org/10.1016/j.patrec.2023.02.004

    Article  Google Scholar 

  39. Wang X, Luo Z, He R, Shao Y (2023). Novel medical question and answer system: graph convolutional neural network based with knowledge graph optimization. Exp Syst Appl, 227(C). https://doi.org/10.1016/j.eswa.2023.120211

  40. Fang Y, Deng J, Zhang F, Wang H (2022) An intelligent question-answering model over educational knowledge graph for sustainable urban living. Sustain 15(2):1139. https://doi.org/10.3390/su15021139

    Article  MATH  Google Scholar 

  41. Bordes A, Usunier U, Garcia-Duran A, Weston J, Yakhnenko O (2013). Translating embeddings for modeling multi-relational data. In NIPS’13 - Proceedings of the 26th International Conference on Neural Information Processing Systems, 2, (pp2787-2795). https://doi.org/10.5555/2999792.2999923

  42. Lin Y, Liu Z, Sun M, Liu Y, Zhu X (2015). Learning entity and relation embeddings for knowledge graph completion. In: AAAI’15 - Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, (pp 2181-2187). https://doi.org/10.5555/2886521.2886624

  43. Ji G, He S, Xu L, Liu K, Zhao J (2015). Knowledge graph embedding via dynamic mapping matrix. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, 1, (pp 687-696). https://doi.org/10.3115/v1/P15-1067

  44. Wang Z, Zhang J, Feng J, Chen Z (2014). Knowledge graph embedding by translating on hyperplanes. In: AAAI’14 - Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, (pp 1112-1119). https://doi.org/10.5555/2893873.2894046

  45. Sun Z, Deng Z H, Nie, J Y, Tang J (2019). RotatE: knowledge graph embedding by relational rotation in complex space. Arxiv. https://doi.org/10.48550/arXiv.1902.10197

  46. Li J, Su X, Zhang F, Gao G (2023). TransERR: translation-based knowledge graph completion via efficient relation rotation. Arxiv. https://doi.org/10.48550/arXiv.2306.14580

  47. Yang B, Yih S W, He X, Gao J, Deng L (2015). Embedding entities and relations for learning and inference in knowledge bases. Proceed Int Conf Learn Represent. https://doi.org/10.48550/arXiv.1412.6575

  48. Trouillon T, Welbl J, Riedel S, Gaussier E, Bouchard G (2016). Complex embeddings for simple link prediction. In: ICML’16 - Proceedings of the 33rd International Conference on International Conference on Machine Learning, 48, (pp 2071-2080). https://doi.org/10.5555/3045390.3045609

  49. Dettmers T, Minervini P, Stenetorp P, Riedel S (2018). Convolutional 2d knowledge graph embeddings. In: AAAI’18/IAAI’18/EAAI’18 - Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and Eighth AAAI Symposium on Educational Advances in Artificial Intelligence, (pp 1811-1818). https://doi.org/10.5555/3504035.3504256

  50. Nguyen D Q, Nguyen T D, Nguyen D Q, Phung D (2018). A novel embedding model for knowledge base completion based on convolutional neural network. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2, (pp 327-333). https://doi.org/10.18653/v1/N18-2053

  51. Jiang X, Wang Q, Wang B (2019). Adaptive convolution for multi-relational learning. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technol, 1, (pp 978-987). https://doi.org/10.18653/v1/N19-1103

  52. Fang H, Wang Y, Tian Z, Ye Y (2023) Learning knowledge graph embedding with a dual-attention embedding network. Exp Syst Appl 212. https://doi.org/10.1016/j.eswa.2022.118806

  53. Liang S, Shao J, Zhang D, Zhang J, Cui B (2023) DRGI: deep relational graph infomax for knowledge graph completion. IEEE Trans Knowl Data Eng 35(3):2486–2499. https://doi.org/10.1109/TKDE.2021.3110898

    Article  MATH  Google Scholar 

  54. Jiang D, Wang R, Xue L, Yang J (2024). Multisource hierarchical neural network for knowledge graph embedding. Exp Syst Appl, 237(PB). https://doi.org/10.1016/j.eswa.2023.121446

  55. Zhang Z, Cai J, Zhang Y, Wang J (2020). Learning hierarchy-aware knowledge graph embeddings for link prediction. Findings of the Association for Computational Linguistics: EMNLP 2021, (pp 3065-3072). https://doi.org/10.1609/aaai.v34i03.5701

  56. Ebisu T, Ichise R (2020) Generalized translation-based embedding of knowledge graph. IEEE Trans Knowl Data Eng 32(5):941–951. https://doi.org/10.1109/TKDE.2019.2893920

    Article  MATH  Google Scholar 

  57. Vashishth S, Sanyal S, Nitin V, Agrawal N, Talukdar P (2020). Interacte: improving convolution-based knowledge graph embeddings by increasing feature interactions. Proceed the AAAI Conf Art Intell, 34, (pp 3009-3016). https://doi.org/10.1609/aaai.v34i03.5694

  58. Vashishth S, Sanyal S, Nitin V, Talukdar P (2019). Composition-based multirelational graph convolutional networks. Arxiv. https://doi.org/10.48550/arXiv.1911.03082

  59. Shang C, Tang Y, Huang J, Bi J, He X, Zhou B (2019) End-to-end structure-aware convolutional networks for knowledge base completion. Proceed the AAAI Conf Art Intell 33(1):3060–3067. https://doi.org/10.1609/aaai.v33i01.33013060

  60. Zhang Z, Li Z, Liu H, Xiong NN (2022) Multi-scale dynamic convolutional network for knowledge graph embedding. IEEE Trans Knowl Data Eng 34(5):2335–2347. https://doi.org/10.1109/TKDE.2020.3005952

    Article  MATH  Google Scholar 

  61. Li Z, Liu H, Zhang Z, Liu T, Xiong NN (2022) Learning knowledge graph embedding with heterogeneous relation attention networks. IEEE Trans Neural Netw Learn Syst 33(8):3961–3973. https://doi.org/10.1109/TNNLS.2021.3055147

    Article  MathSciNet  MATH  Google Scholar 

  62. Zhang D, Rong Z, Xue, C., & Li, G. (2024) SimRE: simple contrastive learning with soft logical rule for knowledge graph embedding. Inf Sci 661. https://doi.org/10.1016/j.ins.2023.120069

  63. Bi Z, Cheng S, Chen J, Liang X, Xiong F, Zhang N (2024) Relphormer: relational graph transformer for knowledge graph representations. Neurocomput 566. https://doi.org/10.1016/j.neucom.2023.127044

  64. Zhu Y, Shimodaira H (2024). Block-diagonal orthogonal relation and matrix entity for knowledge graph embedding. Arxiv. https://doi.org/10.48550/arXiv.2401.05967

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China under Grant No. 62062011 and by the Innovation Project of Guangxi Minzu University Graduate Education under Grant Nos. gxun-chxs2024117 and gxun-chxs2024115. The authors would like to thank the editors and the anonymous reviewers for their kind assistance, constructive comments and recommendations, which have significantly improved the presentation of this paper. We would like to express our appreciation to those who share the datasets used in this paper.

Funding

This research is supported by the National Natural Science Foundation of China under Grant No. 62062011. and the Innovation Project of Guangxi Minzu University Graduate Education under Grant Nos. gxun-chxs2024117 and gxun-chxs2024115.

Author information

Authors and Affiliations

Authors

Contributions

All authors contributed to the study conception and design. Material preparation was performed by Haoran Li and Zhilin Zeng. Data collection and analysis were performed by Chaoqun Zhang and Wanqiu Li. The first draft of the manuscript was written by Wanqiu Li. The paper revision and review were completed by Chaoqun Zhang and Yuanbin Mo. Funded by Weidong Tang, Wanqiu Li and Zhilin Zeng. All authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Wanqiu Li.

Ethics declarations

Conflict of Interest

The authors declare that they have no known competing financial interests or personal relationships.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, C., Li, W., Mo, Y. et al. BHRAM: a knowledge graph embedding model based on bidirectional and heterogeneous relational attention mechanism. Appl Intell 55, 245 (2025). https://doi.org/10.1007/s10489-024-06212-4

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10489-024-06212-4

Keywords