Abstract
Knowledge graph embedding (KGE) is a method designed to predict missing relations between entities in a knowledge graph (KG), which has garnered much attention in recent years due to the incompleteness of KGs. However, existing KGE models have limitations in dealing with heterogeneous KGs and relation direction prediction. To address this issue, a novel KGE model called BHRAM is proposed. It is based on a bidirectional and heterogeneous relational attention mechanism. Specifically, BHRAM comprises three primary components, namely entity aggregation, relation aggregation and triplet prediction. The entity aggregation module divides the adjacency matrix into original and reverse relation adjacency matrices, using graph convolution to aggregate node features and subsequently form entity embedding representations. The relation aggregation module leverages bidirectional relations for feature extraction, learns the weight information of diverse paths independently and generates embedding representations of relation paths through an aggregation function. Finally, the triplet prediction module utilizes a score function for probabilistic predictions. To validate the superiority of BHRAM, comprehensive experiments were conducted on four well-known datasets, including baseline comparisons, relation classification tasks and ablation study. The results demonstrate that BHRAM significantly outperforms the other baselines on the FB15k-237, Kinship and UMLS datasets, while achieving similar or better performance than the baselines on the WN18RR dataset. These findings indicate that BHRAM can serve as a robust and effective model for addressing the heterogeneity in KGs.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data Availability
The data presented in this study are available upon reasonable request from the corresponding author.
References
Berners-Lee T, Hendler J, Lassila O (2001, May 1). The semantic web. Online. Retrieved May 1, 2001
Singhal (2012). Official google blog: introducing the knowledge graph: things, not strings. Google
Miller GA (1995) WordNet: a lexical database for English. Commun ACM 38(11):39–41. https://doi.org/10.1145/219717.219748
Auer S, Bizer C, Kobilarov G, Lehmann J, Cyganiak R, Lves Z (2007) DBpedia: a nucleus for a web of open data. Lect Note Comput Sci 4825:722–735. https://doi.org/10.1007/978-3-540-76298-052
Suchanek, F. M., Kasneci, G., & Weikum, G. (2007). YAGO: a core of semantic knowledge unifying wordnet and Wikipedia. Proceedings of the 16th international conference on World Wide Web, (pp 697-706). https://doi.org/10.1145/1242572.1242667
Bollacker K, Evans C, Paritosh P, Sturge T, Taylor J (2008). Freebase: a collaboratively created graph database for structuring human knowledge. Proceedings of the 2008 ACM SIGMOD international conference on Management of data, (pp 1247-1250). https://doi.org/10.1145/1376616.1376746
Vrandecic D, Krtoetzsch M (2014) Wikidata: a free collaborative knowledgebase. Commun ACM 57(10):78–85. https://doi.org/10.1145/2629489
Zhang JC, Zain AM, Zhou KQ, Chen X, Zhang RM (2024) A review of recommender systems based on knowledge graph embedding. Exp Syst Appl 250. https://doi.org/10.1016/j.eswa.2024.123876
Kim S, Rawat A S, Zaheer M, Jayasumana S, Sadhanala V, Jitkrittum W, Menon A K, Fergus R, Kumar S (2023). EmbedDistill: a geometric knowledge distillation for information retrieval. Arxiv. https://doi.org/10.48550/arXiv.2301.12005
Wang Y, Lipka N, Rossi RA, Siu A, Zhang R, Derr T (2024) Knowledge graph prompting for multi-document question answering. Proceed the AAAI Conf Art Intell 38(17):19206–19214. https://doi.org/10.1609/aaai.v38i17.29889
Ji S, Pan S, Cambria E, Marttinen P, Yu PS (2022) A survey on knowledge graphs: representation, acquisition, and applications. IEEE Trans Neural Netw Learn Syst 33(2):494–514. https://doi.org/10.1109/TNNLS.2021.3070843
Qian W, Fu C, Zhu Y, Cai D, He X (2018). Translating embeddings for knowledge graph completion with relation attention mechanism. In: IJCAI’18 - Proceedings of the 27th International Joint Conference on Artificial Intelligence, (pp 4286-4292). https://doi.org/10.5555/3304222.3304366
Shou Z, Chen Y, Wen H, Liu J, Mo J, Zhang H (2023) A knowledge concept recommendation model based on tensor decomposition and transformer reordering. Electron 12(7):1593. https://doi.org/10.3390/electronics12071593
Zhang D, Feng W, Wu Z, Li G, Ning B (2024) CDRGN-SDE: Cross-dimensional recurrent graph network with neural stochastic differential equation for temporal knowledge graph embedding. Exp Syst Appl 247. https://doi.org/10.1016/j.eswa.2024.123295
Scarselli F, Tsoi AC, Gori M, Hagenbuchner M (2004) Graphical-based learning environments for pattern recognition. Lect Note Comput Sci 3138:42–56. https://doi.org/10.1007/978-3-540-27868-9_4
Scarselli F, Gori M, Tsoi AC, Hagenbuchner M, Monfardini G (2009) The graph neural network model. IEEE Trans Neural Netw 20(1):61–80. https://doi.org/10.1109/TNN.2008.2005605
Ye R, Li X, Fang Y, Zang H, Wang M (2019). A vectorized relational graph convolutional network for multi-relational network alignment. In: IJCAI’19 - Proceedings of the 28th International Joint Conference on Artificial Intelligence, (pp 4135-4141). https://doi.org/10.5555/3367471.3367616
Zhao Y, Qi J, Liu Q, Zhang R (2021). WGCN: graph convolutional networks with weighted structural features. In SIGIR ’21 - Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, (pp 624-633). https://doi.org/10.1145/3404835.3462834
Schlichtkrull M, Kipf T N, Bloem P, Berg R, Titov I, Welling M (2018). Modeling relational data with graph convolutional networks. The Semantic Web: 15th International Conference, (pp 593-607). https://doi.org/10.1007/978-3-319-93417-438
Veličković P, Cucurull G, Casanova A, Romero A, Lio P, Bengio Y (2017). Graph attention networks. Arxiv. https://doi.org/10.48550/arXiv.1710.10903
Zhao Y, Zhou H, Zhang A, Xie R, Li Q, Zhuang F (2023) Connecting embeddings based on multiplex relational graph attention networks for knowledge graph entity typing. IEEE Trans Knowl Data Eng 35(5):4608–4620. https://doi.org/10.1109/TKDE.2022.3142056
Wang W, Suo X, Wei X, Wang B, Wang H, Dai HN, Zhang X (2023) HGATE: Heterogeneous graph attention auto-encoders. IEEE Trans Knowl Data Eng 35(4):3938–3951. https://doi.org/10.1109/TKDE.2021.3138788
Hou Y, Zhang J, Cheng J, Ma K, Ma R T B, Chen H (2020). Measuring and improving the use of graph information in graph neural networks. In: ICLR - International Conference on Learning Representations. https://doi.org/10.48550/arXiv.2206.13170
Prakash SKA, Tucker CS (2021) Node classification using kernel propagation in graph neural networks. Exp Syst Appl 174. https://doi.org/10.1016/j.eswa.2021.114655
Bruna J, Zaremba W, Szlam A, LeCun Y (2014). Spectral networks and deep locally connected networks on graphs. In: ICLR - International Conference on Learning Representations. https://doi.org/10.48550/arXiv.1312.6203
Yu B, Xie H, Cai M, Ding W (2023) MG-GCN: multi-granularity graph convolutional neural network for multi-label classification in multi-label information system. IEEE Trans Emerg Topics in Comput Intell 8(1):1–12. https://doi.org/10.1109/TETCI.2023.3300303
Feng F, He X, Zhang H, Chua TS (2023) Cross-GCN: enhancing graph convolutional network with kk-Order feature interactions. IEEE Trans Knowl Data Eng 35(1):225–236. https://doi.org/10.1109/TKDE.2021.3077524
Zeb A, Haq AU, Zhang D, Chen J, Gong Z (2021) KGEL: a novel end-to-end embedding learning framework for knowledge graph completion. Exp Syst Appl 167. https://doi.org/10.1016/j.eswa.2020.114164
Chen J, Ma T, Xiao C (2018). FastGCN: fast learning with graph convolutional networks via importance sampling. Arxiv. https://doi.org/10.48550/arXiv.1801.10247
Wu L, Wang D, Song K, Feng S, Zhang Y, Yu G (2021) Dual-view hypergraph neural networks for attributed graph learning. Knowl-Based Syst 227. https://doi.org/10.1016/j.knosys.2021.107185
Yang X, Yan M, Pan S, Ye X, Fan D (2023). Simple and efficient heterogeneous graph neural network. Proceed AAAI Conf Art Intell, (pp10816-10824). https://doi.org/10.48550/arXiv.2207.02547
Xie B, Ma X, Wu J, Yang J, Fan H (2024) Knowledge graph enhanced heterogeneous graph neural network for fake news detection. IEEE Trans Consume Electron 70(1):2826–2837. https://doi.org/10.1109/TCE.2023.3324661
Zhao L, Li J, Gao L, Rao Y, Song J, Shen H T (2023) Heterogeneous knowledge network for visual dialog. IEEE Trans Circ Syst Video Technol 33(2):861–871. https://doi.org/10.1109/TCSVT.2022.3207228
Ma T, Huang L, Lu Q, Hu S (2023) KR-GCN: knowledge-aware reasoning with graph convolution network for explainable recommendation. ACM Trans Inf Syst 41(1):1–27. https://doi.org/10.1145/3511019
Li Q, Wang Y, Dong J, Zhang C, Peng K (2024) Multi-node knowledge graph assisted distributed fault detection for large-scale industrial processes based on graph attention network and bidirectional LSTMs. Neural Netw 173. https://doi.org/10.1016/j.neunet.2024.106210
Li Z, Liu X, Wang X, Liu P, Shen Y (2023) TransO: a knowledge-driven representation learning method with ontology information constraints. World Wide Web 26(1):297–319. https://doi.org/10.1007/s11280-022-01016-3
Wang C, Wang X, Li Z, Chen Z, Li J (2023). HyConvE: a novel embedding model for knowledge hypergraph link prediction with convolutional neural networks," In: WWW ’23 - Proceedings of the ACM Web Conference 2023, (pp 188-198). https://doi.org/10.1145/3543507.3583256
Mohamed HA, Pilutti D, James S, Bue AD, Pelillo M, Vascon S (2023) Locality-aware subgraphs for inductive link prediction in knowledge graphs. Pattern Recogn Lett 167:90–97. https://doi.org/10.1016/j.patrec.2023.02.004
Wang X, Luo Z, He R, Shao Y (2023). Novel medical question and answer system: graph convolutional neural network based with knowledge graph optimization. Exp Syst Appl, 227(C). https://doi.org/10.1016/j.eswa.2023.120211
Fang Y, Deng J, Zhang F, Wang H (2022) An intelligent question-answering model over educational knowledge graph for sustainable urban living. Sustain 15(2):1139. https://doi.org/10.3390/su15021139
Bordes A, Usunier U, Garcia-Duran A, Weston J, Yakhnenko O (2013). Translating embeddings for modeling multi-relational data. In NIPS’13 - Proceedings of the 26th International Conference on Neural Information Processing Systems, 2, (pp2787-2795). https://doi.org/10.5555/2999792.2999923
Lin Y, Liu Z, Sun M, Liu Y, Zhu X (2015). Learning entity and relation embeddings for knowledge graph completion. In: AAAI’15 - Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, (pp 2181-2187). https://doi.org/10.5555/2886521.2886624
Ji G, He S, Xu L, Liu K, Zhao J (2015). Knowledge graph embedding via dynamic mapping matrix. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, 1, (pp 687-696). https://doi.org/10.3115/v1/P15-1067
Wang Z, Zhang J, Feng J, Chen Z (2014). Knowledge graph embedding by translating on hyperplanes. In: AAAI’14 - Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, (pp 1112-1119). https://doi.org/10.5555/2893873.2894046
Sun Z, Deng Z H, Nie, J Y, Tang J (2019). RotatE: knowledge graph embedding by relational rotation in complex space. Arxiv. https://doi.org/10.48550/arXiv.1902.10197
Li J, Su X, Zhang F, Gao G (2023). TransERR: translation-based knowledge graph completion via efficient relation rotation. Arxiv. https://doi.org/10.48550/arXiv.2306.14580
Yang B, Yih S W, He X, Gao J, Deng L (2015). Embedding entities and relations for learning and inference in knowledge bases. Proceed Int Conf Learn Represent. https://doi.org/10.48550/arXiv.1412.6575
Trouillon T, Welbl J, Riedel S, Gaussier E, Bouchard G (2016). Complex embeddings for simple link prediction. In: ICML’16 - Proceedings of the 33rd International Conference on International Conference on Machine Learning, 48, (pp 2071-2080). https://doi.org/10.5555/3045390.3045609
Dettmers T, Minervini P, Stenetorp P, Riedel S (2018). Convolutional 2d knowledge graph embeddings. In: AAAI’18/IAAI’18/EAAI’18 - Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and Eighth AAAI Symposium on Educational Advances in Artificial Intelligence, (pp 1811-1818). https://doi.org/10.5555/3504035.3504256
Nguyen D Q, Nguyen T D, Nguyen D Q, Phung D (2018). A novel embedding model for knowledge base completion based on convolutional neural network. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2, (pp 327-333). https://doi.org/10.18653/v1/N18-2053
Jiang X, Wang Q, Wang B (2019). Adaptive convolution for multi-relational learning. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technol, 1, (pp 978-987). https://doi.org/10.18653/v1/N19-1103
Fang H, Wang Y, Tian Z, Ye Y (2023) Learning knowledge graph embedding with a dual-attention embedding network. Exp Syst Appl 212. https://doi.org/10.1016/j.eswa.2022.118806
Liang S, Shao J, Zhang D, Zhang J, Cui B (2023) DRGI: deep relational graph infomax for knowledge graph completion. IEEE Trans Knowl Data Eng 35(3):2486–2499. https://doi.org/10.1109/TKDE.2021.3110898
Jiang D, Wang R, Xue L, Yang J (2024). Multisource hierarchical neural network for knowledge graph embedding. Exp Syst Appl, 237(PB). https://doi.org/10.1016/j.eswa.2023.121446
Zhang Z, Cai J, Zhang Y, Wang J (2020). Learning hierarchy-aware knowledge graph embeddings for link prediction. Findings of the Association for Computational Linguistics: EMNLP 2021, (pp 3065-3072). https://doi.org/10.1609/aaai.v34i03.5701
Ebisu T, Ichise R (2020) Generalized translation-based embedding of knowledge graph. IEEE Trans Knowl Data Eng 32(5):941–951. https://doi.org/10.1109/TKDE.2019.2893920
Vashishth S, Sanyal S, Nitin V, Agrawal N, Talukdar P (2020). Interacte: improving convolution-based knowledge graph embeddings by increasing feature interactions. Proceed the AAAI Conf Art Intell, 34, (pp 3009-3016). https://doi.org/10.1609/aaai.v34i03.5694
Vashishth S, Sanyal S, Nitin V, Talukdar P (2019). Composition-based multirelational graph convolutional networks. Arxiv. https://doi.org/10.48550/arXiv.1911.03082
Shang C, Tang Y, Huang J, Bi J, He X, Zhou B (2019) End-to-end structure-aware convolutional networks for knowledge base completion. Proceed the AAAI Conf Art Intell 33(1):3060–3067. https://doi.org/10.1609/aaai.v33i01.33013060
Zhang Z, Li Z, Liu H, Xiong NN (2022) Multi-scale dynamic convolutional network for knowledge graph embedding. IEEE Trans Knowl Data Eng 34(5):2335–2347. https://doi.org/10.1109/TKDE.2020.3005952
Li Z, Liu H, Zhang Z, Liu T, Xiong NN (2022) Learning knowledge graph embedding with heterogeneous relation attention networks. IEEE Trans Neural Netw Learn Syst 33(8):3961–3973. https://doi.org/10.1109/TNNLS.2021.3055147
Zhang D, Rong Z, Xue, C., & Li, G. (2024) SimRE: simple contrastive learning with soft logical rule for knowledge graph embedding. Inf Sci 661. https://doi.org/10.1016/j.ins.2023.120069
Bi Z, Cheng S, Chen J, Liang X, Xiong F, Zhang N (2024) Relphormer: relational graph transformer for knowledge graph representations. Neurocomput 566. https://doi.org/10.1016/j.neucom.2023.127044
Zhu Y, Shimodaira H (2024). Block-diagonal orthogonal relation and matrix entity for knowledge graph embedding. Arxiv. https://doi.org/10.48550/arXiv.2401.05967
Acknowledgements
This work is supported by the National Natural Science Foundation of China under Grant No. 62062011 and by the Innovation Project of Guangxi Minzu University Graduate Education under Grant Nos. gxun-chxs2024117 and gxun-chxs2024115. The authors would like to thank the editors and the anonymous reviewers for their kind assistance, constructive comments and recommendations, which have significantly improved the presentation of this paper. We would like to express our appreciation to those who share the datasets used in this paper.
Funding
This research is supported by the National Natural Science Foundation of China under Grant No. 62062011. and the Innovation Project of Guangxi Minzu University Graduate Education under Grant Nos. gxun-chxs2024117 and gxun-chxs2024115.
Author information
Authors and Affiliations
Contributions
All authors contributed to the study conception and design. Material preparation was performed by Haoran Li and Zhilin Zeng. Data collection and analysis were performed by Chaoqun Zhang and Wanqiu Li. The first draft of the manuscript was written by Wanqiu Li. The paper revision and review were completed by Chaoqun Zhang and Yuanbin Mo. Funded by Weidong Tang, Wanqiu Li and Zhilin Zeng. All authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Conflict of Interest
The authors declare that they have no known competing financial interests or personal relationships.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Zhang, C., Li, W., Mo, Y. et al. BHRAM: a knowledge graph embedding model based on bidirectional and heterogeneous relational attention mechanism. Appl Intell 55, 245 (2025). https://doi.org/10.1007/s10489-024-06212-4
Accepted:
Published:
DOI: https://doi.org/10.1007/s10489-024-06212-4