Skip to main content
SpringerLink
Log in

Relation-attention semantic-correlative knowledge graph embedding for inductive link prediction

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Link prediction has increasingly been the focus of significant research interest, benefited from the explosion of machine learning and deep learning techniques. Graph embedding has been proven to be an effective method for predicting missing links in graph-based structure. In this work, we propose a novel relation-attention semantic-correlative graph embedding for inductive link prediction. Unlike existing embedding-based methods, we shift the node representation learning from a node’s perspective to a relational subgraph perspective. Our model has a better inductive bias to learn entity-independent relational semantics. We consider two kinds of relational subgraph topology for a given entity pair: relational correlation subgraph and relational path subgraph. Firstly, we capture the structure of neighboring relation-properties of semantic-missing entity by relational correlation subgraph. Secondly, we capture the set of relational paths between given entity pair by relational path subgraph. Finally, we organize the above two modules in a unified framework for relation prediction. Our ablation experiments show that two kinds of relational subgraph topology are important for relation prediction. Experimental results on six benchmark datasets demonstrate that our proposed graph embedding outperforms existing state-of-the-art models for link prediction tasks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data availability

The data that support the findings of this study are openly available in github at https://github.com/Jasminelxn/RASC.

References

  1. Hwang JD, Bhagavatula C, Bras RL, Da J, Sakaguchi K, Bosselut A, Choi Y (2020) COMET-ATOMIC 2020: on symbolic and neural commonsense knowledge Graphs arXiv:2010.05953

  2. Xun G, Jha K, Yuan Y, Zhang A (2019) Topic discovery for biomedical corpus using mesh embeddings. 2019 IEEE EMBS International Conference on Biomedical and Health Informatics, BHI 2019 - Proceedings, 2–5 . https://doi.org/10.1109/BHI.2019.8834559

  3. Bosselut A, Bras RL, Choi Y (2019) Dynamic Neuro-Symbolic Knowledge Graph Construction for Zero-shot Commonsense Question Answering arXiv:1911.03876

  4. Li J, Sun A, Guan Z, Cheema MA, Min G (2022) Real-time dynamic network learning for location inference modelling and computing. Neurocomputing 472:198–200. https://doi.org/10.1016/j.neucom.2021.10.086

    Article  Google Scholar 

  5. Kotnis B, Lawrence C, Niepert M (2020) Answering complex queries in knowledge graphs with bidirectional sequence encoders arXiv:2004.02596

  6. Zhang M, Zhang R, Zou L, Lin Y, Hu S (2021) NAMER: a node-based multitasking framework for multi-hop knowledge base question answering. In: NAACL-HLT 2021 - Proceedings of the 2021 annual conference of the North American chapter of the association for computational linguistics. Human language technologies: demonstrations. pp 18–25. https://doi.org/10.18653/v1/2021.naacl-demos.3

  7. Liu D, Lian J, Liu Z, Wang X, Sun G, Xie X (2021) Reinforced Anchor Knowledge Graph Generation for News Recommendation Reasoning vol. 1, pp. 1055–1065. Association for Computing Machinery . https://doi.org/10.1145/3447548.3467315

  8. Liu H, Tong Y, Han J, Zhang P, Lu X, Xiong H (2022) Incorporating Multi-Source Urban Data for Personalized and Context-Aware Multi-Modal Transportation Recommendation. IEEE Transact Knowl Data Eng 34(2):723–735. https://doi.org/10.1109/TKDE.2020.2985954

    Article  Google Scholar 

  9. Hu X, Xu J, Wang W, Li Z, Liu A (2021) A graph embedding based model for fine-grained POI recommendation. Neurocomputing 428:376–384. https://doi.org/10.1016/j.neucom.2020.01.118

    Article  Google Scholar 

  10. Wu S, Zhang Y, Gao C, Bian K, Cui B (2020) GARG: Anonymous Recommendation of Point-of-Interest in Mobile Networks by Graph Convolution Network. Data Sci Eng 5(4):433–447. https://doi.org/10.1007/s41019-020-00135-z

    Article  Google Scholar 

  11. Praznik L, Srivastava G, Mendhe C, Mago V (2019) Vertex-weighted measures for link prediction in hashtag graphs. Proceedings of the 2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, ASONAM 2019, 1034–1041. https://doi.org/10.1145/3341161.3344828

  12. Berahmand K, Nasiri E, Forouzandeh S, Li Y (2022) A preference random walk algorithm for link prediction through mutual influence nodes in complex networks. J King Saud Univ Comput Inform Sci 34(8), 5375–5387 arXiv:2105.09494. https://doi.org/10.1016/j.jksuci.2021.05.006

  13. Khanam KZ, Srivastava G, Mago V (2022) The homophily principle in social network analysis: a survey. Multimedia Tools Appl. https://doi.org/10.1007/s11042-021-11857-1

    Article  Google Scholar 

  14. Amin S, Varanasi S, Dunfield KA, Neumann G (2020) LowFER: Low-rank bilinear pooling for link prediction. 37th International Conference on Machine Learning, ICML 2020 PartF168147-1, 234–245 arXiv:2008.10858

  15. Peng Y, Zhang J (2020) LineaRE: Simple but powerful knowledge graph embedding for link prediction. Proceedings - IEEE International Conference on Data Mining, ICDM 2020-November(Icdm), 422–431 arXiv:2004.10037. https://doi.org/10.1109/ICDM50108.2020.00051

  16. Balažević I, Allen C, Hospedales TM (2020) Tucker: Tensor factorization for knowledge graph completion. EMNLP-IJCNLP 2019 - 2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing, Proceedings of the Conference, 5185–5194 (2020) arXiv:1901.09590. https://doi.org/10.18653/v1/d19-1522

  17. Hao Y, Cao X, Fang Y, Xie X, Wang S (2020) Inductive link prediction for nodes having only attribute information. IJCAI International Joint Conference on Artificial Intelligence 2021-January, 1209–1215 arXiv:2007.08053. https://doi.org/10.24963/ijcai.2020/168

  18. Daza D, Cochez M, Groth P (2021) Inductive entity representations from text via link prediction. The Web Conference 2021 - Proceedings of the World Wide Web Conference, WWW 2021, 798–808 arXiv:2010.03496. https://doi.org/10.1145/3442381.3450141

  19. Wang P, Agarwal K, Ham C, Choudhury S, Reddy CK (2021) Self-supervised learning of contextual embeddings for link prediction in heterogeneous networks. The Web Conference 2021 - Proceedings of the World Wide Web Conference, WWW 2021, 2946–2957 arXiv:2007.11192. https://doi.org/10.1145/3442381.3450060

  20. Sadeghian A, Armandpour M, Ding P, Wang DZ (2019) DRUM: End-to-end differentiable rule mining on knowledge graphs. Adv Neural Inform Process Syst 32arXiv:1911.00055

  21. Peng Y, Choi B, Xu J (2021) Graph Learning for Combinatorial Optimization: A Survey of State-of-the-Art. Data Sci Eng 6(2), 119–141 arXiv:2008.12646. https://doi.org/10.1007/s41019-021-00155-3

  22. Li Z, Wang X, Li J, Zhang Q (2021) Deep attributed network representation learning of complex coupling and interaction. Knowl-Based Syst 212:106618. https://doi.org/10.1016/j.knosys.2020.106618

    Article  Google Scholar 

  23. Sun Z, Deng ZH, Nie JY, Tang J (2019) Rotate: Knowledge graph embedding by relational rotation in complex space. 7th International Conference on Learning Representations, ICLR 2019 arXiv:1902.10197

  24. Rosso P, Yang D, Cudré-Mauroux P (2019) Knowledge Graph Embeddings. Encyclopedia Big Data Technol 1:1073–1080. https://doi.org/10.1007/978-3-319-77525-8_284

    Article  Google Scholar 

  25. Cai L, Yan B, Mai G, Janowicz K, Zhu R (2019) TransGCN: Coupling transformation assumptions with graph convolutional networks for link prediction. K-CAP 2019 - Proceedings of the 10th International Conference on Knowledge Capture, 131–138 arXiv:1910.00702. https://doi.org/10.1145/3360901.3364441

  26. Banan A, Nasiri A, Taheri-Garavand A (2020) Deep learning-based appearance features extraction for automated carp species identification. Aquacult Eng 89:102053. https://doi.org/10.1016/j.aquaeng.2020.102053

    Article  Google Scholar 

  27. Fan Y, Xu K, Wu H, Zheng Y, Tao B (2020) Spatiotemporal Modeling for Nonlinear Distributed Thermal Processes Based on KL Decomposition. MLP and LSTM Network. IEEE Access 8:25111–25121. https://doi.org/10.1109/ACCESS.2020.2970836

    Article  Google Scholar 

  28. Lin H, Gharehbaghi A, Zhang Q, Band SS, Pai HT, Chau KW, Mosavi A (2022) Time series-based groundwater level forecasting using gated recurrent unit deep neural networks. Eng Appl Comput Fluid Mech 16(1):1655–1672. https://doi.org/10.1080/19942060.2022.2104928

    Article  Google Scholar 

  29. Wang X, Wang S, Xin Y, Yang Y, Li J, Wang X (2020) Distributed Pregel-based provenance-aware regular path query processing on RDF knowledge graphs. World Wide Web 23(3):1465–1496. https://doi.org/10.1007/s11280-019-00739-0

    Article  Google Scholar 

  30. Anirban S, Wang J, Islam MS, Kayesh H, Li J, Huang ML (2022) Compression techniques for 2-hop labeling for shortest distance queries. World Wide Web 25(1):151–174. https://doi.org/10.1007/s11280-021-00977-1

    Article  Google Scholar 

  31. Yang F, Yang Z, Cohen WW (2017) Differentiable learning of logical rules for knowledge base reasoning. Adv Neural Inform Proces Syst 2017-December(Nips), 2320–2329 arXiv:1702.08367

  32. Jaderberg M, Mnih V, Czarnecki WM, Schaul T, Leibo JZ, Silver D, Kavukcuoglu K (2017) Reinforcement learning with unsupervised auxiliary tasks. 5th International Conference on Learning Representations, ICLR 2017 - Conference Track Proceedings, 1–9 arXiv:1611.05397

  33. Omran PG, Wang K, Wang Z (2018) Scalable rule learning via learning representation. IJCAI International Joint Conference on Artificial Intelligence 2018-July, 2149–2155 . https://doi.org/10.24963/ijcai.2018/297

  34. Hamilton WL, Bajaj P, Zitnik M, Jurafsky D, Leskovec J (2018) Embedding logical queries on knowledge graphs. Advances in Neural Information Processing Systems 2018-December(NeurIPS), 2026–2037 arXiv:1806.01445

  35. Lin Y, Liu Z, Luan H, Sun M, Rao S, Liu S (2015) Modeling relation paths for representation learning of knowledge bases. Conference Proceedings - EMNLP 2015: Conference on Empirical Methods in Natural Language Processing, 705–714 arXiv:1506.00379. https://doi.org/10.18653/v1/d15-1082

  36. Zhang F, Wang X, Li Z, Li J (2020) TransRHS: A representation learning method for knowledge graphs with relation hierarchical structure. IJCAI International Joint Conference on Artificial Intelligence 2021-January(1), 2987–2993 . https://doi.org/10.24963/ijcai.2020/413

  37. Teru KK, Denis EG, Hamilton WL (2020) Inductive relation prediction by subgraph reasoning. 37th International Conference on Machine Learning, ICML 2020 PartF168147-13(1), 9390–9399 arXiv:1911.06962

  38. Zhang Z, Zhuang F, Zhu H, Shi Z, Xiong H, He Q (2020) Relational graph neural network with hierarchical attention for knowledge graph completion. AAAI 2020 - 34th AAAI Conference on Artificial Intelligence, 9612–9619. https://doi.org/10.1609/aaai.v34i05.6508

  39. Wang H, Ren H, Leskovec J (2021) Relational Message Passing for Knowledge Graph Completion. Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 1697–1707 arXiv:2002.06757. https://doi.org/10.1145/3447548.3467247

  40. Chen J, He H, Wu F, Wang J (2021) Topology-Aware Correlations Between Relations for Inductive Link Prediction in Knowledge Graphs. AAAI arXiv:2103.03642

  41. Shi C, Li Y, Zhang J, Sun Y, Yu PS (2017) A Survey of Heterogeneous Information Network Analysis. IEEE Transactions on Knowledge and Data Engineering 29(1), 17–37 arXiv:1511.04854. https://doi.org/10.1109/TKDE.2016.2598561

  42. Fu X, Zhang J, Meng Z, King I (2020) MAGNN: Metapath Aggregated Graph Neural Network for Heterogeneous Graph Embedding. The Web Conference 2020 - Proceedings of the World Wide Web Conference, WWW 2020, 2331–2341 arXiv:2002.01680. https://doi.org/10.1145/3366423.3380297

  43. Yang Y, Guan Z, Li J, Zhao W, Cui J, Wang Q (2021) Interpretable and Efficient Heterogeneous Graph Convolutional Network. IEEE Transactions on Knowledge and Data Engineering XX(Xx), 1–13 arXiv:2005.13183. https://doi.org/10.1109/TKDE.2021.3101356

Download references

Author information

Author notes

  1. Li Xiaonan, Li Guanyu and Wang Jie have contributed equally to this work.

Authors and Affiliations

  1. Faculty of Information Science and Technology, Dalian Maritime University, Lingshui, Dalian, 116026, Liaoning, China

    Li Xiaonan, Ning Bo, Li Guanyu & Wang Jie

Authors
  1. Li Xiaonan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ning Bo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Li Guanyu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wang Jie
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ning Bo.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xiaonan, L., Bo, N., Guanyu, L. et al. Relation-attention semantic-correlative knowledge graph embedding for inductive link prediction. Int. J. Mach. Learn. & Cyber. 14, 3799–3811 (2023). https://doi.org/10.1007/s13042-023-01865-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-023-01865-y

Keywords

Search

Navigation