Skip to main content
SpringerLink
Log in

Substructure-aware subgraph reasoning for inductive relation prediction

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Relation prediction aims to infer the missing relations among entities in knowledge graphs, where inductive relation prediction enjoys great popularity due to its effectiveness to be applied to emerging entities. Most existing approaches learn the logical compositional rules or utilize subgraphs to predict the missing relation. Although great progress has been made in the performance, current models are still suboptimal due to their limited ability to capture topological information that is critical for local relation prediction. To address this problem, we propose a novel inductive relation prediction approach called substructure-aware subgraph reasoning which incorporates the substructure information of subgraphs into the reasoning process, thus making the relation prediction more precise. Specifically, we extract the entities and relations around the target entities to form the subgraph and then encode the structure information of nodes and edges by counting the number of certain substructures. Next, the structural information is explicitly applied to the message passing for more accurate reasoning. To improve the performance, we also utilize the semantic correlations between relations as auxiliary information. Experimental results on three benchmark datasets show the effectiveness of the proposed approach for the inductive relation prediction.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availibility

The datasets generated and/or analyzed during the current study are available in https://github.com/moguizhizi/SAGIL.

Code availability

The code implemented during the current study is available in https://github.com/moguizhizi/SAGIL.

References

  1. Zhang Y, Dai H, Kozareva Z, Smola AJ, Song L (2017) Variational reasoning for question answering with knowledge graph

  2. Zhang Z, Han X, Liu Z, Jiang X, Sun M, Liu Q (2019) Ernie: enhanced language representation with informative entities. arXiv preprint arXiv:1905.07129

  3. Wang H, Zhang F, Wang J, Zhao M, Li W, Xie X, Guo M (2018) Ripplenet: propagating user preferences on the knowledge graph for recommender systems. In: Proceedings of the 27th ACM International Conference on Information and Knowledge Management, pp 417–426

  4. Zhao J-Z, Zhang X, Gao C, Li Z-D, Wang B-L (2022) Kg2lib: knowledge-graph-based convolutional network for third-party library recommendation. J Supercomput 79:1–26

    Article  Google Scholar 

  5. Bollacker K, Evans C, Paritosh P, Sturge T, Taylor J (2008) Freebase: a collaboratively created graph database for structuring human knowledge. In: Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, pp 1247–1250

  6. Auer S, Bizer C, Kobilarov G, Lehmann J, Cyganiak R, Ives Z (2007) DBpedia: a nucleus for a web of open data. In: The Semantic Web. Springer, pp 722–735

  7. Bordes A, Usunier N, Garcia-Duran A, Weston J, Yakhnenko O (2013) Translating embeddings for modeling multi-relational data. Adv Neural Inf Process Syst 26

  8. Dettmers T, Minervini P, Stenetorp P, Riedel S (2018) Convolutional 2d knowledge graph embeddings. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 32

  9. Yang B, Yih W-t, He X, Gao J, Deng L (2014) Embedding entities and relations for learning and inference in knowledge bases. arXiv preprint arXiv:1412.6575

  10. Trivedi R, Dai H, Wang Y, Song L (2017) Know-evolve: deep temporal reasoning for dynamic knowledge graphs. In: International Conference on Machine Learning. PMLR, pp 3462–3471

  11. Hamaguchi T, Oiwa H, Shimbo M, Matsumoto Y (2017) Knowledge transfer for out-of-knowledge-base entities: a graph neural network approach. arXiv preprint arXiv:1706.05674

  12. Albooyeh M, Goel R, Kazemi SM (2020) Out-of-sample representation learning for knowledge graphs. In: Findings of the Association for Computational Linguistics: EMNLP 2020, pp 2657–2666

  13. Defferrard M, Bresson X, Vandergheynst P (2016) Convolutional neural networks on graphs with fast localized spectral filtering. Adv Neural Inf Process Syst 29

  14. Kipf TN, Welling M (2016) Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907

  15. Teru K, Denis E, Hamilton W (2020) Inductive relation prediction by subgraph reasoning. In: International Conference on Machine Learning. PMLR, pp 9448–9457

  16. Chen J, He H, Wu F, Wang J (2021) Topology-aware correlations between relations for inductive link prediction in knowledge graphs. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 35, pp 6271–6278

  17. Arvind V, Fuhlbrück F, Köbler J, Verbitsky O (2020) On Weisfeiler–Leman invariance: subgraph counts and related graph properties. J Comput Syst Sci 113:42–59

    Article  MathSciNet  MATH  Google Scholar 

  18. Chen Z, Chen L, Villar S, Bruna J (2020) Can graph neural networks count substructures? Adv Neural Inf Process Syst 33:10383–10395

    Google Scholar 

  19. Nayyeri M, Vahdati S, Aykul C, Lehmann J (2021) 5* knowledge graph embeddings with projective transformations. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 35, pp 9064–9072

  20. Zhou X, Niu L, Zhu Q, Zhu X, Liu P, Tan J, Guo L (2021) Knowledge graph embedding by double limit scoring loss. IEEE Trans Knowl Data Eng 34:5825–5839

    Article  Google Scholar 

  21. Rossi A, Barbosa D, Firmani D, Matinata A, Merialdo P (2021) Knowledge graph embedding for link prediction: a comparative analysis. ACM Trans Knowl Discov Data (TKDD) 15(2):1–49

    Article  Google Scholar 

  22. Wang J, Wang B, Gao J, Hu Y, Yin B (2022) Multi-concept representation learning for knowledge graph completion. ACM Trans Knowl Discov Data (TKDD) 17:1–19

    Google Scholar 

  23. Wang Z, Zhang J, Feng J, Chen Z (2014) Knowledge graph embedding by translating on hyperplanes. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 28

  24. Lin Y, Liu Z, Sun M, Liu Y, Zhu X (2015) Learning entity and relation embeddings for knowledge graph completion. In: Twenty-ninth AAAI Conference on Artificial Intelligence

  25. Suo X, Guo B, Shen Y, Dai S, Wang W, Chen Y, Zhang Z (2022) Krl_match: knowledge graph objects matching for knowledge representation learning. Knowl Inf Syst 65:1–41

    Google Scholar 

  26. Nickel M, Tresp V, Kriegel H-P (2011) A three-way model for collective learning on multi-relational data. In: ICML

  27. Trouillon T, Welbl J, Riedel S, Gaussier É, Bouchard G (2016) Complex embeddings for simple link prediction. In: International Conference on Machine Learning. PMLR, pp 2071–2080

  28. Nguyen DQ, Nguyen TD, Nguyen DQ, Phung D (2017) A novel embedding model for knowledge base completion based on convolutional neural network. arXiv preprint arXiv:1712.02121

  29. Schlichtkrull M, Kipf TN, Bloem P, Berg R, Titov I, Welling M (2018) Modeling relational data with graph convolutional networks. In: European Semantic Web Conference. Springer, pp 593–607

  30. Vashishth S, Sanyal S, Nitin V, Talukdar P (2019) Composition-based multi-relational graph convolutional networks. arXiv preprint arXiv:1911.03082

  31. Shi B, Weninger T (2018) Open-world knowledge graph completion. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 32

  32. Daza D, Cochez M, Groth P (2021) Inductive entity representations from text via link prediction. In: Proceedings of the Web Conference 2021, pp 798–808

  33. Xie R, Liu Z, Luan H, Sun M (2016) Image-embodied knowledge representation learning. arXiv preprint arXiv:1609.07028

  34. Hamilton W, Ying Z, Leskovec J (2017) Inductive representation learning on large graphs. Adv Neural Inf Process Syst 30

  35. Zheng C, Zong B, Cheng W, Song D, Ni J, Yu W, Chen H, Wang W (2020) Robust graph representation learning via neural sparsification. In: International Conference on Machine Learning. PMLR, pp 11458–11468

  36. Wang P, Han J, Li C, Pan R (2019) Logic attention based neighborhood aggregation for inductive knowledge graph embedding. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 33, pp 7152–7159

  37. He Y, Wang Z, Zhang P, Tu Z, Ren Z (2020) Vn network: embedding newly emerging entities with virtual neighbors. In: Proceedings of the 29th ACM International Conference on Information and Knowledge Management, pp 505–514

  38. Bhowmik R, de Melo G (2020) Explainable link prediction for emerging entities in knowledge graphs. In: International Semantic Web Conference. Springer, pp 39–55

  39. Mohamed HA, Pilutti D, James S, Del Bue A, Pelillo M, Vascon S (2023) Locality-aware subgraphs for inductive link prediction in knowledge graphs. Pattern Recognit Lett 167:90–97

    Article  Google Scholar 

  40. Lin Q, Liu J, Xu F, Pan Y, Zhu Y, Zhang L, Zhao T (2022) Incorporating context graph with logical reasoning for inductive relation prediction. In: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp 893–903

  41. Galárraga L, Teflioudi C, Hose K, Suchanek FM (2015) Fast rule mining in ontological knowledge bases with AMIE+. VLDB J 24(6):707–730

    Article  Google Scholar 

  42. Yang F, Yang Z, Cohen WW (2017) Differentiable learning of logical rules for knowledge base reasoning. Adv Neural Inf Process Syst 30

  43. Meilicke C, Fink M, Wang Y, Ruffinelli D, Gemulla R, Stuckenschmidt H (2018) Fine-grained evaluation of rule-and embedding-based systems for knowledge graph completion. In: International Semantic Web Conference. Springer, pp 3–20

  44. Sadeghian A, Armandpour M, Ding P, Wang DZ (2019) Drum: end-to-end differentiable rule mining on knowledge graphs. Adv Neural Inf Process Syst 32

  45. Fürer M (2017) On the combinatorial power of the Weisfeiler–Lehman algorithm. In: International Conference on Algorithms and Complexity. Springer, pp 260–271

  46. Tahmasebi B, Lim D, Jegelka S (2020) Counting substructures with higher-order graph neural networks: possibility and impossibility results. arXiv preprint arXiv:2012.03174

  47. Gilmer J, Schoenholz SS, Riley PF, Vinyals O, Dahl GE (2017) Neural message passing for quantum chemistry. In: International Conference on Machine Learning. PMLR, pp 1263–1272

  48. Paszke A, Gross S, Chintala S, Chanan G, Yang E, DeVito Z, Lin Z, Desmaison A, Antiga L, Lerer A (2017) Automatic differentiation in pytorch

  49. Toutanova K, Chen D, Pantel P, Poon H, Choudhury P, Gamon M (2015) Representing text for joint embedding of text and knowledge bases. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp 1499–1509

  50. Xiong W, Hoang T, Wang WY (2017) Deeppath: a reinforcement learning method for knowledge graph reasoning. arXiv preprint arXiv:1707.06690

  51. Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980

  52. Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning. PMLR, pp 448–456

  53. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958

    MathSciNet  MATH  Google Scholar 

  54. Xu X, Zhang P, He Y, Chao C, Yan C (2022) Subgraph neighboring relations infomax for inductive link prediction on knowledge graphs. arXiv preprint arXiv:2208.00850

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China under Grant U21B2038, Grant U1811463, Grant U19B2039, and Grant 62206007 and in part by the National Key R and D Program of China under Grant No. 2021ZD0111902.

Author information

Authors and Affiliations

  1. Beijing Institute of Artificial Intelligence, Beijing University of Technology, Bijing, 100010, China

    Kai Sun, HuaJie Jiang, Yongli Hu & BaoCai Yin

Authors
  1. Kai Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. HuaJie Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yongli Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. BaoCai Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Related work was carried out by all the authors. The implementation of the proposal and experiments was carried out by KS. HJ and KS drafted, revised, and approved the manuscript, respectively.

Corresponding author

Correspondence to HuaJie Jiang.

Ethics declarations

Conflict of interest

The authors have no competing interests to declare that are relevant to the content of this article.

Consent for publication

Not applicable.

Consent to participate

Not applicable.

Ethics approval

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sun, K., Jiang, H., Hu, Y. et al. Substructure-aware subgraph reasoning for inductive relation prediction. J Supercomput 79, 21008–21027 (2023). https://doi.org/10.1007/s11227-023-05493-9

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-023-05493-9

Keywords

Search

Navigation