Abstract
Relation prediction aims to infer the missing relations among entities in knowledge graphs, where inductive relation prediction enjoys great popularity due to its effectiveness to be applied to emerging entities. Most existing approaches learn the logical compositional rules or utilize subgraphs to predict the missing relation. Although great progress has been made in the performance, current models are still suboptimal due to their limited ability to capture topological information that is critical for local relation prediction. To address this problem, we propose a novel inductive relation prediction approach called substructure-aware subgraph reasoning which incorporates the substructure information of subgraphs into the reasoning process, thus making the relation prediction more precise. Specifically, we extract the entities and relations around the target entities to form the subgraph and then encode the structure information of nodes and edges by counting the number of certain substructures. Next, the structural information is explicitly applied to the message passing for more accurate reasoning. To improve the performance, we also utilize the semantic correlations between relations as auxiliary information. Experimental results on three benchmark datasets show the effectiveness of the proposed approach for the inductive relation prediction.
Similar content being viewed by others
Data availibility
The datasets generated and/or analyzed during the current study are available in https://github.com/moguizhizi/SAGIL.
Code availability
The code implemented during the current study is available in https://github.com/moguizhizi/SAGIL.
References
Zhang Y, Dai H, Kozareva Z, Smola AJ, Song L (2017) Variational reasoning for question answering with knowledge graph
Zhang Z, Han X, Liu Z, Jiang X, Sun M, Liu Q (2019) Ernie: enhanced language representation with informative entities. arXiv preprint arXiv:1905.07129
Wang H, Zhang F, Wang J, Zhao M, Li W, Xie X, Guo M (2018) Ripplenet: propagating user preferences on the knowledge graph for recommender systems. In: Proceedings of the 27th ACM International Conference on Information and Knowledge Management, pp 417–426
Zhao J-Z, Zhang X, Gao C, Li Z-D, Wang B-L (2022) Kg2lib: knowledge-graph-based convolutional network for third-party library recommendation. J Supercomput 79:1–26
Bollacker K, Evans C, Paritosh P, Sturge T, Taylor J (2008) Freebase: a collaboratively created graph database for structuring human knowledge. In: Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, pp 1247–1250
Auer S, Bizer C, Kobilarov G, Lehmann J, Cyganiak R, Ives Z (2007) DBpedia: a nucleus for a web of open data. In: The Semantic Web. Springer, pp 722–735
Bordes A, Usunier N, Garcia-Duran A, Weston J, Yakhnenko O (2013) Translating embeddings for modeling multi-relational data. Adv Neural Inf Process Syst 26
Dettmers T, Minervini P, Stenetorp P, Riedel S (2018) Convolutional 2d knowledge graph embeddings. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 32
Yang B, Yih W-t, He X, Gao J, Deng L (2014) Embedding entities and relations for learning and inference in knowledge bases. arXiv preprint arXiv:1412.6575
Trivedi R, Dai H, Wang Y, Song L (2017) Know-evolve: deep temporal reasoning for dynamic knowledge graphs. In: International Conference on Machine Learning. PMLR, pp 3462–3471
Hamaguchi T, Oiwa H, Shimbo M, Matsumoto Y (2017) Knowledge transfer for out-of-knowledge-base entities: a graph neural network approach. arXiv preprint arXiv:1706.05674
Albooyeh M, Goel R, Kazemi SM (2020) Out-of-sample representation learning for knowledge graphs. In: Findings of the Association for Computational Linguistics: EMNLP 2020, pp 2657–2666
Defferrard M, Bresson X, Vandergheynst P (2016) Convolutional neural networks on graphs with fast localized spectral filtering. Adv Neural Inf Process Syst 29
Kipf TN, Welling M (2016) Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907
Teru K, Denis E, Hamilton W (2020) Inductive relation prediction by subgraph reasoning. In: International Conference on Machine Learning. PMLR, pp 9448–9457
Chen J, He H, Wu F, Wang J (2021) Topology-aware correlations between relations for inductive link prediction in knowledge graphs. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 35, pp 6271–6278
Arvind V, Fuhlbrück F, Köbler J, Verbitsky O (2020) On Weisfeiler–Leman invariance: subgraph counts and related graph properties. J Comput Syst Sci 113:42–59
Chen Z, Chen L, Villar S, Bruna J (2020) Can graph neural networks count substructures? Adv Neural Inf Process Syst 33:10383–10395
Nayyeri M, Vahdati S, Aykul C, Lehmann J (2021) 5* knowledge graph embeddings with projective transformations. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 35, pp 9064–9072
Zhou X, Niu L, Zhu Q, Zhu X, Liu P, Tan J, Guo L (2021) Knowledge graph embedding by double limit scoring loss. IEEE Trans Knowl Data Eng 34:5825–5839
Rossi A, Barbosa D, Firmani D, Matinata A, Merialdo P (2021) Knowledge graph embedding for link prediction: a comparative analysis. ACM Trans Knowl Discov Data (TKDD) 15(2):1–49
Wang J, Wang B, Gao J, Hu Y, Yin B (2022) Multi-concept representation learning for knowledge graph completion. ACM Trans Knowl Discov Data (TKDD) 17:1–19
Wang Z, Zhang J, Feng J, Chen Z (2014) Knowledge graph embedding by translating on hyperplanes. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 28
Lin Y, Liu Z, Sun M, Liu Y, Zhu X (2015) Learning entity and relation embeddings for knowledge graph completion. In: Twenty-ninth AAAI Conference on Artificial Intelligence
Suo X, Guo B, Shen Y, Dai S, Wang W, Chen Y, Zhang Z (2022) Krl_match: knowledge graph objects matching for knowledge representation learning. Knowl Inf Syst 65:1–41
Nickel M, Tresp V, Kriegel H-P (2011) A three-way model for collective learning on multi-relational data. In: ICML
Trouillon T, Welbl J, Riedel S, Gaussier É, Bouchard G (2016) Complex embeddings for simple link prediction. In: International Conference on Machine Learning. PMLR, pp 2071–2080
Nguyen DQ, Nguyen TD, Nguyen DQ, Phung D (2017) A novel embedding model for knowledge base completion based on convolutional neural network. arXiv preprint arXiv:1712.02121
Schlichtkrull M, Kipf TN, Bloem P, Berg R, Titov I, Welling M (2018) Modeling relational data with graph convolutional networks. In: European Semantic Web Conference. Springer, pp 593–607
Vashishth S, Sanyal S, Nitin V, Talukdar P (2019) Composition-based multi-relational graph convolutional networks. arXiv preprint arXiv:1911.03082
Shi B, Weninger T (2018) Open-world knowledge graph completion. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 32
Daza D, Cochez M, Groth P (2021) Inductive entity representations from text via link prediction. In: Proceedings of the Web Conference 2021, pp 798–808
Xie R, Liu Z, Luan H, Sun M (2016) Image-embodied knowledge representation learning. arXiv preprint arXiv:1609.07028
Hamilton W, Ying Z, Leskovec J (2017) Inductive representation learning on large graphs. Adv Neural Inf Process Syst 30
Zheng C, Zong B, Cheng W, Song D, Ni J, Yu W, Chen H, Wang W (2020) Robust graph representation learning via neural sparsification. In: International Conference on Machine Learning. PMLR, pp 11458–11468
Wang P, Han J, Li C, Pan R (2019) Logic attention based neighborhood aggregation for inductive knowledge graph embedding. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 33, pp 7152–7159
He Y, Wang Z, Zhang P, Tu Z, Ren Z (2020) Vn network: embedding newly emerging entities with virtual neighbors. In: Proceedings of the 29th ACM International Conference on Information and Knowledge Management, pp 505–514
Bhowmik R, de Melo G (2020) Explainable link prediction for emerging entities in knowledge graphs. In: International Semantic Web Conference. Springer, pp 39–55
Mohamed HA, Pilutti D, James S, Del Bue A, Pelillo M, Vascon S (2023) Locality-aware subgraphs for inductive link prediction in knowledge graphs. Pattern Recognit Lett 167:90–97
Lin Q, Liu J, Xu F, Pan Y, Zhu Y, Zhang L, Zhao T (2022) Incorporating context graph with logical reasoning for inductive relation prediction. In: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp 893–903
Galárraga L, Teflioudi C, Hose K, Suchanek FM (2015) Fast rule mining in ontological knowledge bases with AMIE+. VLDB J 24(6):707–730
Yang F, Yang Z, Cohen WW (2017) Differentiable learning of logical rules for knowledge base reasoning. Adv Neural Inf Process Syst 30
Meilicke C, Fink M, Wang Y, Ruffinelli D, Gemulla R, Stuckenschmidt H (2018) Fine-grained evaluation of rule-and embedding-based systems for knowledge graph completion. In: International Semantic Web Conference. Springer, pp 3–20
Sadeghian A, Armandpour M, Ding P, Wang DZ (2019) Drum: end-to-end differentiable rule mining on knowledge graphs. Adv Neural Inf Process Syst 32
Fürer M (2017) On the combinatorial power of the Weisfeiler–Lehman algorithm. In: International Conference on Algorithms and Complexity. Springer, pp 260–271
Tahmasebi B, Lim D, Jegelka S (2020) Counting substructures with higher-order graph neural networks: possibility and impossibility results. arXiv preprint arXiv:2012.03174
Gilmer J, Schoenholz SS, Riley PF, Vinyals O, Dahl GE (2017) Neural message passing for quantum chemistry. In: International Conference on Machine Learning. PMLR, pp 1263–1272
Paszke A, Gross S, Chintala S, Chanan G, Yang E, DeVito Z, Lin Z, Desmaison A, Antiga L, Lerer A (2017) Automatic differentiation in pytorch
Toutanova K, Chen D, Pantel P, Poon H, Choudhury P, Gamon M (2015) Representing text for joint embedding of text and knowledge bases. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp 1499–1509
Xiong W, Hoang T, Wang WY (2017) Deeppath: a reinforcement learning method for knowledge graph reasoning. arXiv preprint arXiv:1707.06690
Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980
Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning. PMLR, pp 448–456
Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958
Xu X, Zhang P, He Y, Chao C, Yan C (2022) Subgraph neighboring relations infomax for inductive link prediction on knowledge graphs. arXiv preprint arXiv:2208.00850
Acknowledgements
This work was supported in part by the National Natural Science Foundation of China under Grant U21B2038, Grant U1811463, Grant U19B2039, and Grant 62206007 and in part by the National Key R and D Program of China under Grant No. 2021ZD0111902.
Author information
Authors and Affiliations
Contributions
Related work was carried out by all the authors. The implementation of the proposal and experiments was carried out by KS. HJ and KS drafted, revised, and approved the manuscript, respectively.
Corresponding author
Ethics declarations
Conflict of interest
The authors have no competing interests to declare that are relevant to the content of this article.
Consent for publication
Not applicable.
Consent to participate
Not applicable.
Ethics approval
Not applicable.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Sun, K., Jiang, H., Hu, Y. et al. Substructure-aware subgraph reasoning for inductive relation prediction. J Supercomput 79, 21008–21027 (2023). https://doi.org/10.1007/s11227-023-05493-9
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11227-023-05493-9