Skip to main content
Springer Nature Link
Log in

Towards Few-Shot Inductive Link Prediction on Knowledge Graphs: A Relational Anonymous Walk-Guided Neural Process Approach

  • Conference paper
  • First Online:
Machine Learning and Knowledge Discovery in Databases: Research Track (ECML PKDD 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14171))

  • 1555 Accesses

Abstract

Few-shot inductive link prediction on knowledge graphs (KGs) aims to predict missing links for unseen entities with few-shot links observed. Previous methods are limited to transductive scenarios, where entities exist in the knowledge graphs, so they are unable to handle unseen entities. Therefore, recent inductive methods utilize the sub-graphs around unseen entities to obtain the semantics and predict links inductively. However, in the few-shot setting, the sub-graphs are often sparse and cannot provide meaningful inductive patterns. In this paper, we propose a novel relational anonymous walk-guided neural process for few-shot inductive link prediction on knowledge graphs, denoted as RawNP. Specifically, we develop a neural process-based method to model a flexible distribution over link prediction functions. This enables the model to quickly adapt to new entities and estimate the uncertainty when making predictions. To capture general inductive patterns, we present a relational anonymous walk to extract a series of relational motifs from few-shot observations. These motifs reveal the distinctive semantic patterns on KGs that support inductive predictions. Extensive experiments on typical benchmark datasets demonstrate that our model derives new state-of-the-art performance.

Z. Zhao and L. Luo—Equal contribution.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Baek, J., Lee, D.B., Hwang, S.J.: Learning to extrapolate knowledge: transductive few-shot out-of-graph link prediction. Adv. Neural. Inf. Process. Syst. 33, 546–560 (2020)

    Google Scholar 

  2. Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: Advances in Neural Information Processing Systems, vol. 26 (2013)

    Google Scholar 

  3. Brazdil, P., van Rijn, J.N., Gouk, H., Mohr, F.: Advances in metalearning: ECML/PKDD workshop on meta-knowledge transfer. In: ECML-PKDD Workshop on Meta-Knowledge Transfer, pp. 1–7. PMLR (2022)

    Google Scholar 

  4. Cangea, C., Day, B., Jamasb, A.R., Lio, P.: Message passing neural processes. In: ICLR 2022 Workshop on Geometrical and Topological Representation Learning (2022)

    Google Scholar 

  5. Chen, J., He, H., Wu, F., Wang, J.: Topology-aware correlations between relations for inductive link prediction in knowledge graphs. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 6271–6278 (2021)

    Google Scholar 

  6. Chen, M., Zhang, W., Zhang, W., Chen, Q., Chen, H.: Meta relational learning for few-shot link prediction in knowledge graphs. arXiv preprint arXiv:1909.01515 (2019)

  7. Chen, M., Zhang, W., Zhu, Y., Zhou, H., Yuan, Z., Xu, C., Chen, H.: Meta-knowledge transfer for inductive knowledge graph embedding. In: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 927–937 (2022)

    Google Scholar 

  8. Dong, M., Yuan, F., Yao, L., Xu, X., Zhu, L.: MAMO: memory-augmented meta-optimization for cold-start recommendation. In: Proceedings of the 26th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp. 688–697 (2020)

    Google Scholar 

  9. Fey, M., Lenssen, J.E.: Fast graph representation learning with pytorch geometric. arXiv preprint arXiv:1903.02428 (2019)

  10. Garnelo, M., et al.: Conditional neural processes. In: International Conference on Machine Learning, pp. 1704–1713. PMLR (2018)

    Google Scholar 

  11. Garnelo, M., et al.: Neural processes. arXiv preprint arXiv:1807.01622 (2018)

  12. Hamaguchi, T., Oiwa, H., Shimbo, M., Matsumoto, Y.: Knowledge transfer for out-of-knowledge-base entities: a graph neural network approach. arXiv preprint arXiv:1706.05674 (2017)

  13. Huang, Q., Ren, H., Leskovec, J.: Few-shot relational reasoning via connection subgraph pretraining. In: Advances in Neural Information Processing Systems (2022)

    Google Scholar 

  14. Jin, M., Li, Y.F., Pan, S.: Neural temporal walks: motif-aware representation learning on continuous-time dynamic graphs. In: Advances in Neural Information Processing Systems (2022)

    Google Scholar 

  15. Kim, H., et al.: Attentive neural processes. arXiv preprint arXiv:1901.05761 (2019)

  16. Kingma, D.P., Welling, M.: Auto-encoding variational Bayes. arXiv preprint arXiv:1312.6114 (2013)

  17. Liang, H., Gao, J.: How neural processes improve graph link prediction. In: ICASSP 2022–2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 3543–3547. IEEE (2022)

    Google Scholar 

  18. Lin, X., Wu, J., Zhou, C., Pan, S., Cao, Y., Wang, B.: Task-adaptive neural process for user cold-start recommendation. In: Proceedings of the Web Conference 2021, pp. 1306–1316 (2021)

    Google Scholar 

  19. Liu, S., Grau, B., Horrocks, I., Kostylev, E.: Indigo: GNN-based inductive knowledge graph completion using pair-wise encoding. In: Advances in Neural Information Processing Systems, pp. 2034–2045 (2021)

    Google Scholar 

  20. Luo, L., Fang, Y., Cao, X., Zhang, X., Zhang, W.: Detecting communities from heterogeneous graphs: a context path-based graph neural network model. In: Proceedings of the 30th ACM International Conference on Information & Knowledge Management, pp. 1170–1180 (2021)

    Google Scholar 

  21. Luo, L., Fang, Y., Lu, M., Cao, X., Zhang, X., Zhang, W.: GSim: a graph neural network based relevance measure for heterogeneous graphs. In: IEEE Trans. Knowl. Data Eng. (2023)

    Google Scholar 

  22. Luo, L., Haffari, G., Pan, S.: Graph sequential neural ode process for link prediction on dynamic and sparse graphs. In: Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining, pp. 778–786 (2023)

    Google Scholar 

  23. Luo, L., Li, Y.F., Haffari, G., Pan, S.: Normalizing flow-based neural process for few-shot knowledge graph completion (2023)

    Google Scholar 

  24. Mai, S., Zheng, S., Yang, Y., Hu, H.: Communicative message passing for inductive relation reasoning. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 4294–4302 (2021)

    Google Scholar 

  25. Menon, A.K., Elkan, C.: Link prediction via matrix factorization. In: Gunopulos, D., Hofmann, T., Malerba, D., Vazirgiannis, M. (eds.) ECML PKDD 2011. LNCS (LNAI), vol. 6912, pp. 437–452. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-23783-6_28

    Chapter  Google Scholar 

  26. Naderiparizi, S., Chiu, K., Bloem-Reddy, B., Wood, F.: Uncertainty in neural processes. arXiv preprint arXiv:2010.03753 (2020)

  27. Pan, S., Luo, L., Wang, Y., Chen, C., Wang, J., Wu, X.: Unifying large language models and knowledge graphs: a roadmap. arXiv preprint arXiv:2306.08302 (2023)

  28. Paszke, A., et al.: PyTorch: an imperative style, high-performance deep learning library. In: Advances in Neural Information Processing Systems, vol. 32 (2019)

    Google Scholar 

  29. Perozzi, B., Al-Rfou, R., Skiena, S.: DeepWalk: online learning of social representations. In: Proceedings of the 20th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp. 701–710 (2014)

    Google Scholar 

  30. Schlichtkrull, M., Kipf, T.N., Bloem, P., van den Berg, R., Titov, I., Welling, M.: Modeling relational data with graph convolutional networks. In: Gangemi, A., et al. (eds.) ESWC 2018. LNCS, vol. 10843, pp. 593–607. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-93417-4_38

    Chapter  Google Scholar 

  31. Shi, B., Weninger, T.: Open-world knowledge graph completion. In: Proceedings of the AAAI Conference on Artificial Intelligence (2018)

    Google Scholar 

  32. Singh, G., Yoon, J., Son, Y., Ahn, S.: Sequential neural processes. In: Advances in Neural Information Processing Systems, vol. 32 (2019)

    Google Scholar 

  33. Singh, S., Póczos, B.: Analysis of k-nearest neighbor distances with application to entropy estimation. arXiv preprint arXiv:1603.08578 (2016)

  34. Sun, Z., Deng, Z.H., Nie, J.Y., Tang, J.: Rotate: knowledge graph embedding by relational rotation in complex space. arXiv preprint arXiv:1902.10197 (2019)

  35. Teru, K., Denis, E., Hamilton, W.: Inductive relation prediction by subgraph reasoning. In: International Conference on Machine Learning, pp. 9448–9457. PMLR (2020)

    Google Scholar 

  36. Toutanova, K., Chen, D., Pantel, P., Poon, H., Choudhury, P., Gamon, M.: Representing text for joint embedding of text and knowledge bases. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1499–1509 (2015)

    Google Scholar 

  37. Trouillon, T., Welbl, J., Riedel, S., Gaussier, É., Bouchard, G.: Complex embeddings for simple link prediction. In: International Conference on Machine Learning, pp. 2071–2080. PMLR (2016)

    Google Scholar 

  38. Van Kampen, N.G.: Stochastic differential equations. Phys. Rep. 24(3), 171–228 (1976)

    Article  MathSciNet  Google Scholar 

  39. Wan, G., Pan, S., Gong, C., Zhou, C., Haffari, G.: Reasoning like human: hierarchical reinforcement learning for knowledge graph reasoning. In: Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, pp. 1926–1932 (2021)

    Google Scholar 

  40. Wan, S., Pan, S., Yang, J., Gong, C.: Contrastive and generative graph convolutional networks for graph-based semi-supervised learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 10049–10057 (2021)

    Google Scholar 

  41. Wan, S., et al.: Multi-level graph learning network for hyperspectral image classification. Pattern Recogn. 129, 108705 (2022)

    Article  Google Scholar 

  42. Wan, S., Zhan, Y., Liu, L., Yu, B., Pan, S., Gong, C.: Contrastive graph poisson networks: semi-supervised learning with extremely limited labels. Adv. Neural. Inf. Process. Syst. 34, 6316–6327 (2021)

    Google Scholar 

  43. Wang, C., Zhou, X., Pan, S., Dong, L., Song, Z., Sha, Y.: Exploring relational semantics for inductive knowledge graph completion. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 4184–4192 (2022)

    Google Scholar 

  44. Wang, H., Ren, H., Leskovec, J.: Relational message passing for knowledge graph completion. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp. 1697–1707 (2021)

    Google Scholar 

  45. Wang, H., Zhang, F., Zhao, M., Li, W., Xie, X., Guo, M.: Multi-task feature learning for knowledge graph enhanced recommendation. In: Proceedings of the Web Conference 2019, pp. 2000–2010 (2019)

    Google Scholar 

  46. Wang, P., Han, J., Li, C., Pan, R.: Logic attention based neighborhood aggregation for inductive knowledge graph embedding. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 7152–7159 (2019)

    Google Scholar 

  47. Wang, R., et al.: Learning to sample and aggregate: few-shot reasoning over temporal knowledge graphs. In: Advances in Neural Information Processing Systems (2022)

    Google Scholar 

  48. Wang, Y., Chang, Y.Y., Liu, Y., Leskovec, J., Li, P.: Inductive representation learning in temporal networks via causal anonymous walks. arXiv preprint arXiv:2101.05974 (2021)

  49. Xiao, S., et al.: HMNet: hybrid matching network for few-shot link prediction. In: Jensen, C.S., et al. (eds.) Hmnet: Hybrid matching network for few-shot link prediction. LNCS, vol. 12681, pp. 307–322. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-73194-6_21

    Chapter  Google Scholar 

  50. Xiong, W., Hoang, T., Wang, W.Y.: DeepPath: a reinforcement learning method for knowledge graph reasoning. arXiv preprint arXiv:1707.06690 (2017)

  51. Xiong, W., Yu, M., Chang, S., Guo, X., Wang, W.Y.: One-shot relational learning for knowledge graphs. arXiv preprint arXiv:1808.09040 (2018)

  52. Xu, X., Zhang, P., He, Y., Chao, C., Yan, C.: Subgraph neighboring relations infomax for inductive link prediction on knowledge graphs. In: Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence (2022)

    Google Scholar 

  53. Yang, B., Yih, W.t., He, X., Gao, J., Deng, L.: Embedding entities and relations for learning and inference in knowledge bases. arXiv preprint arXiv:1412.6575 (2014)

  54. Zhang, C., Yao, H., Huang, C., Jiang, M., Li, Z., Chawla, N.V.: Few-shot knowledge graph completion. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 3041–3048 (2020)

    Google Scholar 

  55. Zhang, X., Liang, X., Zheng, X., Wu, B., Guo, Y.: MULTIFORM: few-shot knowledge graph completion via multi-modal contexts. In: Machine Learning and Knowledge Discovery in Databases: European Conference, ECML-PKDD 2022, Grenoble, France, September 19–23, 2022, Proceedings, Part II, pp. 172–187. Springer (2023). https://doi.org/10.1007/978-3-031-26390-3_11

  56. Zhang, Z., Lan, C., Zeng, W., Chen, Z., Chang, S.F.: Uncertainty-aware few-shot image classification. In: Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence (2020)

    Google Scholar 

  57. Zheng, S., Mai, S., Sun, Y., Hu, H., Yang, Y.: Subgraph-aware few-shot inductive link prediction via meta-learning. IEEE Trans. Knowl. Data Eng. (2022)

    Google Scholar 

Download references

Acknowledgement

This research is supported by NSF of China (No: 61973162), NSF of Jiangsu Province (No: BZ2021013), NSF for Distinguished Young Scholar of Jiangsu Province (No: BK20220080), the Fundamental Research Funds for the Central Universities (Nos: 30920032202, 30921013114), CAAI-Huawei MindSpore Open Fund, and “111” Program (No: B13022).

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China

    Zicheng Zhao & Chen Gong

  2. Jiangsu Key Laboratory of Image and Video Understanding for Social Security, Nanjing, China

    Zicheng Zhao

  3. Department of Data Science and AI, Monash University, Melbourne, Australia

    Linhao Luo

  4. School of Information and Communication Technology, Griffith University, Nathan, Australia

    Shirui Pan & Quoc Viet Hung Nguyen

  5. Key Laboratory of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education, Nanjing, China

    Chen Gong

Authors
  1. Zicheng Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Linhao Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shirui Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Quoc Viet Hung Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Chen Gong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chen Gong .

Editor information

Editors and Affiliations

  1. University of Michigan, Ann Arbor, MI, USA

    Danai Koutra

  2. University of Vienna, Vienna, Austria

    Claudia Plant

  3. Max Planck Institute for Software Systems, Kaiserslautern, Germany

    Manuel Gomez Rodriguez

  4. Politecnico di Torino, Turin, Italy

    Elena Baralis

  5. CENTAI, Turin, Italy

    Francesco Bonchi

Ethics declarations

Ethical Statement

In this research, we conducted experiments on publicly available datasets and implemented our approaches using commonly accepted techniques, giving utmost consideration to fairness and avoiding potential biases. We acknowledge the significance of transparency and have furnished comprehensive elucidations regarding our methodology and decision-making process. To conclude, our research adheres to ethical guidelines and poses no potential risks.

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhao, Z., Luo, L., Pan, S., Nguyen, Q.V.H., Gong, C. (2023). Towards Few-Shot Inductive Link Prediction on Knowledge Graphs: A Relational Anonymous Walk-Guided Neural Process Approach. In: Koutra, D., Plant, C., Gomez Rodriguez, M., Baralis, E., Bonchi, F. (eds) Machine Learning and Knowledge Discovery in Databases: Research Track. ECML PKDD 2023. Lecture Notes in Computer Science(), vol 14171. Springer, Cham. https://doi.org/10.1007/978-3-031-43418-1_31

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-43418-1_31

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-43417-4

  • Online ISBN: 978-3-031-43418-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships