Skip to main content

Advertisement

Springer Nature Link
Log in

Meta-Learning Based Few-Shot Link Prediction for Emerging Knowledge Graph

  • Regular Paper
  • Published:
Journal of Computer Science and Technology Aims and scope Submit manuscript

Abstract

Inductive knowledge graph embedding (KGE) aims to embed unseen entities in emerging knowledge graphs (KGs). The major recent studies of inductive KGE embed unseen entities by aggregating information from their neighboring entities and relations with graph neural networks (GNNs). However, these methods rely on the existing neighbors of unseen entities and suffer from two common problems: data sparsity and feature smoothing. Firstly, the data sparsity problem means unseen entities usually emerge with few triplets containing insufficient information. Secondly, the effectiveness of the features extracted from original KGs will degrade when repeatedly propagating these features to represent unseen entities in emerging KGs, which is termed feature smoothing problem. To tackle the two problems, we propose a novel model entitled Meta-Learning Based Memory Graph Convolutional Network (MMGCN) consisting of three different components: 1) the two-layer information transforming module (TITM) developed to effectively transform information from original KGs to emerging KGs; 2) the hyper-relation feature initializing module (HFIM) proposed to extract type-level features shared between KGs and obtain a coarse-grained representation for each entity with these features; and 3) the meta-learning training module (MTM) designed to simulate the few-shot emerging KGs and train the model in a meta-learning framework. The extensive experiments conducted on the few-shot link prediction task for emerging KGs demonstrate the superiority of our proposed model MMGCN compared with state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Bollacker K, Evans C, Paritosh P, Sturge T, Taylor J. Freebase: A collaboratively created graph database for structuring human knowledge. In Proc. the 2008 ACM SIGMOD International Conference on Management of Data, Jun. 2008, pp.1247–1250. DOI: https://doi.org/10.1145/1376616.1376746.

    Chapter  Google Scholar 

  2. Carlson A, Betteridge J, Kisiel B, Settles B, Hruschka E, Mitchell T. Toward an architecture for never-ending language learning. In Proc. the 24th AAAI Conference on Artificial Intelligence, Jul. 2010, pp.1306–1313. DOI: https://doi.org/10.1609/AAAI.V24I1.7519.

    Google Scholar 

  3. Auer S, Bizer C, Kobilarov G, Lehmann J, Cyganiak R, Ives Z. DBpedia: A nucleus for a Web of open data. In Proc. the 6th International Semantic Web Conference, 2nd Asian Semantic Web Conference, Nov. 2007, pp.722–735. DOI: https://doi.org/10.1007/978-3-540-76298-0_52.

    Google Scholar 

  4. Wang Y X, Khan A, Wu T X, Jin J H, Yan H J. Semantic guided and response times bounded top-k similarity search over knowledge graphs. In Proc. the 36th IEEE International Conference on Data Engineering, Apr. 2020, pp.445–456. DOI: https://doi.org/10.1109/ICDE48307.2020.00045.

    Google Scholar 

  5. Yang Z X. Biomedical information retrieval incorporating knowledge graph for explainable precision medicine. In Proc. the 43rd International ACM SIGIR conference on research and development in Information Retrieval, Jul. 2020, p.2486. DOI: https://doi.org/10.1145/3397271.3401458.

    Google Scholar 

  6. Wong C M, Feng F, Zhang W, Vong C M, Chen H, Zhang Y C, He P, Chen H, Zhao K, Chen H J. Improving conversational recommender system by pretraining billion-scale knowledge graph. In Proc. the 37th IEEE International Conference on Data Engineering, Apr. 2021, pp.2607–2612. DOI: https://doi.org/10.1109/ICDE51399.2021.00291.

    Google Scholar 

  7. Deng Z Y, Li C Y, Liu S J, Ali W, Shao J. Knowledge-aware group representation learning for group recommendation. In Proc. the 37th IEEE International Conference on Data Engineering, Apr. 2021, pp.1571–1582. DOI: https://doi.org/10.1109/ICDE51399.2021.00139.

    Google Scholar 

  8. Hu S, Zou L, Yu J X, Wang H X, Zhao D Y. Answering natural language questions by subgraph matching over knowledge graphs (extended abstract). In Proc. the 34th IEEE International Conference on Data Engineering, Apr. 2018, pp.1815–1816. DOI: https://doi.org/10.1109/ICDE.2018.00265.

    Google Scholar 

  9. Kaiser M, Roy R S, Weikum G. Reinforcement learning from reformulations in conversational question answering over knowledge graphs. In Proc. the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Jul. 2021, pp.459–469. DOI: https://doi.org/10.1145/3404835.3462859.

    Google Scholar 

  10. Nickel M, Murphy K, Tresp V, Gabrilovich E. A review of relational machine learning for knowledge graphs. Proceedings of the IEEE, 2016, 104(1): 11–33. DOI: https://doi.org/10.1109/JPROC.2015.2483592.

    Article  Google Scholar 

  11. Bordes A, Usunier N, García-Durán A, Weston J, Yakhnenko O. Translating embeddings for modeling multi-relational data. In Proc. the 26th Annual Conference on Neural Information Processing Systems, Dec. 2013, pp.2787–2795.

    Google Scholar 

  12. Yang B S, Yih W T, He X D, Gao J F, Deng L. Embedding entities and relations for learning and inference in knowledge bases. In Proc. the 3rd International Conference on Learning Representations, May 2015.

    Google Scholar 

  13. Shi B X, Weninger T. Open-world knowledge graph completion. In Proc. the 32nd AAAI Conference on Artificial Intelligence, Feb. 2018, pp.1957–1964. DOI: https://doi.org/10.1609/aaai.v32i1.11535.

    Google Scholar 

  14. Hamaguchi T, Oiwa H, Shimbo M, Matsumoto Y. Knowledge transfer for out-of-knowledge-base entities: A graph neural network approach. In Proc. the 26th International Joint Conference on Artificial Intelligence, Aug. 2017, pp.1802–1808. DOI: https://doi.org/10.24963/ijcai.2017/250.

    Google Scholar 

  15. Wang P F, Han J L, Li C L, Pan R. Logic attention based neighborhood aggregation for inductive knowledge graph embedding. In Proc. the 33rd AAAI Conference on Artificial Intelligence, Jan. 27-Feb. 1, 2019, pp.7152–7159. DOI: https://doi.org/10.1609/aaai.v33i01.33017152.

    Google Scholar 

  16. Chen M Y, Zhang W, Zhang W, Chen Q, Chen H J. Meta relational learning for few-shot link prediction in knowledge graphs. In Proc. the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, Nov. 2019, pp.4217–4226. DOI: https://doi.org/10.18653/V1/D19-1431.

    Google Scholar 

  17. He Z L, Chen P F, Li X Y, Wang Y F, Yu G B, Chen C L, Li X R, Zheng Z B. A spatiotemporal deep learning approach for unsupervised anomaly detection in cloud systems. IEEE Trans. Neural Networks and Learning Systems, 2023, 34(4): 1705–1719. DOI: https://doi.org/10.1109/TNNLS.2020.3027736.

    Article  Google Scholar 

  18. Zhang Y F, Wang W Q, Chen W, Xu J J, Liu A, Zhao L. Meta-learning based hyper-relation feature modeling for out-of-knowledge-base embedding. In Proc. the 30th ACM International Conference on Information and Knowledge Management, Oct. 2021, pp.2637–2646. DOI: https://doi.org/10.1145/3459637.3482367.

    Google Scholar 

  19. Wang Z, Zhang J W, Feng J L, Chen Z. Knowledge graph embedding by translating on hyperplanes. In Proc. the 28th AAAI Conference on Artificial Intelligence, Jul. 2014, pp.1112–1119. DOI: https://doi.org/10.1609/AAAI.V28I1.8870.

    Google Scholar 

  20. Sun Z Q, Deng Z H, Nie J Y, Tang J. RotatE: Knowledge graph embedding by relational rotation in complex space. In Proc. the 7th International Conference on Learning Representations, May 2019.

    Google Scholar 

  21. Zhang Z Q, Cai J Y, Zhang Y D, Wang J. Learning hierarchy-aware knowledge graph embeddings for link prediction. In Proc. the 34th AAAI Conference on Artificial Intelligence, Feb. 2020, pp.3065–3072. DOI: https://doi.org/10.1609/aaai.v34i03.5701.

    Google Scholar 

  22. Nickel M, Tresp V, Kriegel H P. A three-way model for collective learning on multi-relational data. In Proc. the 28th International Conference on Machine Learning, Jun. 28-Jul. 1, 2011, pp.809–816.

    Google Scholar 

  23. Dettmers T, Minervini P, Stenetorp P, Riedel S. Convolutional 2D knowledge graph embeddings. In Proc. the 32nd AAAI Conference on Artificial Intelligence, Feb. 2018, pp.1811–1818. DOI: https://doi.org/10.1609/AAAI.V32I1.11573.

    Google Scholar 

  24. Nguyen D Q, Nguyen T D, Nguyen D Q, Phung D. A novel embedding model for knowledge base completion based on convolutional neural network. In Proc. the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Jun. 2018, pp.327–333. DOI: https://doi.org/10.18653/v1/n18-2053.

    Google Scholar 

  25. Schlichtkrull M, Kipf T N, Bloem P, van den Berg R, Titov I, Welling M. Modeling relational data with graph convolutional networks. In Proc. the 15th International Conference on Semantic Web, Jun. 2018, pp.593–607. DOI: https://doi.org/10.1007/978-3-319-93417-4_38.

    Google Scholar 

  26. Shang C, Tang Y, Huang J, Bi J B, He X D, Zhou B W. End-to-end structure-aware convolutional networks for knowledge base completion. In Proc. the 33rd AAAI Conference on Artificial Intelligence, Jan. 27-Feb. 1, 2019, pp.3060–3067. DOI: https://doi.org/10.1609/AAAI.V33I01.33013060.

    Google Scholar 

  27. Xiong W H, Yu M, Chang S Y, Guo X X, Wang W Y. One-shot relational learning for knowledge graphs. In Proc. the 2018 Conference on Empirical Methods in Natural Language Processing, Oct. 31–Nov. 7, 2018, pp.1980–1990. DOI: https://doi.org/10.18653/V1/D18-1223.

    Chapter  Google Scholar 

  28. Zhang C X, Yao H X, Huang C, Jiang M, Li Z H, Chawla N V. Few-shot knowledge graph completion. In Proc. the 34th AAAI Conference on Artificial Intelligence, Feb. 2020, pp.3041–3048. DOI: https://doi.org/10.1609/AAAI.V34I03.5698.

    Google Scholar 

  29. Muggleton S. Inductive logic programming. New Generation Computing, 1991, 8(4): 295–318. DOI: https://doi.org/10.1007/BF03037089.

    Article  Google Scholar 

  30. Yang F, Yang Z L, Cohen W W. Differentiable learning of logical rules for knowledge base reasoning. In Proc. the 31st Annual Conference on Neural Information Processing Systems, Dec. 2017, pp.2316–2325.

    Google Scholar 

  31. Cohen W W. TensorLog: A differentiable deductive database. arXiv: 1605.06523, 2016. https://arxiv.org/abs/1605.06523, Sept. 2024.

    Google Scholar 

  32. Sadeghian A, Armandpour M, Ding P, Wang D Z. DRUM: End-to-end differentiable rule mining on knowledge graphs. In Proc. the 33rd Annual Conference on Neural Information Processing Systems, Dec. 2019, Article No. 1375.

  33. Qu M, Chen J K, Xhonneux L P A C, Bengio Y, Tang J. RNNlogic: Learning logic rules for reasoning on knowledge graphs. In Proc. the 9th International Conference on Learning Representations, May 2021.

    Google Scholar 

  34. Cheng K W, Liu J H, Wang W, Sun Y Z. RLogic: Recursive logical rule learning from knowledge graphs. In Proc. the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Aug. 2022, pp.179–189. DOI: https://doi.org/10.1145/3534678.3539421.

    Chapter  Google Scholar 

  35. Chen S Y, Fang H, Cai Y F, Huang X, Sun M M. Differentiable neuro-symbolic reasoning on large-scale knowledge graphs. In Proc. the 37th Annual Conference on Neural Information Processing Systems, Dec. 2023, Article No. 1222.

  36. He Y Q, Wang Z H, Zhang P, Tu Z P, Ren Z C. VN network: Embedding newly emerging entities with virtual neighbors. In Proc. the 29th ACM International Conference on Information and Knowledge Management, Oct. 2020, pp.505–514. DOI: https://doi.org/10.1145/3340531.3411865.

    Google Scholar 

  37. Baek J, Lee D B, Hwang S J. Learning to extrapolate knowledge: Transductive few-shot out-of-graph link prediction. In Proc. the 34th Annual Conference on Neural Information Processing Systems, Dec. 2020, Article No. 47.

  38. Wang H W, Ren H Y, Leskovec J. Relational message passing for knowledge graph completion. In Proc. the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Aug. 2021, pp.1697–1707. DOI: https://doi.org/10.1145/3447548.3467247.

    Chapter  Google Scholar 

  39. Zhu Z C, Zhang Z B, Xhonneux L P, Tang J. Neural bellman-ford networks: A general graph neural network framework for link prediction. In Proc. the 35th Annual Conference on Neural Information Processing Systems, Dec. 2021, Article No. 2256.

  40. Zhang Y Q, Yao Q M. Knowledge graph reasoning with relational digraph. In Proc. the 2022 ACM Web Conference, Apr. 2022, pp.912–924. DOI: https://doi.org/10.1145/3485447.3512008.

    Chapter  Google Scholar 

  41. Wang C J, Zhou X F, Pan S R, Dong L H, Song Z L, Sha Y. Exploring relational semantics for inductive knowledge graph completion. In Proc. the 36th AAAI Conference on Artificial Intelligence, Feb. 22-Mar. 1, 2022, pp.4184–4192. DOI: https://doi.org/10.1609/AAAI.V36I4.20337.

    Google Scholar 

  42. Zhang Y Q, Zhou Z K, Yao Q M, Chu X W, Han B. AdaProp: Learning adaptive propagation for graph neural network based knowledge graph reasoning. In Proc. the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Aug. 2023, pp.3446–3457. DOI: https://doi.org/10.1145/3580305.3599404.

    Chapter  Google Scholar 

  43. Lee J, Chung C, Whang J J. InGram: Inductive knowledge graph embedding via relation graphs. In Proc. the 40th International Conference on Machine Learning, Jul. 2023, pp.18796–18809.

    Google Scholar 

  44. Teru K K, Denis E G, Hamilton W L. Inductive relation prediction by subgraph reasoning. In Proc. the 37th International Conference on Machine Learning, Jul. 2020, pp.9448–9457.

    Google Scholar 

  45. Chen J J, He H R, Wu F, Wang J. Topology-aware correlations between relations for inductive link prediction in knowledge graphs. In Proc. the 35th AAAI Conference on Artificial Intelligence, Feb. 2021, pp.6271–6278. DOI: https://doi.org/10.1609/AAAI.V35I7.16779.

    Google Scholar 

  46. Liu S W, Grau B C, Horrocks I, Kostylev E V. INDIGO: GNN-based inductive knowledge graph completion using pair-wise encoding. In Proc. the 35th Annual Conference on Neural Information Processing Systems, Dec. 2021, Article No. 156.

  47. Chen M Y, Zhang W, Zhu Y S, Zhou H T, Yuan Z G, Xu C L, Chen H J. Meta-knowledge transfer for inductive knowledge graph embedding. In Proc. the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, Jul. 2022, pp.927–937. DOI: https://doi.org/10.1145/3477495.3531757.

    Google Scholar 

  48. Xu X H, Zhang P, He Y Q, Chao C P, Yan C Y. Subgraph neighboring relations infomax for inductive link prediction on knowledge graphs. In Proc. the 31st International Joint Conference on Artificial Intelligence, Jul. 2022, pp.2341–2347. DOI: https://doi.org/10.24963/IJCAI.2022/325.

    Google Scholar 

  49. Zhang Y F, Wang W Q, Yin H Z, Zhao P P, Chen W, Zhao L. Disconnected emerging knowledge graph oriented inductive link prediction. In Proc. the 39th IEEE International Conference on Data Engineering, Apr. 2023, pp.381–393. DOI: https://doi.org/10.1109/ICDE55515.2023.00036.

    Google Scholar 

  50. Geng Y X, Chen J Y, Pan J Z, Chen M Y, Jiang S, Zhang W, Chen H J. Relational message passing for fully inductive knowledge graph completion. In Proc. the 39th IEEE International Conference on Data Engineering, Apr. 2023, pp.1221–1233. DOI: https://doi.org/10.1109/ICDE55515.2023.00098.

    Google Scholar 

  51. Toutanova K, Chen D Q. Observed versus latent features for knowledge base and text inference. In Proc. the 3rd Workshop on Continuous Vector Space Models and their Compositionality, Jul. 2015, pp.57–66. DOI: https://doi.org/10.18653/V1/W15-4007.

    Chapter  Google Scholar 

  52. Xiong W H, Hoang T, Wang W Y. DeepPath: A reinforcement learning method for knowledge graph reasoning. In Proc. the 2017 Conference on Empirical Methods in Natural Language Processing, Sept. 2017, pp.564–573. DOI: https://doi.org/10.18653/v1/d17-1060.

    Google Scholar 

  53. Bordes A, Weston J, Collobert R, Bengio Y. Learning structured embeddings of knowledge bases. In Proc. the 25th AAAI Conference on Artificial Intelligence, Aug. 2011, pp.301–306. DOI: https://doi.org/10.1609/AAAI.V25I1.7917.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Soochow University, Suzhou, 215000, China

    Yu-Feng Zhang  (张钰峰), Wei Chen  (陈　伟), Peng-Peng Zhao  (赵朋朋), Jia-Jie Xu  (许佳捷), Jun-Hua Fang  (房俊华) & Lei Zhao  (赵　雷)

Authors
  1. Yu-Feng Zhang  (张钰峰)
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei Chen  (陈　伟)
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Peng-Peng Zhao  (赵朋朋)
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jia-Jie Xu  (许佳捷)
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jun-Hua Fang  (房俊华)
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Lei Zhao  (赵　雷)
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Wei Chen  (陈　伟) or Lei Zhao  (赵　雷).

Ethics declarations

Conflict of Interest The authors declare that they have no conflict of interest.

Additional information

A preliminary version of the paper was published in the Proceedings of CIKM 2021.

This work was supported by the National Natural Science Foundation of China under Grant No. 62272332 and the Major Program of the Natural Science Foundation of Jiangsu Higher Education Institutions of China under Grant No. 22KJA520006.

Wei Chen is the principal investigator of the two funding projects; Lei Zhao is the designer of the research framework.

Yu-Feng Zhang received his B.S. degree in software engineering from Soochow University, Suzhou, in 2020. He is currently a Ph.D. candidate at the School of Computer Science and Technology, Soochow University, Suzhou. His main research interests include knowledge graph, graph representation learning, and graph databases.

Wei Chen received his Ph.D. degree in computer science from Soochow University, Suzhou, in 2018. He is currently an associate professor at the School of Computer Science and Technology, Soochow University, Suzhou. His research interests include heterogeneous information network analysis, cross-platform linkage and recommendation, spatio-temporal database, and knowledge graph embedding and refinement.

Peng-Peng Zhao received his Ph.D. degree in computer science from Soochow University, Suzhou, in 2008. He is now a professor at the School of Computer Science and Technology, Soochow University, Suzhou. His current research interests include data mining, deep learning, big data analysis, and recommender systems.

Jia-Jie Xu received his M.S. degree from the University of Queensland, Brisbane, in 2006, and his Ph.D. degree from the Swinburne University of Technology, Melbourne, in 2011. He is currently a professor at the School of Computer Science and Technology, Soochow University, Suzhou. His research interests include spatiotemporal database systems, big data analytics, and recommendation systems.

Jun-Hua Fang received his Ph.D. degree in computer science from East China Normal University, Shanghai, in 2017. He is currently an associate professor at the School of Computer Science and Technology, Soochow University, Suzhou. His research interests mainly include spatio-temporal database, cloud computing and distributed stream processing.

Lei Zhao received his Ph.D. degree in computer science from Soochow University, Suzhou, in 2006. He is now a professor at the School of Computer Science and Technology, Soochow University, Suzhou. His recent research is to analyze large graph databases in an effective, efficient, and secure way.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, YF., Chen, W., Zhao, PP. et al. Meta-Learning Based Few-Shot Link Prediction for Emerging Knowledge Graph. J. Comput. Sci. Technol. 39, 1058–1077 (2024). https://doi.org/10.1007/s11390-024-2863-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11390-024-2863-8

Keywords