Skip to main content
SpringerLink
Log in

Enabling inductive knowledge graph completion via structure-aware attention network

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Knowledge graph completion (KGC) aims at complementing missing entities and relations in a knowledge graph (KG). Popular KGC approaches based on KG embedding are typically limited to the transductive setting, i.e., all entities must be seen during training, which is impractical for real-world KGs where new entities are emerging daily. Recent inductive KG embedding approaches propose to train a neighborhood aggregator in conjunction with entity and relation embeddings, which helps to embed unseen entities via existing neighbors. However, existing methods do not fully take advantage of the structural information of neighbors and are unable to handle triplets involving unseen relations. In this paper, we work further and propose a novel and unified inductive KGC model, namely structure-aware attention network (SAAN), which can efficiently generate embeddings of unseen entities and relations by aggregating neighbors with structure-aware attention weights. Unlike conventional embedding-based attention methods, SAAN can naturally learn importance weights by modeling structural correlations between nodes in an embedding-independent manner and can be applied to any existing KG embedding model. Experimental results on both transductive and inductive KGC tasks show that our model significantly outperforms state-of-the-art methods.

Graphical abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Algorithm 1
Algorithm 2
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Data Availibility Statement

Data will be made available on reasonable request

Notes

  1. https://github.com/takuo-h/GNN-for-OOKB/tree/master/2-OOKB-setting/datasets

  2. https://github.com/Lion-ZS/DualE/tree/main/DualE-master

  3. https://github.com/NaoDist/SAAN_Datasets

  4. https://github.com/thunlp/OpenKE

  5. https://github.com/dmlc/dgl

  6. https://github.com/uma-pi1/kge

References

  1. Zhang Y, Yao Q, Kwok JT (2023) Bilinear scoring function search for knowledge graph learning. IEEE Trans Pattern Anal Mach Int 45(2):1458–1473

    Article  Google Scholar 

  2. Li Q, Wang D, Feng S, Niu C, Zhang Y (2022) Global graph attention embedding network for relation prediction in knowledge graphs. IEEE Trans Neural Netw Learning Syst 33(11):6712–6725

    Article  MathSciNet  Google Scholar 

  3. Li Q, Peng H, Li J, Xia C, Yang R, Sun L, Yu PS, He L (2022) A survey on text classification: From traditional to deep learning. ACM Trans Int Syst Technol 13(2):1–41

    Google Scholar 

  4. Cui H, Peng T, Xiao F, Han J, Han R, Liu L (2023) Incorporating anticipation embedding into reinforcement learning framework for multi-hop knowledge graph question answering. Inf Sci 619:745–761

    Article  Google Scholar 

  5. Suissa O, Zhitomirsky-Geffet M, Elmalech A (2023) Question answering with deep neural networks for semi-structured heterogeneous genealogical knowledge graphs. Semantic Web 14(2):209–237

    Article  Google Scholar 

  6. Wang C, Huang Z, Wan Y, Wei J, Zhao J, Wang P (2023) Fualign: Crosslingual entity alignment via multi-view representation learning of fused knowledge graphs. Inf Fusion 89:41–52

    Article  Google Scholar 

  7. Oh B, Seo S, Hwang J, Lee D, Lee K-H (2022) Open-world knowledge graph completion for unseen entities and relations via attentive feature aggregation. Inf Sci 586:468–484

    Article  Google Scholar 

  8. Bordes A, Usunier N, Garcia-Duran A, Weston J, Akhnenko OY (2013) Translating embeddings for modeling multi-relational data. In: Proceedings of the 27th Conference on Neural Information Processing Systems, pp 2787–2795

  9. Hu B, Ye Y, Zhong Y, Pan J, Hu M (2022) Transmkr: Translation-based knowledge graph enhanced multi-task point-of-interest recommendation. Neurocomputing 474:107–114

    Article  Google Scholar 

  10. Xie R, Liu Z, Jia J, Luan H, Sun M (2016) Representation learning of knowledge graphs with entity descriptions. In: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, pp 2659–2665

  11. Zhou X, Li S, Li Z, Li W (2021) Information diffusion across cyber-physicalsocial systems in smart city: A survey. Neurocomputing 444:203–213

    Article  Google Scholar 

  12. Yu X, Li W, Yang B, Li X, Chen J, Fu G (2023) Deviation distance entropy: A method for quantifying the dynamic features of biomedical time series. Chaos Solition Fractals 168:113157

    Article  Google Scholar 

  13. Shi B, Weninger T (2018) Open-world knowledge graph completion. In: Proceedings of the 32th AAAI Conference on Artificial Intelligence, pp 1957–1964

  14. Zhang J, Huang J, Gao J, Han R, Zhou C (2022) Knowledge graph embedding by logical-default attention graph convolution neural network for link prediction. Inf Sci 593:201–215

    Article  Google Scholar 

  15. Omran PG, Taylor K, Méndez SJR, Haller A (2022) Active knowledge graph completion. Inf Sci 604:267–279

    Article  Google Scholar 

  16. Chen P, Wang Y, Yu X, Feng R (2022) Qlogice: Quantum logic empowered embedding for knowledge graph completion. Knowl Based Syst 239:107963

    Article  Google Scholar 

  17. Le T, Le N, Le B (2023) Knowledge graph embedding by relational rotation and complex convolution for link prediction. Expert Syst Appl 214:119122

    Article  Google Scholar 

  18. Huang X, Tang J, Tan Z, Zeng W, Wang J, Zha X (2021) Knowledge graph embedding by relational and entity rotation. Knowl Based Syst 229:107310

    Article  Google Scholar 

  19. Sun Z, Deng Z-H, Nie J, Tang J (2019) Rotate: Knowledge graph embedding by relational rotation in complex space. In: Proceedings of the 7th International Conference on Learning Representations, pp 1–18

  20. Tang Y, Huang J, Wang G, He X, Zhou B (2020) Orthogonal relation transforms with graph context modeling for knowledge graph embedding. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp 2713–2722

  21. Zhang Z, Cai J, Zhang Y, Wang J (2020) Learning hierarchy-aware knowledge graph embeddings for link prediction. In: Proceedings of the 35th AAAI Conference on Artificial Intelligence, pp 3065–3072

  22. Cao Z, Xu Q, Yang Z, Cao X, Huang Q (2021) Dual quaternion knowledge graph embeddings. In: Proceedings of the 35th AAAI Conference on Artificial Intelligence, pp 6894–6902

  23. Nickel M, Tresp V, Kriegel H-P (2011) A three-way model for collective learning on multi-relational data. In: Proceedings of the 28th International Conference on Machine Learning, pp 809–816

  24. Yang B, Yih W-t, He X, Gao J, Deng L (2015) Embedding entities and relations for learning and inference in knowledge bases. In: Proceedings of the 3rd International Conference on Learning Representations

  25. Nickel M, Tresp V, Kriegel H-P (2018) Simple embedding for link prediction in knowledge graphs. In: Proceedings of the Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, pp 4289–4300

  26. Xu Y, Zhuang Z, Li W, Zhou X (2018) Effective community division based on improved spectral clustering. Neurocomputing 279:54–62

    Article  Google Scholar 

  27. Li W, Li Z, Luvembe AM, Yang C (2021) Influence maximization algorithm based on gaussian propagation model. Inf Sci 568:386–402

    Article  MathSciNet  Google Scholar 

  28. Nickel M, Rosasco L, Poggio TA (2016) Holographic embeddings of knowledge graphs. In: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, pp 1955–1961

  29. Zhang Z, Li Z, Liu H, Xiong NN (2022) Multi-scale dynamic convolutional network for knowledge graph embedding. IEEE Trans Knowl Data Eng 34(5):2335–2347

    Article  Google Scholar 

  30. Li W, Ni L, Wang J, Wang C (2022) Collaborative representation learning for nodes and relations via heterogeneous graph neural network. Knowl Based Syst 255:109673

    Article  Google Scholar 

  31. Zhang C, Li W, Wei D, Liu Y, Li Z (2022) Network dynamic gcn influence maximization algorithm with leader fake labeling mechanism. IEEE Trans Comput Social Syst 1–9

  32. Schlichtkrull MS, Kipf TN, Bloem P, van den Berg R, Titov I, Welling M (2018) Modeling relational data with graph convolutional networks. In: Proceedings of the 15th International Conference on The Semantic Web, pp 593–607

  33. Huang Z, Li X, Ye Y, Zhang B, Xu G, Gan W (2023) Multi-view knowledge graph fusion via knowledge-aware attentional graph neural network. App Int 53(4):3652–3671

    Article  Google Scholar 

  34. Hu W, Li W, Zhou X, Kawai A, Fueda K, Qian Q, Wang J (2022) Spatio-temporal graph convolutional networks via view fusion for trajectory data analytics. IEEE Transactions on Intelligent Transportation Systems

  35. Ravindra NG, Sehanobish A, Pappalardo JL, Hafler DA, van Dijk D (2020) Disease state prediction from single-cell data using graph attention networks. In: Proceedings of ACM Conference on Health, Inference, and Learning, pp 121–130

  36. Shanthamallu US, Thiagarajan JJ, Spanias A (2020) A regularized attention mechanism for graph attention networks. In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, pp 3372–3376

  37. Nathani D, Chauhan J, Sharma C, Kaul M (2020) Learning attention-based embeddings for relation prediction in knowledge graphs. In: Proceedings of the 57th Conference of the Association for Computational Linguistics, pp 4710–4723

  38. Dettmers T, Minervini P, Stenetorp P, Riedel S (2018) Convolutional 2d knowledge graph embeddings. In: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, pp 1811–1818

  39. Zhou Z, Wang C, Feng Y, Chen D (2022) Jointe: Jointly utilizing 1d and 2d convolution for knowledge graph embedding. Knowl Based Syst 240:108100

    Article  Google Scholar 

  40. Ren J, Jin W, Wu Y, Sun Z (2023) A grouping-attention convolutional neural network for performance degradation estimation of high-speed train lateral damper. Appl Int 53(1):658–682

    Article  Google Scholar 

  41. Yao S, Pi D, Chen J, Xu Y (2022) Gckg: Novel gated convolutional embedding model for knowledge graphs. Expert Syst Appl 208:118142

    Article  Google Scholar 

  42. Li W, Li Y, Liu W, Wang C (2022) An influence maximization method based on crowd emotion under an emotion-based attribute social network. Inf Process Manag 59(2):102818

    Article  Google Scholar 

  43. Oh B, Seo S, Hwang J, Lee D, Lee K-H (2022) Open-world knowledge graph completion for unseen entities and relations via attentive feature aggregation. Inf Sci 586:468–484

    Article  Google Scholar 

  44. Chen X, Jia S, Ding L, Shen H, Xiang Y (2020) Sdt: An integrated model for open-world knowledge graph reasoning. Expert Systems with Applications 162:113889

    Article  Google Scholar 

  45. Niu L, Fu C, Yang Q, Li Z, Chen Z, Liu Q, Zheng K (2021) Open-world knowledge graph completion with multiple interaction attention. World Wide Web 24(1):419–439

    Article  Google Scholar 

  46. Shah H, Villmow J, Ulges A, Schwanecke U, Shafait F (2019) An openworld extension to knowledge graph completion models. In: Proceedings of the 33th AAAI Conference on Artificial Intelligence, pp 3044–3051

  47. Hamaguchi T, Oiwa H, Shimbo M, Matsumoto Y (2017) Knowledge transfer for out-of-knowledge-base entities: A graph neural network approach. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence, pp 1802–1808

  48. Wang P, Han J, Li C, Pan R (2019) Logic attention based neighborhood aggregation for inductive knowledge graph embedding. In: Proceedings of the 33th AAAI Conference on Artificial Intelligence, pp 7152–7159

  49. Teru KK, Denis EG, Hamilton WL (2020) Inductive relation prediction by subgraph reasoning. In: Proceedings of the 37th International Conference on Machine Learning, pp 9448–9457

  50. Chen J, He H, Wu F, Wang J (2021) Topology-aware correlations between relations for inductive link prediction in knowledge graphs. In: Proceedings of the 35th AAAI Conference on Artificial Intelligence, pp 6271–6278

  51. Xu X, Zhang P, He Y, Chao C, Yan C (2022) Subgraph neighboring relations infomax for inductive link prediction on knowledge graphs. In: Proceedings of the 31th International Joint Conference on Artificial Intelligence, pp 2341–2347

  52. Butt S, Bakhtyar M, Noor W, Baber J, Ullah I, Ahmed A, Basit A, Kakar MSH (2022) Semantic similarity based food entities recognition using wordnet. J Int Fuzzy Syst 43(2):2069–2078

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Engineering and Science, Shanghai University, Shanghai, 200444, China

    Jingchao Wang, Weimin Li & Wei Liu

  2. School of Information and Communication Technology, Griffith University, Queensland, 4222, Australia

    Can Wang

  3. Faculty of Human Sciences, Waseda University, Tokorozawa, 169-8050, Japan

    Qun Jin

Authors
  1. Jingchao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Weimin Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wei Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Can Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Qun Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Weimin Li.

Ethics declarations

Conflicts of interest

The authors declare that they have no conflicts of interest

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, J., Li, W., Liu, W. et al. Enabling inductive knowledge graph completion via structure-aware attention network. Appl Intell 53, 25003–25027 (2023). https://doi.org/10.1007/s10489-023-04768-1

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-023-04768-1

Keywords

Search

Navigation