Skip to main content
SpringerLink
Log in

Enhancing knowledge graph embedding by composite neighbors for link prediction

  • Regular Paper
  • Published:
Computing Aims and scope Submit manuscript

Abstract

Knowledge graph embedding (KGE) aims to represent entities and relations in a low-dimensional continuous vector space. Recent KGE works focus on incorporating additional information, such as local neighbors and textual descriptions, to learn valuable representations. However, the non-uniformity and redundancy hinder the effectiveness of entity features from those information sources. In this paper, we propose a novel end-to-end framework, called composite neighborhood embedding (CoNE), utilizing composite neighbors to enhance the existing KGE methods. To ease past problems, the new composite neighbors are gathered from both entity descriptions and local neighbors. We design a novel Graph Memory Networks to extract entity features from composite neighbors, and fulfill the entity representation in the target KGE method. The experimental results show that CoNE effectively enhances three different KGE methods, TransE, ConvE, and RotatE, and achieves the state-of-the-art results on four real-world large datasets. Furthermore, our approach outperforms the recent text-enhanced models with fewer parameters and calculation. The source code of our work can be obtained from https://github.com/KyneWang/CoNE.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Bollacker KD, Evans C, Paritosh P, Sturge T, Taylor J (2008) Freebase: a collaboratively created graph database for structuring human knowledge. In: Proceedings of the 2008 ACM SIGMOD international conference on management of data, pp 1247–1250

  2. Bordes A, Glorot X, Weston J, Bengio Y (2014) A semantic matching energy function for learning with multi-relational data. Mach Learn 94:233–259

    Article  MathSciNet  Google Scholar 

  3. Bordes A, Usunier N, García-Durán A, Weston J, Yakhnenko O (2013) Translating embeddings for modeling multi-relational data. In: Proceedings of advances in neural information processing systems (NIPS 2013), pp 2787–2795

  4. Dettmers T, Minervini P, Stenetorp P, Riedel S (2018) Convolutional 2D knowledge graph embeddings. In: Proceedings of the 32th AAAI conference on artificial intelligence, pp 1811–1818

  5. Duvenaud DK, Maclaurin D, Aguilera-Iparraguirre J, Gómez-Bombarelli R, Hirzel T, Aspuru-Guzik A, Adams RP (2015) Convolutional networks on graphs for learning molecular fingerprints. In: Proceedings of advances in neural information processing systems (NIPS 2015), pp 2224–2232

  6. Fan M, Zhou Q, Zheng TF, Grishman R (2017) Distributed representation learning for knowledge graphs with entity descriptions. Pattern Recognit Lett 93:31–37

    Article  Google Scholar 

  7. Feng J, Huang M, Yang Y, Zhu X (2016) Gake: Graph aware knowledge embedding. In: Proceedings of the 32th international conference on computational linguistics (COLING 2016), pp 641–651

  8. Guo D, Tang D, Duan N, Zhou M, Yin J (2018) Dialog-to-action: conversational question answering over a large-scale knowledge base. In: Proceedings of advances in neural information processing systems (NIPS 2018), pp 2942–2951

  9. Ji G, He S, Xu L, Liu K, Zhao J (2015) Knowledge graph embedding via dynamic mapping matrix. In: Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing, pp 687–696

  10. Ji S, Pan S, Cambria E, Marttinen P, Yu PS (2020) A survey on knowledge graphs: Representation, acquisition and applications. CoRR arXiv:2002.00388

  11. Kingma DP, Ba JL (2015) Adam: a method for stochastic optimization. In: Proceedings of international conference on learning representations (ICLR 2015)

  12. Lin Y, Liu Z, Sun M, Liu Y, Zhu X (2015) Learning entity and relation embeddings for knowledge graph completion. In: Proceedings of the 29th AAAI conference on artificial intelligence, pp 2181–2187

  13. Miller GA (1995) Wordnet: a lexical database for english. Commun ACM 38:39–41

    Article  Google Scholar 

  14. Nathani D, Chauhan J, Sharma C, Kaul M (2019) Learning attention-based embeddings for relation prediction in knowledge graphs. In: Proceedings of the 57th conference of the association for computational linguistics, ACL 2019, pp 4710–4723

  15. Nguyen DQ, Nguyen TD, Nguyen DQ, Phung DQ (2017) A novel embedding model for knowledge base completion based on convolutional neural network. In: Proceedings of the 2017 conference of the North American chapter of the Association for Computational Linguistics, vol 2, pp 327–333

  16. Nguyen DQ, Vu T, Nguyen TD, Nguyen DQ, Phung DQ (2019) A capsule network-based embedding model for knowledge graph completion and search personalization. In: Proceedings of the 2019 conference of the North American chapter of the Association for Computational Linguistics, NAACL-HLT 2019, pp 2180–2189

  17. Nickel M, Tresp V, Kriegel HP (2011) A three-way model for collective learning on multi-relational data. In: Proceedings of the 28th international conference on machine learning, pp 809–816

  18. Pujara J, Augustine E, Getoor L (2017) Sparsity and noise: where knowledge graph embeddings fall short. In: Proceedings of the 2017 conference on empirical methods in natural language processing, pp 1751–1756

  19. Schlichtkrull MS, Kipf TN, Bloem P, van den Berg R, Titov I, Titov I, Welling M, Welling M (2018) Modeling relational data with graph convolutional networks. In: Proceedings of extended semantic web conference, pp 593–607

  20. Shi J, Gao H, Qi G, Zhou Z (2017) Knowledge graph embedding with triple context. In: Proceedings of the 2017 ACM on conference on information and knowledge management, pp 2299–2302

  21. Socher R, Chen D, Manning CD, Ng AY (2013) Reasoning with neural tensor networks for knowledge base completion. In: Proceedings of advances in neural information processing systems (NIPS 2013), pp 926–934

  22. Sukhbaatar S, Szlam A, Weston J, Fergus R (2015) End-to-end memory networks. In: Proceedings of advances in neural information processing systems (NIPS 2015), pp 2440–2448

  23. Sun Z, Deng Z, Nie J, Tang J (2019) Rotate: Knowledge graph embedding by relational rotation in complex space. In: 7th international conference on learning representations, ICLR 2019

  24. Sun Z, Hu W, Zhang Q, Qu Y (2018) Bootstrapping entity alignment with knowledge graph embedding. In: Proceedings of the 26th international joint conference on artificial intelligence, pp 4396–4402

  25. Toutanova K, Chen D (2015) Observed versus latent features for knowledge base and text inference. In: Proceedings of the 3rd workshop on continuous vector space models and their compositionality

  26. Trouillon T, Welbl J, Riedel S, Gaussier É, Bouchard G (2016) Complex embeddings for simple link prediction. In: Proceedings of the 33th international conference on machine learning, pp 2071–2080

  27. Wang Q, Mao Z, Wang B, Guo L (2017) Knowledge graph embedding: a survey of approaches and applications. IEEE Trans Knowl Data Eng 29:2724–2743

    Article  Google Scholar 

  28. Wang Z, Zhang J, Feng J, Chen Z (2014) Knowledge graph embedding by translating on hyperplanes. In: Proceedings of the 28th AAAI conference on artificial intelligence, pp 1112–1119

  29. Weston J, Chopra S, Bordes A (2015) Memory networks. In: Proceedings of international conference on learning representations (ICLR 2015)

  30. Wu Z, Pan S, Chen F, Long G, Zhang C, Yu PS (2019) A comprehensive survey on graph neural networks. CoRR arXiv:1901.00596

  31. Xie R, Liu Z, Jia J, Luan H, Sun M (2016) Representation learning of knowledge graphs with entity descriptions. In: Proceedings of the 30th AAAI conference on artificial intelligence, pp 2659–2665

  32. Xu J, Qiu X, Chen K, Huang X (2017) Knowledge graph representation with jointly structural and textual encoding. In: Proeedings of the 26th international joint conference on artificial intelligence, pp 1318–1324

  33. Yang B, tau Yih W, He X, Gao J, Deng L (2015) Embedding entities and relations for learning and inference in knowledge bases. In: Proceedings of international conference on learning representations (ICLR 2015)

Download references

Acknowledgements

This research is supported by the National Natural Science Foundation in China (Grant: 61672128) and the Fundamental Research Fund for Central University (Grant: DUT20TD107).

Author information

Authors and Affiliations

  1. School of Software, the Key Laboratory for Ubiquitous Network and Service Software of Liaoning Province, Dalian University of Technology, Dalian, 116023, China

    Kai Wang, Yu Liu & Xiujuan Xu

  2. Department of Computing, Macquarie University, Sydney, NSW, 2109, Australia

    Quan Z. Sheng

Authors
  1. Kai Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yu Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiujuan Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Quan Z. Sheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kai Wang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, K., Liu, Y., Xu, X. et al. Enhancing knowledge graph embedding by composite neighbors for link prediction. Computing 102, 2587–2606 (2020). https://doi.org/10.1007/s00607-020-00842-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00607-020-00842-5

Keywords

Mathematics Subject Classification

Search

Navigation