Skip to main content
SpringerLink
Log in

Modeling path information for knowledge graph completion

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Knowledge graphs (KGs) store real-world information in the form of graphs consisting of relationships between entities and have been widely used in the Semantic Web community since it is readable by machines. However, most KGs are known to be very incomplete. The issues of structure sparseness and noise paths in large-scale KGs create a substantial barrier to representation learning. In this paper, we propose an Attribute-embodied neural Relation Path Prediction (ARPP) model to predict missing relations between entities in a KG. The ARPP framework leverages both structural information and textual information from the KG to enrich the representation learning and aid in learning more valuable information from noise paths for relation prediction. To handle the overlooked equal path weight distribution issue which hinders the performance of KG completion, our method evaluates the information propagation for the path by mining neighboring nodes. In order to verify the benefits of incorporating structural information and textual information and the effectiveness of path weight re-distribution, we conduct experiments from various aspects to evaluate the quantitative results for link prediction and entity prediction task, the accuracy change caused by the ablation studies, the effectiveness of the entity attribute and entity/sequence attention, the applicability of the proposed method on Knowledge Graph Completion task, and case study. Results demonstrate that the ARPP model significantly outperforms the state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. http://nlp.stanford.edu/data/glove.6B.zip.

References

  1. Bordes A, Usunier N, Garcia-Duran A, Weston J, Yakhnenko O (2013) Translating embeddings for modeling multi-relational data. In: Advances in neural information processing systems, pp. 2787–2795

  2. Bühmann L, Lehmann J (2013) Pattern based knowledge base enrichment. In: International Semantic Web Conference. Springer, pp. 33–48

  3. Das R, Dhuliawala S, Zaheer M, Vilnis L, Durugkar I, Krishnamurthy A, Smola A, McCallum A (2017) Go for a walk and arrive at the answer: reasoning over paths in knowledge bases using reinforcement learning. arXiv preprint arXiv:1711.05851

  4. Das R, Neelakantan A, Belanger D, McCallum A (2016) Chains of reasoning over entities, relations, and text using recurrent neural networks. arXiv preprint arXiv:1607.01426

  5. Deng Y, Li Y, Du N, Fan W, Shen Y, Lei K (2018) When truth discovery meets medical knowledge graph: estimating trustworthiness degree for medical knowledge condition. arXiv preprint arXiv:1809.10404

  6. Gardner M, Mitchell T (2015) Efficient and expressive knowledge base completion using subgraph feature extraction. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1488–1498

  7. Jagvaral Batselem, Lee Wan-Kon, Roh Jae-Seung, Kim Min-Sung, Park Young-Tack (2020) Path-based reasoning approach for knowledge graph completion using cnn-bilstm with attention mechanism. Expert Syst Appl 142:112960

    Article  Google Scholar 

  8. Ji G, He S, Xu L, Liu K, Zhao J (2015) Knowledge graph embedding via dynamic mapping matrix. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), vol 1, pp. 687–696

  9. Guoliang J, Kang L, Shizhu H, Jun Z, Knowledge graph completion with adaptive sparse transfer matrix. In: AAAI, pp. 985–991

  10. Lao Ni, Cohen William W (2010) Relational retrieval using a combination of path constrained random walks. Mach Learn 81(1):53–67

    Article  MathSciNet  Google Scholar 

  11. Lao N, Mitchell T, Cohen William W (2011) Random walk inference and learning in a large scale knowledge base. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, pp. 529–539

  12. Lin Y, Liu Z, Luan H, Sun M, Rao S, Liu S (2015) Modeling relation paths for representation learning of knowledge bases. arXiv preprint arXiv:1506.00379

  13. Lin Z, Feng M, dos Santos Cicero N, Yu M, Xiang B, Zhou B, Bengio Y (2017) A structured self-attentive sentence embedding. arXiv preprint arXiv:1703.03130

  14. Nathani D, Chauhan J, Sharma C, Kaul M (2019) Learning attention-based embeddings for relation prediction in knowledge graphs. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 4710–4723

  15. Neelakantan A, Roth B, McCallum A (2015) Compositional vector space models for knowledge base completion. arXiv preprint arXiv:1504.06662

  16. Nguyen D Q, Sirts K, Qu L, Johnson M (2016) Stranse: a novel embedding model of entities and relationships in knowledge bases. arXiv preprint arXiv:1606.08140

  17. Adam P, Geoff S (2007) First order reasoning on a large ontology. ESARLT 257

  18. Pujara J, Miao H, Getoor L, Cohen W (2013) Knowledge graph identification. In: International Semantic Web Conference. Springer, pp. 542–557

  19. Shen Yelong, Huang P-S, Chang M-W, Gao J (2017) Modeling largescale structured relationships with shared memory for knowledge base completion. In: Proceedings of the 2nd Workshop on Representation Learning for NLP, pp. 57–68

  20. Shen Ying, Chen Daoyuan, Tang Buzhou, Yang Min, Lei Kai (2018) Eapb: entropy-aware pathbased metric for ontology quality. J Biomed Semant 9(1):20

    Article  Google Scholar 

  21. Shen Y, Ding N, Zheng H-T, Li Y, Yang M (2020) Modeling relation paths for knowledge graph completion. IEEE Trans Knowled Data Eng

  22. Shi Baoxu, Weninger Tim (2017) Proje: Embedding projection for knowledge graph completion. AAAI 17:1236–1242

    Google Scholar 

  23. Socher R, Chen D, Manning C D, Ng A (2013) Reasoning with neural tensor networks for knowledge base completion. In: Advances in neural information processing systems, pp. 926–934

  24. Socher R, Huval B, Manning C D, Ng A Y (2012) Semantic compositionality through recursive matrixvector spaces. In: Proceedings of the 2012 joint conference on empirical methods in natural language processing and computational natural language learning. Association for Computational Linguistics, pp. 1201–1211

  25. Sun Z, Deng Z-H, Nie J-Y, Tang J (2019) Rotate: Knowledge graph embedding by relational rotation in complex space. In: International Conference on Learning Representations

  26. Toutanova K, Chen D, Pantel P, Poon H, Choudhury P, Gamon M (2015) Representing text for joint embedding of text and knowledge bases. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1499–1509

  27. Toutanova K, Lin V, Yih W-T, Poon H, Quirk C (2016) Compositional learning of embeddings for relation paths in knowledge base and text. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), vol 1, pp. 1434–1444

  28. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A N, Kaiser Ł, Polosukhin I (2017) Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008

  29. Veličković P, Cucurull G, Casanova A, Romero A, Lio P, Bengio Y (2018) Graph attention networks. In: Proceedings of the International Conference on Learning Representations (ICLR)

  30. Wang Q, Liu J, Luo Y, Wang B, Lin C-Y (2016) Knowledge base completion via coupled path ranking. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), vol 1, pp. 1308–1318

  31. Wang Q, Wang B, Guo L et al (2015) Knowledge base completion using embeddings and rules. In: IJCAI, pp. 1859–1866

  32. Wang Z, Li J-Z (2016) Text-enhanced representation learning for knowledge graph. In: IJCAI, pp. 1293–1299

  33. Wen D, Liu Y, Yuan K, Si S, Shen Y (2017) Attention-aware pathbased relation extraction for medical knowledge graph. In: International Conference on Smart Computing and Communication. Springer, pp. 321–331

  34. West R, Gabrilovich E, Murphy K, Sun S, Gupta R, Lin D (2014) Knowledge base completion via search based question answering. In: Proceedings of the 23rd international conference on World wide web. ACM, pp. 515–526

  35. Xu J, Chen K, Qiu X, Huang X (2016) Knowledge graph representation with jointly structural and textual encoding. arXiv preprint arXiv:1611.08661

Download references

Acknowledgements

This work was financially supported by the National Natural Science Foundation of China (No.61602013) and the Shenzhen General Research Project (No. JCYJ20190808182805919).

Author information

Authors and Affiliations

  1. School of Intelligent Systems Engineering, Sun Yat-Sen University, Guangzhou, China

    Ying Shen & Du Nan

  2. Next Generation Internet Research Institute, Macau University of Science and Technology, Macau, China

    Dagang Li

Authors
  1. Ying Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dagang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Du Nan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ying Shen.

Ethics declarations

Conflict of interest

We wish to confirm that there are no known conflicts of interest associated with this publication and there has been no significant financial support for this work that could have influenced its outcome. We confirm that the manuscript has been read and approved by all named authors and that there are no other persons who satisfied the criteria for authorship but are not listed. We further confirm that the order of authors listed in the manuscript has been approved by all of us.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shen, Y., Li, D. & Nan, D. Modeling path information for knowledge graph completion. Neural Comput & Applic 34, 1951–1961 (2022). https://doi.org/10.1007/s00521-021-06460-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-06460-2

Keywords

Search

Navigation