Skip to main content
SpringerLink
Log in

ConvHiA: convolutional network with hierarchical attention for knowledge graph multi-hop reasoning

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Knowledge graphs can provide a rich resource for constructing question answering systems and recommendation systems. However, most knowledge graphs still encounter knowledge incompleteness. The path-based approach predicts the unknown relation between pairwise entities based on existing path facts. This approach is one of the most promising approaches for knowledge graph completion. A critical challenge of such approaches is integrating path sequence information to achieve the goal of better reasoning. Existing researches focus more on the features between neighboring entities and relations in a path, ignoring the semantic relations of the whole triple. A single path consists of entities and relations, but triples contain valuable semantic information. Moreover, the importance of different triples on each path is disparate. To address these problems, we propose a method convolutional network with hierarchical attention to complete the knowledge graph. Firstly, we use a convolutional network and bidirectional long short-term memory to extract the features of each triple in the path. Then, we employ a novel hierarchical attention network, including triple-level attention and path-level attention, picking up path features at multiple granularities. In addition, we elaborate a multistep reasoning component that repeats multiple interactions with the hierarchical attention module to obtain more plausible inference evidence. Finally, we predict the relation between query entities and provide the most dominant path to explain our answer. The experimental results show that our method outperforms existing approaches by 1–3\(\%\) on four datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Suchanek FM, Kasneci G, Weikum G (2007) Yago: a core of semantic knowledge. In: Proceedings of the 16th international conference on world wide web, pp 697–706

  2. Auer S, Bizer C, Kobilarov G, Lehmann J, Cyganiak R, Ives Z (2007) Dbpedia: a nucleus for a web of open data. In: Proceedings of the 6th international the semantic web and 2nd Asian conference on Asian semantic web conference, 2007, pp 722–735

  3. Miller GA (1995) Wordnet: a lexical database for english. Commun Acm 38(11):39–41

  4. Bollacker K, Evans C, Paritosh P, Sturge T, Taylor J (2008) Freebase: a collaboratively created graph database for structuring human knowledge. In: Proceedings of the 2008 ACM SIGMOD international conference on management of data, pp 1247–1250

  5. Xiong W, Yu M, Chang S, Guo X, Wang WY (2019) Improving question answering over incomplete KBs with knowledge-aware reader. In: Proceedings of the 57th annual meeting of the association for computational linguistics, pp 4258–4264

  6. Fararni KA, Nafis F, Aghoutane B, Yahyaouy A, Riffi J, Sabri A (2021) Hybrid recommender system for tourism based on big data and ai: a conceptual framework, no 1, p 9

  7. Zhang D, Jia Q, Yang S, Han X, Xu C, Liu X, Xie Y (2022) Traditional Chinese medicine automated diagnosis based on knowledge graph reasoning. CMC Comput Mater Continua 71(1):159–170

    Google Scholar 

  8. Nie K, Zeng K, Meng Q (2020) Knowledge reasoning method for military decision support knowledge graph mixing rule and graph neural networks learning together. In: (2020) Chinese automation congress (CAC). IEEE 2020, pp 4013–4018

  9. Bordes A, Usunier N, Garcia-Durán A, Weston J, Yakhnenko O (2013) Translating embeddings for modeling multi-relational data, 2013, pp 2787–2795

  10. Wang Z, Zhang J, Feng J, Chen Z (2014) Knowledge graph embedding by translating on hyperplanes. In: Proceedings of the AAAI conference on artificial intelligence, vol 28, no 1, pp 1112–1119

  11. Yang B, Yih W-t, He X, Gao J, Deng L (2014) Embedding entities and relations for learning and inference in knowledge bases. arXiv:1412.6575

  12. Lin Y, Liu Z, Luan H, Sun M, Rao S, Liu S (2015) Modeling relation paths for representation learning of knowledge bases

  13. Nickel M, Tresp V, Kriegel H-P (2011) A three-way model for collective learning on multi-relational data. In: Proceedings of the 28th international conference on international conference on machine learning, 2011, pp 809–816

  14. Trouillon T, Welbl J, Riedel S, Gaussier éric, Bouchard G (2016) Complex embeddings for simple link prediction, pp 2071–2080

  15. Dettmers T, Minervini P, Stenetorp P, Riedel S (2018) Convolutional 2d knowledge graph embeddings, pp 1811–1818

  16. Balažević I, Allen C, Hospedales TM (2019) Hypernetwork knowledge graph embeddings. In: International conference on artificial neural networks, 2019, pp 553–565

  17. Sun Z, Deng Z-H, Nie J-Y, Tang J (2019) Rotate: knowledge graph embedding by relational rotation in complex space. In: International conference on learning representations

  18. Lao N, Cohen WW (2010) Relational retrieval using a combination of path-constrained random walks. Mach Learn 81(1):53–67

    Article  MathSciNet  MATH  Google Scholar 

  19. Wang Q, Liu J, Luo Y, Wang B, Lin C-Y (2016) Knowledge base completion via coupled path ranking. In: Meeting of the association for computational linguistics, pp 1308–1318

  20. Lin Y, Liu Z, Luan H, Sun M, Rao S, Liu S (2015) Modeling relation paths for representation learning of knowledge bases. Comput Sci 705–714

  21. Neelakantan A, Roth B, McCallum A (2015) Compositional vector space models for knowledge base completion, pp 156–166

  22. Das R, Neelakantan A, Belanger D, Mccallum A (2017) Chains of reasoning over entities, relations, and text using recurrent neural networks. arXiv:1607.01426, pp 132–141

  23. (2020) Path-based reasoning approach for knowledge graph completion using cnn-bilstm with attention mechanism. Expert Syst Appl 142

  24. Xiong W, Hoang T, Wang WY (2017) Deeppath: a reinforcement learning method for knowledge graph reasoning, pp 564–573

  25. Das R, Dhuliawala S, Zaheer M, Vilnis L, Durugkar I, Krishnamurthy A, Smola A, McCallum A (2018) Go for a walk and arrive at the answer: Reasoning over paths in knowledge bases using reinforcement learning. In: International conference on learning representations

  26. Li S, Wang H, Pan R, Mao M (2021) Memorypath: a deep reinforcement learning framework for incorporating memory component into knowledge graph reasoning. Neurocomputing 419:273–286

    Article  Google Scholar 

  27. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. In: Advances in neural information processing systems, pp 5998–6008

  28. Yang Z, Yang D, Dyer C, He X, Smola A, Hovy E (2016) Hierarchical attention networks for document classification. NAACL 2016:1480–1489

    Google Scholar 

  29. Shen Y, Ding N, Zheng H-T, Li Y, Yang M (2020) Modeling relation paths for knowledge graph completion. IEEE Trans Knowl Data Eng 33(11):3607–3617

    Article  Google Scholar 

  30. Li C, Peng X, Zhang S, Peng H, Philip SY, He M, Du L, Wang L (2020) Modeling relation paths for knowledge base completion via joint adversarial training. Knowl Based Syst 201:105865

    Article  Google Scholar 

  31. Wang Y, Xiao W, Tan Z, Zhao X (2021) Caps-owkg: a capsule network model for open-world knowledge graph. Int J Mach Learn Cybern 12(6):1627–1637

    Article  Google Scholar 

  32. Ma T, Lv S, Huang L, Hu S (2021) Hiam: a hierarchical attention based model for knowledge graph multi-hop reasoning. Neural Netw

  33. Gan Z, Cheng Y, Kholy AE, Li L, Liu J, Gao J (2019) Multi-step reasoning via recurrent dual attention for visual dialog. arXiv:1902.00579

  34. Sukhbaatar S, Weston J, Fergus R et al (2015) End-to-end memory networks. Advances in neural information processing systems, vol 28

  35. Kingma D, Ba J (2014) Adam: a method for stochastic optimization. Comput Sci

  36. Toutanova K, Chen D (2015) Observed versus latent features for knowledge base and text inference. In: Proceedings of the 3rd workshop on continuous vector space models and their compositionality, pp 57–66

  37. Mitchell T, Cohen W, Hruschka E, Talukdar P, Yang B, Betteridge J, Carlson A, Dalvi B, Gardner M, Kisiel B et al (2018) Never-ending learning, vol 61, no 5, pp 103–115

  38. Bahdanau D, Cho K, Bengio Y (2016) Neural machine translation by jointly learning to align and translate

  39. Kok S, Domingos P (2007) Statistical predicate invention. Association for Computing Machinery, 2007, pp 433–440

  40. Zhang Y, Yao Q, Shao Y, Chen L (2019) Nscaching: simple and efficient negative sampling for knowledge graph embedding. In: 2019 IEEE 35th international conference on data engineering (ICDE). IEEE, 2019, pp 614–625

  41. Gu W, Gao F, Li R, Zhang J (2021) Learning universal network representation via link prediction by graph convolutional neural network. Soc Comput 2(1):9

    Google Scholar 

  42. Ma G, Yan H, Qian Y, Wang L, Zhao Z (2021) Path-based estimation for link prediction. Int J Mach Learn Cybern 3

Download references

Funding

This paper supported by Shanxi Province key technology and generic technology R &D project (2020XXX007); National Natural Science Foundation of China (62002255); National Major Scientific Research Instrument Development Project (62027819); The General Object of National Natural Science Foundation(62076177); Graduate Education Innovation Project of Shanxi Province (2021Y199).

Author information

Authors and Affiliations

  1. Shanxi, College of Data Science, Taiyuan University of Technology, Taiyuan, 030024, China

    Dengao Li, Shuyi Miao, Yu Zhou, Ding Feng & Xupeng Niu

  2. Shanxi, College of Information and Computer, Taiyuan University of Technology, Taiyuan, 030024, China

    Jumin Zhao

  3. Key Laboratory of Big Data Fusion Analysis and Application of Shanxi Province, Taiyuan, 030024, China

    Dengao Li, Shuyi Miao, Yu Zhou, Ding Feng, Jumin Zhao & Xupeng Niu

  4. Intelligent Perception Engineering Technology Center of Shanxi, Taiyuan, 030024, China

    Dengao Li, Shuyi Miao, Yu Zhou, Ding Feng, Jumin Zhao & Xupeng Niu

  5. College of Mining Engineering, Taiyuan University of Technology, Taiyuan, 030024, China

    Baofeng Zhao

Authors
  1. Dengao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shuyi Miao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Baofeng Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yu Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ding Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jumin Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Xupeng Niu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dengao Li.

Ethics declarations

Conflict of interest

(Check journal-specific guidelines for which heading to use) No conflict of interest

Consent for publication

Yes.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, D., Miao, S., Zhao, B. et al. ConvHiA: convolutional network with hierarchical attention for knowledge graph multi-hop reasoning. Int. J. Mach. Learn. & Cyber. 14, 2301–2315 (2023). https://doi.org/10.1007/s13042-022-01764-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-022-01764-8

Keywords

Search

Navigation