Xingchen Zhou
ABSTRACT
Some unseen facts in knowledge graphs (KGs) can be complemented by multi-hop knowledge graph reasoning. The process of multi-hop reasoning can be presented as a serialized decision problem, and then be solved with reinforcement learning (RL). However, existing RL-based reasoning methods cannot deal with the hierarchical information in KGs effectively. To solve this issue, we propose a novel RL-based multi-hop KG reasoning model Path Additional Action-space Ranking (PAAR). Our model first proposes a hyperbolic knowledge graph embedding (KGE) model which can effectively capture hierarchical information in KGs. To introduce hierarchical information into RL, the hyperbolic KGE vector is subsequently added to the state space and helps to expand the action space. In order to alleviate the reward sparsity problem in RL, our model utilizes the score function of hyperbolic KGE model as a soft reward. Finally, the metric of hyperbolic space is added to the training of RL as a penalty strategy to constrain the sufficiency of multi-hop reasoning paths. Experimental results on two real-world datasets show that our proposed model not only provides effective answers but also offers sufficient paths.
- M A Abramowicz, I Bengtsson, V Karas, 2002. Poincaré Ball Embeddings of the Optical Geometry. Classical and Quantum Gravity(2002), 3963.Google Scholar
- Sören Auer, Christian Bizer, Georgi Kobilarov, 2007. DBpedia: a Nucleus for a Web of Open Data. In Proceedings of the 6th International Semantic Web Conference, Vol. 4825. 722–735.Google ScholarDigital Library
- Ivana Balažević, Carl Allen, and Timothy Hospedales. 2019. Multi-relational poincaré graph embeddings. In Proceedings of the 33rd International Conference on Neural Information Processing Systems. 4463–4473.Google Scholar
- Kurt D. Bollacker, Colin Evans, Praveen K. Paritosh, 2008. Freebase: a collaboratively created graph database for structuring human knowledge. In Proceedings of the 26th ACM International Conference on Management of Data. 1247–1250.Google ScholarDigital Library
- Silvere Bonnabel. 2013. Stochastic Gradient Descent on Riemannian Manifolds. IEEE Trans. Automat. Control(2013), 2217–2229.Google ScholarCross Ref
- Antoine Bordes, Nicolas Usunier, Alberto García-Durán, 2013. Translating embeddings for modeling multi-relational data. In Proceedings of the 26th International Conference in Neural Information Processing Systems. 2787–2795.Google Scholar
- James W Cannon, William J Floyd, Richard Kenyon, 1997. Hyperbolic Geometry. Flavors of geometry 31(1997), 59–115.Google Scholar
- Ines Chami, Zhitao Ying, Christopher Ré, 2019. Hyperbolic Graph Convolutional Neural Networks. In Proceedings of the 33rd Neural Information Processing Systems. 4869–4880.Google Scholar
- Rajarshi Das, Shehzaad Dhuliawala, Manzil Zaheer, 2018. Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases using Reinforcement Learning. In Proceedings of the 6th International Conference on Learning Representations.Google Scholar
- Rajarshi Das, Arvind Neelakantan, David Belanger, 2017. Chains of reasoning over entities, relations, and text using recurrent neural networks. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics. 132–141.Google ScholarCross Ref
- Hao Fei, Yafeng Ren, Yue Zhang, 2021. Enriching contextualized language model from knowledge graph for biomedical information extraction. Briefings in Bioinformatics(2021), bbaa110.Google Scholar
- Cong Fu, Tong Chen, Meng Qu, 2019. Collaborative Policy Learning for Open Knowledge Graph Reasoning. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. 2672–2681.Google ScholarCross Ref
- Ni Lao, Einat Minkov, and William W. Cohen. 2015. Learning relational features with backward random walks. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. 666–675.Google ScholarCross Ref
- Ni Lao, Tom M. Mitchell, and William W. Cohen. 2011. Random walk inference and learning in a large scale knowledge base. In Proceedings of the 16th Conference on Empirical Methods in Natural Language Processing. 529–539.Google ScholarDigital Library
- Xi Victoria Lin, Richard Socher, and Caiming Xiong. 2018. Multi-Hop Knowledge Graph Reasoning with Reward Shaping. In Proceedings of the 23rd Conference on Empirical Methods in Natural Language Processing. 3243–3253.Google ScholarCross Ref
- Federico López, Benjamin Heinzerling, and Michael Strube. 2019. Fine-Grained Entity Typing in Hyperbolic Space. In Proceedings of the 4th Workshop on Representation Learning for NLP. 169–180.Google ScholarCross Ref
- Xin Lv, Xu Han, Lei Hou, 2020. Dynamic Anticipation and Completion for Multi-Hop Reasoning over Sparse Knowledge Graph. In Proceedings of the 25th Conference on Empirical Methods in Natural Language Processing. 5694–5703.Google ScholarCross Ref
- Farzaneh Mahdisoltani, Joanna Biega, and Fabian M. Suchanek. 2014. Yago3: A knowledge base from multilingual wikipedias. In Proceedings of the 7th biennial conference on innovative data systems research.Google Scholar
- Tom Mitchell and Edward Fredkin. 2014. Never ending language learning. In Proceedings of the IEEE International Conference on Big Data. 1–1.Google ScholarCross Ref
- Arvind Neelakantan, Benjamin Roth, and Andrew McCallum. 2015. Compositional vector space models for knowledge base completion. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. 156–166.Google ScholarCross Ref
- Tim Rocktäschel and Sebastian Riedel. 2017. End-to-end Differentiable Proving. In Proceedings of the 30th International Conference on Neural Information Processing Systems. 3788–3800.Google Scholar
- Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, 2019. RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space. In Proceedings of the 7th International Conference on Learning Representations.Google Scholar
- Quan Wang, Jing Liu, Yuanfei Luo, 2016. Knowledge base completion via coupled path ranking. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistic. 1308–1318.Google ScholarCross Ref
- Zhen Wang, Jianwen Zhang, Jianlin Feng, 2014. Knowledge graph embedding by translating on hyperplanes. In Proceedings of the 28th AAAI Conference on Artificial Intelligence. 1112–1119.Google ScholarCross Ref
- Yongliang Wu and Shuliang Zhao. 2021. Community answer generation based on knowledge graph. Information Sciences(2021), 132–152.Google Scholar
- Lijie Xie, Zhaoming Hu, Xingjuan Cai, 2021. Explainable recommendation based on knowledge graph and multi-objective optimization. Complex & Intelligent Systems(2021), 1241–1252.Google Scholar
- Wenhan Xiong, Thien Hoang, and William Yang Wang. 2017. Deeppath: a reinforcement learning method for knowledge graph reasoning. In Proceedings of the 22nd Conference on Empirical Methods in Natural Language Processing. 564–573.Google ScholarCross Ref
- Fan Yang, Zhilin Yang, and William W. Cohen. 2017. Differentiable learning of logical rules for knowledge base reasoning. In Proceedings of the 31st International Conference on Neural Information Processing Systems. 2316–2325.Google Scholar
Index Terms
- Multi-hop Knowledge Graph Reasoning Based on Hyperbolic Knowledge Graph Embedding and Reinforcement Learning
Recommendations
EN-DIVINE: An Enhanced Generative Adversarial Imitation Learning Framework for Knowledge Graph Reasoning
Knowledge Science, Engineering and ManagementAbstractKnowledge Graphs (KGs) are often incomplete and sparse. Knowledge graph reasoning aims at completing the KG by predicting missing paths between entities. The reinforcement learning (RL) based method is one of the state-of-the-art approaches to ...
Attention Based Reinforcement Learning with Reward Shaping for Knowledge Graph Reasoning
Natural Language Processing and Chinese ComputingAbstractKnowledge graph reasoning aims at solving certain tasks by finding reasoning paths, which has aroused extensive attention. Recently, a solution for path reasoning that combines reinforcement learning has achieved successful progress. But these ...
Explainable Knowledge Reasoning Framework Using Multiple Knowledge Graph Embedding
IJCKG '21: Proceedings of the 10th International Joint Conference on Knowledge GraphsKnowledge reasoning using knowledge graphs has attracted much attention. However, there is difficulty in integrating various related works to realize complex reasoning with explanation using multiple knowledge graphs. To do this, I propose a reasoning ...
Comments