Abstract
Knowledge graph reasoning is a crucial part of knowledge discovery and knowledge graph completion tasks. The solution based on generative adversarial imitation learning (GAIL) has made great progress in recent researches and solves the problem of relying heavily on the design of the reward function in reinforcement learning-based reasoning methods. However, only the semantic feature is considered in existing GAIL-based methods, which is not enough to assess the quality of reasoning paths. While logical rules contain rich factual logic that can be used for reasoning. Thus, we introduce the first-order predicate logic rule in our model called Rule Injection-based Generative Adversarial Path Reasoning. The key idea is to train the generator to learn reasoning strategies by imitating the demonstration from both semantic and rule levels. Particularly, we design a path discriminator and a logic rule discriminator to distinguish paths respectively from these two levels. Furthermore, both discriminator feedback to the generator a self-adaptively reward by assessing the quality of the generated reasoning path. Extensively experiments on two benchmarks show that our method improves the performance than the state-of-the-art baseline and some cases study also confirmed the explainability of our model.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Change history
10 December 2021
In the originally published version of chapter 27, the name of the author Xiaoying Chen was spelled incorrectly. This has been corrected.
References
Das, R., Dhuliawala, S., Zaheer, M., et al.: Go for a walk and arrive at the answer: reasoning over paths in knowledge bases using reinforcement learning. In: ICLR (2018)
Xian, Y., Fu, Z., et al.: Reinforcement knowledge graph reasoning for explainable recommendation. In: SIGIR, pp. 285–294 (2019)
Wang, J., Wang, Z., Zhang, D., et al.: Combining knowledge with deep convolutional neural networks for short text classification. In: IJCAI, pp. 2015–2921 (2017)
Li, R., Cheng, X.: DIVINE: a generative adversarial imitation learning framework for knowledge graph reasoning. In: EMNLP-IJCNLP, pp. 2642–2651 (2019)
Guo, S., Wang, Q., Wang, L., et al.: Jointly embedding knowledge graphs and logical rules. In: EMNLP, pp. 192–202 (2016)
Hájek, P.: Metamathematics of Fuzzy Logic. Springer, Dordrecht (2013). https://doi.org/10.1007/978-94-011-5300-3
Bordes, A., et al.: Translating embeddings for modeling multi-relational data. In: Advances in Neural Information Processing Systems, pp. 2787–2795 (2013)
Wang, Z., Zhang, J., Feng, J., et al.: Knowledge graph embedding by translating on hyperplanes. In: AAAI, pp. 1112–1119 (2014)
Lin, Y., et al.: Learning entity and relation embeddings for knowledge graph completion. In: AAAI 2015, pp. 2181–2187 (2015)
Nickel, M., Tresp, V., Kriegel, H.P.: A three-way model for collective learning on multi-relational data. In: ICML 2011, pp. 809–816 (2011)
Yang, B., Yih, W.T., He, X., Gao, J., Deng, L.: Embedding entities and relations for learning and inference in knowledge bases. arXiv preprint arXiv:1412.6575 (2014)
Trouillon, T., Welbl, J., Riedel, S., Gaussier, É., Bouchard, G.: Complex embeddings for simple link prediction. In: ICML, pp. 2071–2080 (2016)
Lao, N., Cohen, W.W.: Relational retrieval using a combination of path-constrained random walks. Mach. Learn. 81(1), 53–67 (2010)
Gardner, M., Talukdar, P., Krishnamurthy, J., et al.: Incorporating vector space similarity in random walk inference over knowledge bases. In: EMNLP, pp. 397–406 (2014)
Xiong, W., Hoang, T., Wang, W.Y.: DeepPath: a reinforcement learning method for knowledge graph reasoning. In: EMNLP, pp. 564–573 (2017)
Wang, H., Li, S., Pan, R., Mao, M.: Incorporating graph attention mechanism into knowledge graph reasoning based on deep reinforcement learning. In: EMNLP-IJCNLP, pp. 2623–2631 (2019)
Li, Z., Jin, X., Guan, S., et al.: Path reasoning over knowledge graph: a multi-agent and reinforcement learning based method. In: ICDMW, pp. 929–936 (2018)
Lin, X.V., Socher, R., Xiong, C.: Multi-hop knowledge graph reasoning with reward shaping. In: EMNLP, pp. 3243–3253 (2018)
Qiu, Y., Wang, Y., et al.: Stepwise reasoning for multi-relation question answering over knowledge graph with weak supervision. In: WSDM, pp.474–482 (2020)
Ho, J., Ermon, S.: Generative adversarial imitation learning. In: NIPS, pp. 4565–4573 (2016)
Gulrajani, I., Ahmed, F., Arjovsky, M., et al.: Improved training of Wasserstein GANs. In: NIPS, pp. 5767–5777 (2017)
Williams, R.J.: Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach. Learn. 8(3–4), 229–256 (1992)
Dettmers, T., Minervini, P., Stenetorp, P., Riedel, S.: Convolutional 2D knowledge graph embeddings. In: AAAI, pp. 1811–1818 (2018)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: ICLR (2015)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, S., Chen, X., Xiong, S. (2021). Rule Injection-Based Generative Adversarial Imitation Learning for Knowledge Graph Reasoning. In: Karlapalem, K., et al. Advances in Knowledge Discovery and Data Mining. PAKDD 2021. Lecture Notes in Computer Science(), vol 12714. Springer, Cham. https://doi.org/10.1007/978-3-030-75768-7_27
Download citation
DOI: https://doi.org/10.1007/978-3-030-75768-7_27
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-75767-0
Online ISBN: 978-3-030-75768-7
eBook Packages: Computer ScienceComputer Science (R0)