Skip to main content

Rule Injection-Based Generative Adversarial Imitation Learning for Knowledge Graph Reasoning

  • Conference paper
  • First Online:
Advances in Knowledge Discovery and Data Mining (PAKDD 2021)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 12714))

Included in the following conference series:

  • 1962 Accesses

  • The original version of this chapter was revised: A spelling mistake in the name of the author Xiaoying Chen has been corrected. The correction to this chapter is available at https://doi.org/10.1007/978-3-030-75768-7_34

Abstract

Knowledge graph reasoning is a crucial part of knowledge discovery and knowledge graph completion tasks. The solution based on generative adversarial imitation learning (GAIL) has made great progress in recent researches and solves the problem of relying heavily on the design of the reward function in reinforcement learning-based reasoning methods. However, only the semantic feature is considered in existing GAIL-based methods, which is not enough to assess the quality of reasoning paths. While logical rules contain rich factual logic that can be used for reasoning. Thus, we introduce the first-order predicate logic rule in our model called Rule Injection-based Generative Adversarial Path Reasoning. The key idea is to train the generator to learn reasoning strategies by imitating the demonstration from both semantic and rule levels. Particularly, we design a path discriminator and a logic rule discriminator to distinguish paths respectively from these two levels. Furthermore, both discriminator feedback to the generator a self-adaptively reward by assessing the quality of the generated reasoning path. Extensively experiments on two benchmarks show that our method improves the performance than the state-of-the-art baseline and some cases study also confirmed the explainability of our model.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Change history

  • 10 December 2021

    In the originally published version of chapter 27, the name of the author Xiaoying Chen was spelled incorrectly. This has been corrected.

References

  1. Das, R., Dhuliawala, S., Zaheer, M., et al.: Go for a walk and arrive at the answer: reasoning over paths in knowledge bases using reinforcement learning. In: ICLR (2018)

    Google Scholar 

  2. Xian, Y., Fu, Z., et al.: Reinforcement knowledge graph reasoning for explainable recommendation. In: SIGIR, pp. 285–294 (2019)

    Google Scholar 

  3. Wang, J., Wang, Z., Zhang, D., et al.: Combining knowledge with deep convolutional neural networks for short text classification. In: IJCAI, pp. 2015–2921 (2017)

    Google Scholar 

  4. Li, R., Cheng, X.: DIVINE: a generative adversarial imitation learning framework for knowledge graph reasoning. In: EMNLP-IJCNLP, pp. 2642–2651 (2019)

    Google Scholar 

  5. Guo, S., Wang, Q., Wang, L., et al.: Jointly embedding knowledge graphs and logical rules. In: EMNLP, pp. 192–202 (2016)

    Google Scholar 

  6. Hájek, P.: Metamathematics of Fuzzy Logic. Springer, Dordrecht (2013). https://doi.org/10.1007/978-94-011-5300-3

    Book  MATH  Google Scholar 

  7. Bordes, A., et al.: Translating embeddings for modeling multi-relational data. In: Advances in Neural Information Processing Systems, pp. 2787–2795 (2013)

    Google Scholar 

  8. Wang, Z., Zhang, J., Feng, J., et al.: Knowledge graph embedding by translating on hyperplanes. In: AAAI, pp. 1112–1119 (2014)

    Google Scholar 

  9. Lin, Y., et al.: Learning entity and relation embeddings for knowledge graph completion. In: AAAI 2015, pp. 2181–2187 (2015)

    Google Scholar 

  10. Nickel, M., Tresp, V., Kriegel, H.P.: A three-way model for collective learning on multi-relational data. In: ICML 2011, pp. 809–816 (2011)

    Google Scholar 

  11. Yang, B., Yih, W.T., He, X., Gao, J., Deng, L.: Embedding entities and relations for learning and inference in knowledge bases. arXiv preprint arXiv:1412.6575 (2014)

  12. Trouillon, T., Welbl, J., Riedel, S., Gaussier, É., Bouchard, G.: Complex embeddings for simple link prediction. In: ICML, pp. 2071–2080 (2016)

    Google Scholar 

  13. Lao, N., Cohen, W.W.: Relational retrieval using a combination of path-constrained random walks. Mach. Learn. 81(1), 53–67 (2010)

    Article  MathSciNet  Google Scholar 

  14. Gardner, M., Talukdar, P., Krishnamurthy, J., et al.: Incorporating vector space similarity in random walk inference over knowledge bases. In: EMNLP, pp. 397–406 (2014)

    Google Scholar 

  15. Xiong, W., Hoang, T., Wang, W.Y.: DeepPath: a reinforcement learning method for knowledge graph reasoning. In: EMNLP, pp. 564–573 (2017)

    Google Scholar 

  16. Wang, H., Li, S., Pan, R., Mao, M.: Incorporating graph attention mechanism into knowledge graph reasoning based on deep reinforcement learning. In: EMNLP-IJCNLP, pp. 2623–2631 (2019)

    Google Scholar 

  17. Li, Z., Jin, X., Guan, S., et al.: Path reasoning over knowledge graph: a multi-agent and reinforcement learning based method. In: ICDMW, pp. 929–936 (2018)

    Google Scholar 

  18. Lin, X.V., Socher, R., Xiong, C.: Multi-hop knowledge graph reasoning with reward shaping. In: EMNLP, pp. 3243–3253 (2018)

    Google Scholar 

  19. Qiu, Y., Wang, Y., et al.: Stepwise reasoning for multi-relation question answering over knowledge graph with weak supervision. In: WSDM, pp.474–482 (2020)

    Google Scholar 

  20. Ho, J., Ermon, S.: Generative adversarial imitation learning. In: NIPS, pp. 4565–4573 (2016)

    Google Scholar 

  21. Gulrajani, I., Ahmed, F., Arjovsky, M., et al.: Improved training of Wasserstein GANs. In: NIPS, pp. 5767–5777 (2017)

    Google Scholar 

  22. Williams, R.J.: Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach. Learn. 8(3–4), 229–256 (1992)

    MATH  Google Scholar 

  23. Dettmers, T., Minervini, P., Stenetorp, P., Riedel, S.: Convolutional 2D knowledge graph embeddings. In: AAAI, pp. 1811–1818 (2018)

    Google Scholar 

  24. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: ICLR (2015)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wang, S., Chen, X., Xiong, S. (2021). Rule Injection-Based Generative Adversarial Imitation Learning for Knowledge Graph Reasoning. In: Karlapalem, K., et al. Advances in Knowledge Discovery and Data Mining. PAKDD 2021. Lecture Notes in Computer Science(), vol 12714. Springer, Cham. https://doi.org/10.1007/978-3-030-75768-7_27

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-75768-7_27

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-75767-0

  • Online ISBN: 978-3-030-75768-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics