Skip to main content
SpringerLink
Log in

RLPath: a knowledge graph link prediction method using reinforcement learning based attentive relation path searching and representation learning

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Due to containing rich patterns between entities, relation paths have been widely used in knowledge graph link prediction. The state-of-the-art link prediction methods considering relation paths obtain relation paths by reinforcement learning with an untrainable reward setting, and realize link prediction by path-ranking algorithm (PRA), which ignores information in entities. In this paper, we propose a new link prediction method RLPath to employ information in both relation paths and entities, which alternately trains a reinforcement learning model with a trainable reward setting to search high-quality relation paths, and a translation-based model to realize link prediction. Simultaneously, we propose a novel reward setting for the reinforcement learning model, which shares the parameters with the attention of the translation-based model, so that these parameters can not only measure the contributions of relation paths, but also guide agents to search relation paths that have high contributions for link prediction, forming mutual promotion. In experiments, we compare RLPath with the state-of-the-art link prediction methods. The results show that RLPath has competitive performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Han B, Chen L, Tian X (2018) Knowledge based collection selection for distributed information retrieval. Inf Process Manag 54(1):116–128

    Article  Google Scholar 

  2. Romadhony A, Widyantoro D, Purwarianti A (2019) Utilizing structured knowledge bases in open IE based event template extraction. Appl Intell 49:206–219

    Article  Google Scholar 

  3. Fang Y, Wang H, Zhao L, Yu F, Wang C (2020) Dynamic knowledge graph based fake-review detection. Appl Intell 50(12):4281–4295

    Article  Google Scholar 

  4. Lin L, Liu J, Lv Y, Guo F (2020) A similarity model based on reinforcement local maximum connected same destination structure oriented to disordered fusion of knowledge graphs. Appl Intell 50:2867–2886

    Article  Google Scholar 

  5. Bollacker, K., Evans, C., Paritosh, P., Sturge, T., and Taylor, J. (2008). Freebase: a collaboratively created graph database for structuring human knowledge. In proceedings of the 14th ACM SIGMOD international conference on Management of Data, 1247–1250

  6. Bizer C, Lehmann J, Kobilarov G, Auer S (2009) Dbpedia-a crystallization point for the web of data. J Web Semantics 7(3):154–165

    Article  Google Scholar 

  7. Dong, X., Gabrilovich, E., Heitz, G., Horn, W., and Lao, N. (2014). Knowledge vault: a web-scale approach to probabilistic knowledge fusion. In proceedings of the 20th ACM SIGKDD international conference on knowledge discovery and data mining, 601–610

  8. Bordes, A., Usunier, N., García-Durán, A., and Yakhnenko, O. (2013). Translating Embeddings for Modeling Multi-Relational Data. In Proceedings of the 27th Annual Conference on Neural Information Processing Systems, 2787–2795

  9. Chen, X., Chen, M., Shi, W., Sun, Y., and Zaniolo, C. (2019). Embedding uncertain knowledge graphs. In Proceedings of the 33rd AAAI Conference on Artificial Intelligence, 33(01), 3363–3370

  10. Liu, H., Wu, Y., and Yang, Y. (2017). Analogical Inference for Multi-Relational Embeddings,” In Proceedings of the 34th International Conference on Machine Learning, 70, 2168–2178

  11. Nickel, M., Tresp, V., and Kriegel, H. (2011). A three-way model for collective learning on multi-relational data. In proceedings of the 28th international conference on international conference on machine learning, 809–816

  12. Sun, Z., Deng, Z., Nie, J., and Tang, J. (2019). RotatE: knowledge graph embedding by relational rotation in complex space. In Proceedings of the 7th International Conference on Learning Representations

  13. Lin, Y., Liu, Z., Sun, M., Liu, Y., and Zhu, X. (2015). Learning entity and relation Embeddings for knowledge graph completion. In proceedings of the 29th AAAI conference on artificial intelligence, 2181-2187

  14. Zhang, Z., Cai, J., Zhang, Y., and Wang, J. (2020). Learning hierarchy-aware knowledge graph Embeddings for link prediction. In proceedings of the 34th AAAI conference on artificial intelligence, 3065-3072

  15. Tang X, Chen L, Cui J, Wei B (2019) Knowledge representation learning with entity descriptions, hierarchical types, and textual relations. Inf Process Manag 56(3):809–822

    Article  Google Scholar 

  16. Wang, Z., Zhang, J., Feng, J., and Chen, Z. (2014). Knowledge graph embedding by translating on Hyperplanes. In proceedings of the 28th AAAI conference on artificial intelligence, 1112-1119

  17. Yang, B., Yih, W., He, X., Gao, J., and Deng, L. (2015). Embedding entities and relations for learning and inference in knowledge bases. In Proceedings of the 3rd International Conference on Learning Representation

  18. Gardner, M., Talukdar, P., Krishnamurthy, J., and Mitchell, T. (2014). Incorporating vector space similarity in random walk inference over knowledge bases. In proceedings of the 18th conference on empirical methods in natural language processing, 397–406

  19. Lao N, Cohen W (2010) Relational retrieval using a combination of path-constrained random walks. Mach Learn 81(1):53–67

    Article  MathSciNet  Google Scholar 

  20. Neelakantan, A., Roth, B., McCallum, A. (2015). Compositional vector space models for Knowledge Base completion. Computer Ence, 1–16

  21. Lin, Y., Liu, Z., Luan, H., and Sun, M. (2015). Modeling relation paths for representation learning of knowledge bases. In proceedings of the 19th conference on empirical methods in natural language processing, 705–714

  22. Huang, W., Li, G., and Jin, Z. (2017). Improved Knowledge Base completion by the path-augmented TransR model. Knowledge Science, Engineering and Management, 149–159

  23. Das, R., Dhuliawala, S., Zaheer, M., Vilnis, L., Durugkar, I., Krishnamurthy, A., Smola, A., and McCallum, A. (2018). Go for a walk and arrive at the answer: reasoning over paths in knowledge bases using reinforcement learning. In Proceedings of the 6th International Conference on Learning Representations

  24. Xiong, W., Hoang, T., and Wang, W. (2017). Deeppath: a reinforcement learning method for knowledge graph reasoning. In proceedings of the 21th conference on empiraical methods in natural language processing, 564–573

  25. Li, Z., Jin, X., Guan, S., Wang, Y., and Cheng, X. (2018). Path reasoning over knowledge graph: a multi-agent and reinforcement learning based method. In proceedings of the 18th IEEE international conference on data mining workshops, 929–936

  26. Fu, C., Chen, T., Qu, M., Jin, W., and Ren, X. (2019). Collaborative policy learning for open knowledge graph reasoning. In proceedings of the 23th conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), 2672–2681

  27. Lin, X., Socher, R., and Xiong, C. (2018). Multi-hop knowledge graph reasoning with reward shaping. In proceedings of the 22th conference on empirical methods, 3243–3253

  28. Lv, X., Gu, Y., Han, X., Hou, L., Li, J., and Liu, Z. (2019). Adapting Meta knowledge graph information for multi-hop reasoning over Fewshot relations. In proceedings of the 23th conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), 3367–3372

  29. Shen, Y., Chen, J., Huang, P., Guo, Y., and Gao, J. (2018). M-walk: learning to walk over graphs using Monte Carlo tree search. In proceedings of the 32nd advances in neural information processing systems, 6786-6797

  30. Mnih V, Kavukcuoglu K, Silver D, Rusu A, Veness J, Bellemare M, Graves A, Riedmiller M, Fidjeland A, Ostrovski G (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529–533

    Article  Google Scholar 

  31. Lin, Y., Shen, S., Liu, Z., Luan, H., and Sun, M. (2016). Neural relation extraction with selective attention over instances, in proceedings of the 54th annual meeting of the Association for Computational Linguistics, 2124–2133

  32. Williams R (1992) Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach Learn 8(3–4):229–256

    MATH  Google Scholar 

  33. Kingma, D. and Ba, J. (2014). Adam: a method for stochastic optimization. In Proceedings of the 2nd International Conference on Learning Representations

  34. Toutanova, K., Chen, D., Pantel, P., Poon, H., Choudhury, P., and Gamon, M. (2015). Representing text for joint embedding of text and knowledge bases. In proceedings of the 19th conference on empirical methods in natural language processing, 1499–1509

  35. Théo, T., Johannes, W., Sebastian, R., Éric, G., and Guillaume, B. (2016). Complex Embeddings for simple link prediction. In proceedings of the 33th international conference on machine learning, 2071–2080

  36. Tim, D., Minervini, P., Stenetorp, P., and Riedel, S. (2018). Convolutional 2D knowledge graph Embeddings. In proceedings of the 32th AAAI conference on artificial intelligence, 1811-1818

Download references

Acknowledgments

This work was funded by the National Key Research and Development Program of China (No. 2018YFB0505000) and the Fundamental Research Funds for the Central Universities (No.2020QNA5017).

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Zhejiang University, Hangzhou, 310027, China

    Ling Chen, Jun Cui, Xing Tang & Yuntao Qian

  2. School of Remote Sensing and Information Engineering, Wuhan University, Wuhan, 430079, China

    Yansheng Li & Yongjun Zhang

Authors
  1. Ling Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jun Cui
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xing Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yuntao Qian
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yansheng Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yongjun Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ling Chen.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, ., Cui, J., Tang, X. et al. RLPath: a knowledge graph link prediction method using reinforcement learning based attentive relation path searching and representation learning. Appl Intell 52, 4715–4726 (2022). https://doi.org/10.1007/s10489-021-02672-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02672-0

Keywords

Search

Navigation