Abstract
Due to containing rich patterns between entities, relation paths have been widely used in knowledge graph link prediction. The state-of-the-art link prediction methods considering relation paths obtain relation paths by reinforcement learning with an untrainable reward setting, and realize link prediction by path-ranking algorithm (PRA), which ignores information in entities. In this paper, we propose a new link prediction method RLPath to employ information in both relation paths and entities, which alternately trains a reinforcement learning model with a trainable reward setting to search high-quality relation paths, and a translation-based model to realize link prediction. Simultaneously, we propose a novel reward setting for the reinforcement learning model, which shares the parameters with the attention of the translation-based model, so that these parameters can not only measure the contributions of relation paths, but also guide agents to search relation paths that have high contributions for link prediction, forming mutual promotion. In experiments, we compare RLPath with the state-of-the-art link prediction methods. The results show that RLPath has competitive performance.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Han B, Chen L, Tian X (2018) Knowledge based collection selection for distributed information retrieval. Inf Process Manag 54(1):116–128
Romadhony A, Widyantoro D, Purwarianti A (2019) Utilizing structured knowledge bases in open IE based event template extraction. Appl Intell 49:206–219
Fang Y, Wang H, Zhao L, Yu F, Wang C (2020) Dynamic knowledge graph based fake-review detection. Appl Intell 50(12):4281–4295
Lin L, Liu J, Lv Y, Guo F (2020) A similarity model based on reinforcement local maximum connected same destination structure oriented to disordered fusion of knowledge graphs. Appl Intell 50:2867–2886
Bollacker, K., Evans, C., Paritosh, P., Sturge, T., and Taylor, J. (2008). Freebase: a collaboratively created graph database for structuring human knowledge. In proceedings of the 14th ACM SIGMOD international conference on Management of Data, 1247–1250
Bizer C, Lehmann J, Kobilarov G, Auer S (2009) Dbpedia-a crystallization point for the web of data. J Web Semantics 7(3):154–165
Dong, X., Gabrilovich, E., Heitz, G., Horn, W., and Lao, N. (2014). Knowledge vault: a web-scale approach to probabilistic knowledge fusion. In proceedings of the 20th ACM SIGKDD international conference on knowledge discovery and data mining, 601–610
Bordes, A., Usunier, N., García-Durán, A., and Yakhnenko, O. (2013). Translating Embeddings for Modeling Multi-Relational Data. In Proceedings of the 27th Annual Conference on Neural Information Processing Systems, 2787–2795
Chen, X., Chen, M., Shi, W., Sun, Y., and Zaniolo, C. (2019). Embedding uncertain knowledge graphs. In Proceedings of the 33rd AAAI Conference on Artificial Intelligence, 33(01), 3363–3370
Liu, H., Wu, Y., and Yang, Y. (2017). Analogical Inference for Multi-Relational Embeddings,” In Proceedings of the 34th International Conference on Machine Learning, 70, 2168–2178
Nickel, M., Tresp, V., and Kriegel, H. (2011). A three-way model for collective learning on multi-relational data. In proceedings of the 28th international conference on international conference on machine learning, 809–816
Sun, Z., Deng, Z., Nie, J., and Tang, J. (2019). RotatE: knowledge graph embedding by relational rotation in complex space. In Proceedings of the 7th International Conference on Learning Representations
Lin, Y., Liu, Z., Sun, M., Liu, Y., and Zhu, X. (2015). Learning entity and relation Embeddings for knowledge graph completion. In proceedings of the 29th AAAI conference on artificial intelligence, 2181-2187
Zhang, Z., Cai, J., Zhang, Y., and Wang, J. (2020). Learning hierarchy-aware knowledge graph Embeddings for link prediction. In proceedings of the 34th AAAI conference on artificial intelligence, 3065-3072
Tang X, Chen L, Cui J, Wei B (2019) Knowledge representation learning with entity descriptions, hierarchical types, and textual relations. Inf Process Manag 56(3):809–822
Wang, Z., Zhang, J., Feng, J., and Chen, Z. (2014). Knowledge graph embedding by translating on Hyperplanes. In proceedings of the 28th AAAI conference on artificial intelligence, 1112-1119
Yang, B., Yih, W., He, X., Gao, J., and Deng, L. (2015). Embedding entities and relations for learning and inference in knowledge bases. In Proceedings of the 3rd International Conference on Learning Representation
Gardner, M., Talukdar, P., Krishnamurthy, J., and Mitchell, T. (2014). Incorporating vector space similarity in random walk inference over knowledge bases. In proceedings of the 18th conference on empirical methods in natural language processing, 397–406
Lao N, Cohen W (2010) Relational retrieval using a combination of path-constrained random walks. Mach Learn 81(1):53–67
Neelakantan, A., Roth, B., McCallum, A. (2015). Compositional vector space models for Knowledge Base completion. Computer Ence, 1–16
Lin, Y., Liu, Z., Luan, H., and Sun, M. (2015). Modeling relation paths for representation learning of knowledge bases. In proceedings of the 19th conference on empirical methods in natural language processing, 705–714
Huang, W., Li, G., and Jin, Z. (2017). Improved Knowledge Base completion by the path-augmented TransR model. Knowledge Science, Engineering and Management, 149–159
Das, R., Dhuliawala, S., Zaheer, M., Vilnis, L., Durugkar, I., Krishnamurthy, A., Smola, A., and McCallum, A. (2018). Go for a walk and arrive at the answer: reasoning over paths in knowledge bases using reinforcement learning. In Proceedings of the 6th International Conference on Learning Representations
Xiong, W., Hoang, T., and Wang, W. (2017). Deeppath: a reinforcement learning method for knowledge graph reasoning. In proceedings of the 21th conference on empiraical methods in natural language processing, 564–573
Li, Z., Jin, X., Guan, S., Wang, Y., and Cheng, X. (2018). Path reasoning over knowledge graph: a multi-agent and reinforcement learning based method. In proceedings of the 18th IEEE international conference on data mining workshops, 929–936
Fu, C., Chen, T., Qu, M., Jin, W., and Ren, X. (2019). Collaborative policy learning for open knowledge graph reasoning. In proceedings of the 23th conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), 2672–2681
Lin, X., Socher, R., and Xiong, C. (2018). Multi-hop knowledge graph reasoning with reward shaping. In proceedings of the 22th conference on empirical methods, 3243–3253
Lv, X., Gu, Y., Han, X., Hou, L., Li, J., and Liu, Z. (2019). Adapting Meta knowledge graph information for multi-hop reasoning over Fewshot relations. In proceedings of the 23th conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), 3367–3372
Shen, Y., Chen, J., Huang, P., Guo, Y., and Gao, J. (2018). M-walk: learning to walk over graphs using Monte Carlo tree search. In proceedings of the 32nd advances in neural information processing systems, 6786-6797
Mnih V, Kavukcuoglu K, Silver D, Rusu A, Veness J, Bellemare M, Graves A, Riedmiller M, Fidjeland A, Ostrovski G (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529–533
Lin, Y., Shen, S., Liu, Z., Luan, H., and Sun, M. (2016). Neural relation extraction with selective attention over instances, in proceedings of the 54th annual meeting of the Association for Computational Linguistics, 2124–2133
Williams R (1992) Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach Learn 8(3–4):229–256
Kingma, D. and Ba, J. (2014). Adam: a method for stochastic optimization. In Proceedings of the 2nd International Conference on Learning Representations
Toutanova, K., Chen, D., Pantel, P., Poon, H., Choudhury, P., and Gamon, M. (2015). Representing text for joint embedding of text and knowledge bases. In proceedings of the 19th conference on empirical methods in natural language processing, 1499–1509
Théo, T., Johannes, W., Sebastian, R., Éric, G., and Guillaume, B. (2016). Complex Embeddings for simple link prediction. In proceedings of the 33th international conference on machine learning, 2071–2080
Tim, D., Minervini, P., Stenetorp, P., and Riedel, S. (2018). Convolutional 2D knowledge graph Embeddings. In proceedings of the 32th AAAI conference on artificial intelligence, 1811-1818
Acknowledgments
This work was funded by the National Key Research and Development Program of China (No. 2018YFB0505000) and the Fundamental Research Funds for the Central Universities (No.2020QNA5017).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Chen, ., Cui, J., Tang, X. et al. RLPath: a knowledge graph link prediction method using reinforcement learning based attentive relation path searching and representation learning. Appl Intell 52, 4715–4726 (2022). https://doi.org/10.1007/s10489-021-02672-0
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-021-02672-0