Abstract
Introducing knowledge graphs (KGs) into recommendation systems can improve their performance, while reinforcement learning (RL) methods can help utilize graph data for recommendation. We investigate existing RL-based methods for recommendation on KGs, and find that such approaches do not make full use of information from user reviews. Introducing user reviews into a recommendation system can reveal user preferences more deeply and equip a RL agent with a stronger ability to distinguish users’ preferences for an item or not, which in turn improves the accuracy of recommendation results. We propose Reinforced Knowledge Graph Reasoning with User Reviews (RKGR-UR) by introducing user reviews into a RL-based recommendation model, which combines a rating prediction task to transform predicted ratings into rewards feedback for the RL agent. Experiments on three real datasets demonstrate the effectiveness of our method.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Xian, Y., Fu, Z., Muthukrishnan, S., De Melo, G., Zhang, Y.: Reinforcement knowledge graph reasoning for explainable recommendation. In: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 285–294 (2019)
Park, S.J., Chae, D.K., Bae, H.K., Park, S., Kim, S.W.: Reinforcement learning over sentiment-augmented knowledge graphs towards accurate and explainable recommendation. In: Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, pp. 784–793 (2022)
Wang, X., Liu, K., Wang, D., Wu, L., Fu, Y., Xie, X.: Multi-level recommendation reasoning over knowledge graphs with reinforcement learning. In: Proceedings of the ACM Web Conference 2022, pp. 2098–2108 (2022)
Shani, G., Heckerman, D., Brafman, R.I., Boutilier, C.: An MDP-based recommender system. J. Mach. Learn. Res. 6(9) (2005)
Cui, Z., et al.: Reinforced KGs reasoning for explainable sequential recommendation. World Wide Web 25(2), 631–654 (2022)
Chen, X., Huang, C., Yao, L., Wang, X., Zhang, W., et al.: Knowledge-guided deep reinforcement learning for interactive recommendation. In: 2020 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE (2020)
He, X., et al.: Learning to collaborate in multi-module recommendation via multi-agent reinforcement learning without communication. In: Fourteenth ACM Conference on Recommender Systems, pp. 210–219 (2020)
Song, W., Duan, Z., Yang, Z., Zhu, H., Zhang, M., Tang, J.: Explainable knowledge graph-based recommendation via deep reinforcement learning. arXiv preprint arXiv:1906.09506 (2019)
Srifi, M., Oussous, A., Ait Lahcen, A., Mouline, S.: Recommender systems based on collaborative filtering using review texts-a survey. Information 11(6), 317 (2020)
Zheng, L., Noroozi, V., Yu, P.S.: Joint deep modeling of users and items using reviews for recommendation. In: Proceedings of the Tenth ACM International Conference on Web Search and Data Mining, pp. 425–434 (2017)
Tan, Y., Zhang, M., Liu, Y., Ma, S.: Rating-boosted latent topics: understanding users and items with ratings and reviews. IJCAI 16, 2640–2646 (2016)
Zhang, Y., Lai, G., Zhang, M., Zhang, Y., Liu, Y., Ma, S.: Explicit factor models for explainable recommendation based on phrase-level sentiment analysis. In: Proceedings of the 37th International ACM SIGIR Conference on Research & Development in Information Retrieval, pp. 83–92 (2014)
Konda, V., Tsitsiklis, J.: Actor-critic algorithms. In: Advances in Neural Information Processing Systems 12 (1999)
Sutton, R.S., McAllester, D., Singh, S., Mansour, Y.: Policy gradient methods for reinforcement learning with function approximation. In: Advances in Neural Information Processing Systems 12 (1999)
Koren, Y., Bell, R., Volinsky, C.: Matrix factorization techniques for recommender systems. Computer 42(8), 30–37 (2009)
Rendle, S., Freudenthaler, C., Gantner, Z., Schmidt-Thieme, L.: BPR: Bayesian personalized ranking from implicit feedback. arXiv preprint arXiv:1205.2618 (2012)
Zhang, W., Yuan, Q., Han, J., Wang, J.: Collaborative multi-level embedding learning from reviews for rating prediction. IJCAI 16, 2986–2992 (2016)
Wang, X., He, X., Cao, Y., Liu, M., Chua, T.S.: KGAT: knowledge graph attention network for recommendation. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 950–958 (2019)
Acknowledgments
This work was partially supported by the National Natural Science Foundation of China (No. 61977002) and the State Key Laboratory of Software Development Environment of China (No. SKLSDE-2022ZX-14). The authors of this work take full responsibilities for its content. We thank the anonymous reviewers for their insightful comments and suggestions on this paper.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, S., Ouyang, Y., Liu, Z., He, W., Rong, W., Xiong, Z. (2023). Reinforcement Learning-Based Recommendation with User Reviews on Knowledge Graphs. In: Jin, Z., Jiang, Y., Buchmann, R.A., Bi, Y., Ghiran, AM., Ma, W. (eds) Knowledge Science, Engineering and Management. KSEM 2023. Lecture Notes in Computer Science(), vol 14119. Springer, Cham. https://doi.org/10.1007/978-3-031-40289-0_12
Download citation
DOI: https://doi.org/10.1007/978-3-031-40289-0_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-40288-3
Online ISBN: 978-3-031-40289-0
eBook Packages: Computer ScienceComputer Science (R0)