skip to main content
10.1145/3488560.3498515acmconferencesArticle/Chapter ViewAbstractPublication PageswsdmConference Proceedingsconference-collections
research-article

Reinforcement Learning over Sentiment-Augmented Knowledge Graphs towards Accurate and Explainable Recommendation

Published: 15 February 2022 Publication History

Abstract

Explainable recommendation has gained great attention in recent years. A lot of work in this research line has chosen to use the knowledge graphs (KG) where relations between entities can serve as explanations. However, existing studies have not considered sentiment on relations in KG, although there can be various types of sentiment on relations worth considering (e.g., a user's satisfaction on an item). In this paper, we propose a novel recommendation framework based on KG integrated with sentiment analysis for more accurate recommendation as well as more convincing explanations. To this end, we first construct a Sentiment-Aware Knowledge Graph (namely, SAKG) by analyzing reviews and ratings on items given by users. Then, we perform item recommendation and reasoning over SAKG through our proposed Sentiment-Aware Policy Learning (namely, SAPL) based on a reinforcement learning strategy. To enhance the explainability for end-users, we further developed an interactive user interface presenting textual explanations as well as a collection of reviews related with the discovered sentiment. Experimental results on three real-world datasets verified clear improvements on both the accuracy of recommendation and the quality of explanations.

References

[1]
Qingyao Ai, Vahid Azizi, Xu Chen, and Yongfeng Zhang. 2018. Learning Heterogeneous Knowledge base Embeddings for Explainable Recommendation. Algorithms, Vol. 11, 9 (2018), 137.
[2]
John Brooke et al. 1996. SUS-A Quick and Dirty Usability Scale. Usability Evaluation in Industry, Vol. 189, 194 (1996), 4--7.
[3]
Bruno De Lemos Ribeiro Pinto Cardoso, Gayane Sedrakyan, Francisco Gutié rrez, Denis Parra, Peter Brusilovsky, and Katrien Verbert. 2019. IntersectionExplorer, a Multi-Perspective Approach for Exploring Recommendations. International Journal of Human-Computer Studies, Vol. 121 (2019), 73--92.
[4]
Rose Catherine, Kathryn Mazaitis, Maxine Eské nazi, and William W. Cohen. 2017. Explainable Entity-based Recommendations with Knowledge Graphs. In Workshop Proceedings of the 11th Conference on Recommender Systems (RecSys).
[5]
Dong-Kyu Chae, Jin-Soo Kang, Sang-Wook Kim, and Jung-Tae Lee. 2018. CFGAN: A Generic Collaborative Filtering Framework based on Generative Adversarial Networks. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management (CIKM). ACM, 137--146.
[6]
Dong-Kyu Chae, Sang-Wook Kim, and Jung-Tae Lee. 2019. Autoencoder-based Personalized Ranking Framework Unifying Explicit and Implicit Feedback for Accurate Top-N Recommendation. Knowledge-Based Systems, Vol. 176 (2019), 110--121.
[7]
Li Chen and Feng Wang. 2017. Explaining Recommendations Based on Feature Sentiments in Product Reviews. In Proceedings of the 22nd International Conference on Intelligent User Interfaces (IUI). ACM, 17--28.
[8]
Zhongxia Chen, Xiting Wang, Xing Xie, Tong Wu, Guoqing Bu, Yining Wang, and Enhong Chen. 2019. Co-Attentive Multi-Task Learning for Explainable Recommendation. In Proceedings of the 28th International Joint Conference on Artificial Intelligence (IJCAI). 2137--2143.
[9]
Kyung-Jae Cho, Yeon-Chang Lee, Kyungsik Han, Jaeho Choi, and Sang-Wook Kim. 2019. No, That's Not My Feedback: TV Show Recommendation Using Watchable Interval. In 2019 IEEE 35th International Conference on Data Engineering (ICDE). IEEE, 316--327.
[10]
Yaakov Danone, Tsvi Kuflik, and Osnat Mokryn. 2018. Visualizing Reviews Summaries as a Tool for Restaurants Recommendation. In Proceedings of the 23rd International Conference on Intelligent User Interfaces (IUI). ACM, 607--616.
[11]
Guglielmo Faggioli, Mirko Polato, and Fabio Aiolli. 2019. Tag-Based User Profiling: A Game Theoretic Approach. In Proceedings of the 27th Conference on User Modeling, Adaptation and Personalization (UMAP). ACM, 267--271.
[12]
Jingyue Gao, Xiting Wang, Yasha Wang, and Xing Xie. 2019. Explainable Recommendation through Attentive Multi-View Learning. In Proceedings of the 33rd Conference on Artificial Intelligence (AAAI). AAAI Press, 3622--3629.
[13]
Marko Gasparic, Andrea Janes, Francesco Ricci, and Marco Zanellati. 2017. GUI Design for IDE Command Recommendations. In Proceedings of the 22nd International Conference on Intelligent User Interfaces (IUI). ACM, 595--599.
[14]
Jin Huang, Wayne Xin Zhao, Hongjian Dou, Ji-Rong Wen, and Edward Y Chang. 2018. Improving Sequential Recommendation with Knowledge-Enhanced Memory Networks. In Proceedings of the 41st International Conference on Research & Development in Information Retrieval (SIGIR). ACM, 505--514.
[15]
Bart P. Knijnenburg, Martijn C. Willemsen, Zeno Gantner, Hakan Soncu, and Chris Newell. 2012. Explaining the User Experience of Recommender Systems. User Modeling and User-Adapted Interaction, Vol. 22, 4--5 (2012), 441--504.
[16]
Yunyong Ko, Jae-Seo Yu, Hong-Kyun Bae, Yongjun Park, Dongwon Lee, and Sang-Wook Kim. 2021. MASCOT: A Quantization Framework for Efficient Matrix Factorization in Recommender Systems. In Proceedings of the 21st IEEE International Conference on Data Mining (ICDM). IEEE, 290--299.
[17]
Johannes Kunkel, Benedikt Loepp, and Jü rgen Ziegler. 2017. A 3D Item Space Visualization for Presenting and Manipulating User Preferences in Collaborative Filtering. In Proceedings of the 22nd International Conference on Intelligent User Interfaces (IUI). ACM, 3--15.
[18]
Trung-Hoang Le and Hady W. Lauw. 2020. Synthesizing Aspect-Driven Recommendation Explanations from Reviews. In Proceedings of the 29th International Joint Conference on Artificial Intelligence (IJCAI). 2427--2434.
[19]
Yeon-Chang Lee, Taeho Kim, Jaeho Choi, Xiangnan He, and Sang-Wook Kim. 2021. M-BPR: A Novel Approach to Improving BPR for Recommendation with Multi-Type Pair-Wise Preferences. Information Sciences, Vol. 547 (2021), 255--270.
[20]
Yue Lu, Malú Castellanos, Umeshwar Dayal, and ChengXiang Zhai. 2011. Automatic Construction of a Context-Aware Sentiment Lexicon: An Optimization Approach. In Proceedings of the 20th International Conference on World Wide Web (WWW). ACM, 347--356.
[21]
Julian McAuley and Jure Leskovec. 2013. Hidden Factors and Hidden Topics: Understanding Rating Dimensions with Review Text. In Proceedings of the 7th Conference on Recommender Systems (RecSys). ACM, 165--172.
[22]
Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton Van Den Hengel. 2015. Image-based Recommendations on Styles and Substitutes. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR). 43--52.
[23]
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed Representations of Words and Phrases and Their Compositionality. In Proceedings of the 27th Conference on Artificial Intelligence (AAAI). 3111--3119.
[24]
Martijn Millecamp, Nyi Nyi Htun, Cristina Conati, and Katrien Verbert. 2019. To Explain or Not to Explain: The Effects of Personal Characteristics When Explaining Music Recommendations. In Proceedings of the 24th International Conference on Intelligent User Interfaces (IUI). ACM, 397--407.
[25]
Denis Parra, Peter Brusilovsky, and Christoph Trattner. 2014. See What You Want to See: Visual User-Driven Approach for Hybrid Recommendation. In Proceedings of the 19th International Conference on Intelligent User Interfaces (IUI). ACM, 235--240.
[26]
Pearl Pu, Li Chen, and Rong Hu. 2011. A User-Centric Evaluation Framework for Recommender Systems. In Proceedings of the Fifth ACM Conference on Recommender systems (RecSys). 157--164.
[27]
Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian Personalized Ranking from Implicit Feedback. In Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence (AAAI). 452--461.
[28]
Sungyong Seo, Jing Huang, Hao Yang, and Yan Liu. 2017. Interpretable Convolutional Neural Networks with Dual Local and Global Attention for Review Rating Prediction. In Proceedings of the 11th Conference on Recommender Systems (RecSys). ACM, 297--305.
[29]
Richard S Sutton and Andrew G Barto. 2018. Reinforcement Learning - An Introduction .MIT press, Cambridge, MA, USA.
[30]
Hongwei Wang, Fuzheng Zhang, Jialin Wang, Miao Zhao, Wenjie Li, Xing Xie, and Minyi Guo. 2018b. Ripplenet: Propagating User Preferences on the Knowledge Graph for Recommender Systems. In Proceedings of the 27th International Conference on Information and Knowledge Management (CIKM). ACM, 417--426.
[31]
Nan Wang, Hongning Wang, Yiling Jia, and Yue Yin. 2018a. Explainable Recommendation via Multi-Task Learning in Opinionated Text Data. In Proceedings of the 41st International Conference on Research & Development in Information Retrieval (SIGIR). ACM, 165--174.
[32]
Xiang Wang, Xiangnan He, Yixin Cao, Meng Liu, and Tat-Seng Chua. 2019 a. KGAT: Knowledge Graph Attention Network for Recommendation. In Proceedings of the 25th International Conference on Knowledge Discovery & Data Mining (SIGKDD). ACM, 950--958.
[33]
Xiang Wang, Dingxian Wang, Canran Xu, Xiangnan He, Yixin Cao, and Tat-Seng Chua. 2019 b. Explainable Reasoning over Knowledge Graphs for Recommendation. In Proceedings of the 33rd Conference on Artificial Intelligence (AAAI). 5329--5336.
[34]
Yikun Xian, Zuohui Fu, S Muthukrishnan, Gerard De Melo, and Yongfeng Zhang. 2019. Reinforcement Knowledge Graph Reasoning for Explainable Recommendation. In Proceedings of the 42nd International Conference on Research & Development in Information Retrieval (SIGIR). ACM, 285--294.
[35]
Yongfeng Zhang and Xu Chen. 2020. Explainable Recommendation: A Survey and New Perspectives. Foundations and Trends in Information Retrieval, Vol. 14, 1 (2020), 1--101.
[36]
Yongfeng Zhang, Guokun Lai, Min Zhang, Yi Zhang, Yiqun Liu, and Shaoping Ma. 2014. Explicit Factor Models for Explainable Recommendation based on Phrase-Level Sentiment Analysis. In Proceedings of the 37th International Conference on Research & Development in Information Retrieval (SIGIR). ACM, 83--92.
[37]
Kangzhi Zhao, Xiting Wang, Yuren Zhang, Li Zhao, Zheng Liu, Chunxiao Xing, and Xing Xie. 2020. Leveraging Demonstrations for Reinforcement Recommendation Reasoning over Knowledge Graphs. In Proceedings of the 43rd International Conference on Research & Development in Information Retrieval (SIGIR). ACM, 239--248.
[38]
Qiannan Zhu, Xiaofei Zhou, Jia Wu, Jianlong Tan, and Li Guo. 2020. A Knowledge-Aware Attentional Reasoning Network for Recommendation. In Proceedings of the 34th Conference on Artificial Intelligence (AAAI). AAAI Press, 6999--7006.

Cited By

View all
  • (2025)Explainable Session-Based Recommendation via Path ReasoningIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.348632637:1(278-290)Online publication date: Jan-2025
  • (2024)On the unexpected effectiveness of reinforcement learning for sequential recommendationProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3693918(45432-45450)Online publication date: 21-Jul-2024
  • (2024)ASKAT: Aspect Sentiment Knowledge Graph Attention Network for RecommendationElectronics10.3390/electronics1301021613:1(216)Online publication date: 3-Jan-2024
  • Show More Cited By

Index Terms

  1. Reinforcement Learning over Sentiment-Augmented Knowledge Graphs towards Accurate and Explainable Recommendation

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      WSDM '22: Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining
      February 2022
      1690 pages
      ISBN:9781450391320
      DOI:10.1145/3488560
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 15 February 2022

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. explainable recommendation
      2. knowledge graph
      3. sentiment analysis

      Qualifiers

      • Research-article

      Funding Sources

      • This work was supported by the National Research Foundation of Korea(NRF) grant funded by the Korea government(MSIT).

      Conference

      WSDM '22

      Acceptance Rates

      Overall Acceptance Rate 498 of 2,863 submissions, 17%

      Upcoming Conference

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)225
      • Downloads (Last 6 weeks)12
      Reflects downloads up to 17 Jan 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2025)Explainable Session-Based Recommendation via Path ReasoningIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.348632637:1(278-290)Online publication date: Jan-2025
      • (2024)On the unexpected effectiveness of reinforcement learning for sequential recommendationProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3693918(45432-45450)Online publication date: 21-Jul-2024
      • (2024)ASKAT: Aspect Sentiment Knowledge Graph Attention Network for RecommendationElectronics10.3390/electronics1301021613:1(216)Online publication date: 3-Jan-2024
      • (2024)DiaFrame: A Framework for Understanding Bengali Dialects in Human-AI Collaborative Creative Writing SpacesCompanion Publication of the 2024 Conference on Computer-Supported Cooperative Work and Social Computing10.1145/3678884.3681862(268-274)Online publication date: 11-Nov-2024
      • (2024)Beyond Relevance: Factor-level Causal Explanation for User Travel Decisions with Counterfactual Data AugmentationACM Transactions on Information Systems10.1145/365367342:5(1-31)Online publication date: 29-Apr-2024
      • (2024)Collaborative Sequential Recommendations via Multi-view GNN-transformersACM Transactions on Information Systems10.1145/364943642:6(1-27)Online publication date: 25-Jun-2024
      • (2024)A Survey on Reinforcement Learning for Recommender SystemsIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2023.328016135:10(13164-13184)Online publication date: Oct-2024
      • (2024)Towards Knowledge-Aware and Deep Reinforced Cross-Domain Recommendation Over Collaborative Knowledge GraphIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.339126836:11(7171-7187)Online publication date: Nov-2024
      • (2024)Multimodal Contrastive Transformer for Explainable RecommendationIEEE Transactions on Computational Social Systems10.1109/TCSS.2023.327627311:2(2632-2643)Online publication date: Apr-2024
      • (2024)Recent Developments in Recommender Systems: A Survey [Review Article]IEEE Computational Intelligence Magazine10.1109/MCI.2024.336398419:2(78-95)Online publication date: May-2024
      • Show More Cited By

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media