Ali Montazeralghaem
James Allan
ABSTRACT
Conversational search and recommendation systems can ask clarifying questions through the conversation and collect valuable information from users. However, an important question remains: how can we extract relevant information from the user's utterances and use it in the retrieval or recommendation in the next turn of the conversation? Utilizing relevant information from users' utterances leads the system to better results at the end of the conversation. In this paper, we propose a model based on reinforcement learning, namely RelInCo, which takes the user's utterances and the context of the conversation and classifies each word in the user's utterances as belonging to the relevant or non-relevant class. RelInCo uses two Actors: 1) Arrangement-Actor, which finds the most relevant order of words in user's utterances, and 2) Selector-Actor, which determines which words, in the order provided by the arrangement Actor, can bring the system closer to the target of the conversation. In this way, we can find relevant information in the user's utterance and use it in the conversation. The objective function in our model is designed in such a way that it can maximize any desired retrieval and recommendation metrics (i.e., the ultimate
- Qingyao Ai, Yongfeng Zhang, Keping Bi, Xu Chen, and W Bruce Croft. 2017. Learning a hierarchical embedding model for personalized product search. In SIGIR'17. 645--654.Google Scholar
Digital Library
- Dzmitry Bahdanau, Philemon Brakel, Kelvin Xu, Anirudh Goyal, Ryan Lowe, Joelle Pineau,Aaron Courville, and Yoshua Bengio. 2016. An actor-critic algorithm for sequence prediction. arXiv preprint arXiv:1607.07086 (2016).Google Scholar
- Nicholas J Belkin, Colleen Cool, Adelheit Stein, and Ulrich Thiel. 1995. Cases, scripts, and information-seeking strategies: On the design of interactive information retrieval systems. Expert systems with applications 9, 3 (1995), 379--395.Google Scholar
- Richard Bellman. 1966. Dynamic programming. Science 153, 3731 (1966), 34--37.Google Scholar
- Keping Bi, Qingyao Ai, Yongfeng Zhang, andWBruce Croft. 2019. Conversational product search based on negative feedback. In CIKM'19. 359--368.Google ScholarDigital Library
- Christian Bizer, Jens Lehmann, Georgi Kobilarov, Sören Auer, Christian Becker, Richard Cyganiak, and Sebastian Hellmann. 2009. Dbpedia-a crystallization point for the web of data. Journal of web semantics 7, 3 (2009), 154--165.Google ScholarDigital Library
- Qibin Chen, Junyang Lin, Yichang Zhang, Ming Ding, Yukuo Cen, Hongxia Yang, and Jie Tang. 2019. Towards knowledge-based recommender dialog system. arXiv preprint arXiv:1908.05391 (2019).Google Scholar
- W Bruce Croft and Roger H Thompson. 1987. I3R: A new approach to the design of document retrieval systems. Journal of the american society for information science 38, 6 (1987), 389--404.Google ScholarCross Ref
- Fernando Diaz. 2015. Condensed list relevance models. In ICTIR'15. 313--316.Google ScholarDigital Library
- Junhua He, Hankz Hankui Zhuo, and Jarvan Law. 2017. Distributedrepresentation based hybrid recommender system with short item descriptions. arXiv preprint arXiv:1703.04854 (2017).Google Scholar
- Ayyoob Imani, Amir Vakili, Ali Montazer, and Azadeh Shakery. 2019. An Axiomatic Study of Query Terms Order in Ad-hoc Retrieval. In ECIR'19. Springer, 196--202.Google Scholar
- Kalervo Järvelin and Jaana Kekäläinen. 2002. Cumulated gain-based evaluation of IR techniques. TOIS'02 20, 4 (2002), 422--446.Google ScholarDigital Library
- Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).Google Scholar
- Vijay R Konda and John N Tsitsiklis. 2000. Actor-critic algorithms. In NIPS'00. 1008--1014.Google Scholar
- Wenqiang Lei, Xiangnan He, Yisong Miao, Qingyun Wu, Richang Hong, Min- Yen Kan, and Tat-Seng Chua. 2020. Estimation-action-reflection: Towards deep interaction between conversational and recommender systems. In WSDM'20. 304--312.Google ScholarDigital Library
- Raymond Li, Samira Ebrahimi Kahou, Hannes Schulz, Vincent Michalski, Laurent Charlin, and Chris Pal. 2018. Towards deep conversational recommendations. NIPS'18 31 (2018).Google Scholar
- Lizi Liao, Ryuichi Takanobu, Yunshan Ma, Xun Yang, Minlie Huang, and Tat-Seng Chua. 2019. Deep conversational recommender in travel. arXiv preprint arXiv:1907.00710 (2019).Google Scholar
- Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. NIPS'13 26 (2013).Google Scholar
- Ali Montazeralghaem and James Allan. 2022. Learning Relevant Questions for Conversational Product Search using Deep Reinforcement Learning. In WSDM'22. 746--754.Google ScholarDigital Library
- Ali Montazeralghaem, James Allan, and Philip S Thomas. 2021. Large-scale Interactive Conversational Recommendation System using Actor-Critic Framework. In RecSys'21. 220--229.Google Scholar
- Ali Montazeralghaem, Razieh Rahimi, and James Allan. 2020. Relevance Ranking Based on Query-Aware Context Analysis. ECIR'20 12035 (2020), 446.Google Scholar
- Ali Montazeralghaem, Hamed Zamani, and James Allan. 2020. A reinforcement learning framework for relevance feedback. In SIGIR'20. 59--68.Google ScholarDigital Library
- Ali Montazeralghaem, Hamed Zamani, and Azadeh Shakery. 2016. Axiomatic analysis for improving the log-logistic feedback model. In SIGIR'16. 765--768.Google ScholarDigital Library
- Ali Montazeralghaem, Hamed Zamani, and Azadeh Shakery. 2017. Term proximity constraints for pseudo-relevance feedback. In SIGIR'17. 1085--1088.Google ScholarDigital Library
- Seungwhan Moon, Pararth Shah, Anuj Kumar, and Rajen Subba. 2019. Opendialkg: Explainable conversational reasoning with attention-based walks over knowledge graphs. In ACL'19. 845--854.Google ScholarCross Ref
- Romain Paulus, Caiming Xiong, and Richard Socher. 2017. A deep reinforced model for abstractive summarization. arXiv preprint arXiv:1705.04304 (2017).Google Scholar
- Ivaylo Popov, Nicolas Heess, Timothy Lillicrap, Roland Hafner, Gabriel Barth-Maron, Matej Vecerik, Thomas Lampe, Yuval Tassa, Tom Erez, and Martin Riedmiller. 2017. Data-efficient deep reinforcement learning for dexterous manipulation. arXiv preprint arXiv:1704.03073 (2017).Google Scholar
- Filip Radlinski and Nick Craswell. 2017. A theoretical framework for conversational search. In CHIIR'17. 117--126.Google ScholarDigital Library
- Razieh Rahimi, Ali Montazeralghaem, and James Allan. 2019. Listwise neural ranking models. In ICTIR'19. 101--104.Google ScholarDigital Library
- Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084 (2019).Google Scholar
- Steven J Rennie, Etienne Marcheret, Youssef Mroueh, Jerret Ross, and Vaibhava Goel. 2017. Self-critical sequence training for image captioning. In CVPR'17. 7008--7024.Google ScholarCross Ref
- David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. 1986. Learning representations by back-propagating errors. nature 323, 6088 (1986), 533.Google Scholar
- John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017).Google Scholar
- David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al. 2016. Mastering the game of Go with deep neural networks and tree search. nature 529, 7587 (2016), 484.Google Scholar
- David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, et al. 2017. Mastering the game of go without human knowledge. Nature 550, 7676 (2017), 354--359.Google Scholar
- Yueming Sun and Yi Zhang. 2018. Conversational recommender system. In SIGIR'18. 235--244.Google ScholarDigital Library
- Richard S Sutton and Andrew G Barto. 2018. Reinforcement learning: An introduction. MIT press.Google ScholarDigital Library
- Richard S Sutton, Andrew G Barto, et al. 1998. Introduction to reinforcement learning. Vol. 135. MIT press Cambridge.Google Scholar
- Lakshmi Vikraman, Ali Montazeralghaem, Helia Hashemi, W Bruce Croft, and James Allan. 2021. Passage Similarity and Diversification in Non-factoid Question Answering. In ICTIR'21. 271--280.Google Scholar
- Ronald J Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning 8, 3--4 (1992), 229--256.Google Scholar
- Liu Yang, Hamed Zamani, Yongfeng Zhang, Jiafeng Guo, and W Bruce Croft. 2017. Neural matching models for question retrieval and next question prediction in conversation. arXiv preprint arXiv:1707.05409 (2017).Google Scholar
- Xiaoying Zhang, Hong Xie, Hang Li, and John CS Lui. 2019. Toward building conversational recommender systems: A contextual bandit approach. arXiv preprint arXiv:1906.01219 (2019).Google Scholar
- Yongfeng Zhang, Xu Chen, Qingyao Ai, Liu Yang, and W Bruce Croft. 2018. Towards conversational search and recommendation: System ask, user respond. In CIKM'18. 177--186.Google ScholarDigital Library
- Xiangyu Zhao, Long Xia, Liang Zhang, Zhuoye Ding, Dawei Yin, and Jiliang Tang. 2018. Deep reinforcement learning for page-wise recommendations. In RecSys'18. 95--103.Google Scholar
- Kun Zhou, Xiaolei Wang, Yuanhang Zhou, Chenzhan Shang, Yuan Cheng, Wayne Xin Zhao, Yaliang Li, and Ji-Rong Wen. 2021. CRSLab: An Open-Source Toolkit for Building Conversational Recommender System. arXiv preprint arXiv:2101.00939 (2021).Google Scholar
- Kun Zhou, Wayne Xin Zhao, Shuqing Bian, Yuanhang Zhou, Ji-Rong Wen, and Jingsong Yu. 2020. Improving conversational recommender systems via knowledge graph based semantic fusion. In SIGKDD'20. 1006--1014.Google ScholarDigital Library
- Kun Zhou, Yuanhang Zhou, Wayne Xin Zhao, Xiaoke Wang, and Ji-Rong Wen. 2020. Towards topic-guided conversational recommender system. arXiv preprint arXiv:2010.04125 (2020).Google Scholar
- Jie Zou, Yifan Chen, and Evangelos Kanoulas. 2020. Towards question-based recommender systems. In SIGIR'20. 881--890.Google ScholarDigital Library
Index Terms
- Extracting Relevant Information from User's Utterances in Conversational Search and Recommendation
Recommendations
Towards Conversational Search and Recommendation: System Ask, User Respond
CIKM '18: Proceedings of the 27th ACM International Conference on Information and Knowledge ManagementConversational search and recommendation based on user-system dialogs exhibit major differences from conventional search and recommendation tasks in that 1) the user and system can interact for multiple semantically coherent rounds on a task through ...
Large-scale Interactive Conversational Recommendation System using Actor-Critic Framework
RecSys '21: Proceedings of the 15th ACM Conference on Recommender SystemsWe propose AC-CRS, a novel conversational recommendation system based on reinforcement learning that better models user interaction compared to prior work. Interactive recommender systems expect an initial request from a user and then iterate by asking ...
Exploring Conversational Search With Humans, Assistants, and Wizards
CHI EA '17: Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing SystemsChatbots and conversational assistants are becoming increasingly popular. However, for information seeking scenarios, these systems still have very limited conversational abilities, and primarily serve as proxies to existing web search engines. In this ...
Comments