short-paper

Improving Conversational Recommender Systems via Transformer-based Sequential Modelling

Authors:

Evangelos Kanoulas,

Cheng LongAuthors Info & Claims

SIGIR '22: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval

Pages 2319 - 2324

https://doi.org/10.1145/3477495.3531852

Published: 07 July 2022 Publication History

Abstract

In Conversational Recommender Systems (CRSs), conversations usually involve a set of related items and entities e.g., attributes of items. These items and entities are mentioned in order following the development of a dialogue. In other words, potential sequential dependencies exist in conversations. However, most of the existing CRSs neglect these potential sequential dependencies. In this paper, we propose a Transformer-based sequential conversational recommendation method, named TSCR, which models the sequential dependencies in the conversations to improve CRS. We represent conversations by items and entities, and construct user sequences to discover user preferences by considering both mentioned items and entities. Based on the constructed sequences, we deploy a Cloze task to predict the recommended items along a sequence. Experimental results demonstrate that our TSCR model significantly outperforms state-of-the-art baselines.

References

[1]

Keyu Chen and Shiliang Sun. 2021. Knowledge-Based Conversational Recommender Systems Enhanced by Dialogue Policy Learning. In The 10th International Joint Conference on Knowledge Graphs. 10--18.

Digital Library

[2]

Qibin Chen, Junyang Lin, Yichang Zhang, Ming Ding, Yukuo Cen, Hongxia Yang, and Jie Tang. 2019. Towards Knowledge-Based Recommender Dialog System. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 1803--1813.

[3]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).

[4]

Dan Hendrycks and Kevin Gimpel. 2016. Bridging nonlinearities and stochastic regularizers with gaussian error linear units. (2016).

[5]

Dietmar Jannach and Ahtsham Manzoor. 2020. End-to-End Learning for Conversational Recommendation: A Long Way to Go?. In IntRS@ RecSys. 72--76.

[6]

Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).

[7]

Wenqiang Lei, Xiangnan He, Yisong Miao, Qingyun Wu, Richang Hong, MinYen Kan, and Tat-Seng Chua. 2020. Estimation-action-reflection: Towards deep interaction between conversational and recommender systems. In Proceedings of the 13th International Conference on Web Search and Data Mining. 304--312.

Digital Library

[8]

Wenqiang Lei, Gangyi Zhang, Xiangnan He, Yisong Miao, Xiang Wang, Liang Chen, and Tat-Seng Chua. 2020. Interactive Path Reasoning on Graph for Conversational Recommendation. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (Virtual Event, CA, USA) (KDD '20). 2073--2083.

Digital Library

[9]

Raymond Li, Samira Ebrahimi Kahou, Hannes Schulz, Vincent Michalski, Laurent Charlin, and Chris Pal. 2018. Towards Deep Conversational Recommendations. In Advances in Neural Information Processing Systems. 9748--9758.

[10]

Yu Lu, Junwei Bao, Yan Song, Zichen Ma, Shuguang Cui, Youzheng Wu, and Xiaodong He. 2021. RevCore: Review-augmented conversational recommendation. arXiv preprint arXiv:2106.00957 (2021).

[11]

Wenchang Ma, Ryuichi Takanobu, and Minlie Huang. 2021. CR-Walker: TreeStructured Graph Reasoning and Dialog Acts for Conversational Recommendation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. 1839--1851.

[12]

Ahtsham Manzoor and Dietmar Jannach. 2021. Generation-based vs. Retrievalbased Conversational Recommendation: A User-Centric Comparison. In Fifteenth ACM Conference on Recommender Systems. 515--520.

[13]

Zhaochun Ren, Zhi Tian, Dongdong Li, Pengjie Ren, Liu Yang, Xin Xin, Huasheng Liang, Maarten de Rijke, and Zhumin Chen. 2022. Variational Reasoning about User Preferences for Conversational Recommendation. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval.

Digital Library

[14]

Rajdeep Sarkar, Koustava Goswami, Mihael Arcan, and John Philip McCrae. 2020. Suggest me a movie for tonight: Leveraging Knowledge Graphs for Conversational Recommendation. In Proceedings of the 28th International Conference on Computational Linguistics. 4179--4189.

[15]

Andrew I Schein, Alexandrin Popescul, Lyle H Ungar, and David M Pennock. 2002. Methods and metrics for cold-start recommendations. In Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval. 253--260.

Digital Library

[16]

Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. 2019. BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 1441--1450.

Digital Library

[17]

Yueming Sun and Yi Zhang. 2018. Conversational recommender system. In The 41st international acm sigir conference on research & development in information retrieval. 235--244.

[18]

Wilson L Taylor. 1953. "Cloze procedure": A new tool for measuring readability. Journalism quarterly 30, 4 (1953), 415--433.

[19]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ?ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems. 5998--6008.

[20]

Lingzhi Wang, Huang Hu, Lei Sha, Can Xu, Kam-Fai Wong, and Daxin Jiang. 2021. Finetuning Large-Scale Pre-trained Language Models for Conversational Recommendation with Knowledge Graph. arXiv preprint arXiv:2110.07477 (2021).

[21]

Xiaoyi Wang, Yaguang Li, and Jie Liu. 2021. Conversation Recommender System Based on Knowledge Graph and Time-series Feature. In The 5th International Conference on Computer Science and Application Engineering. 1--5.

[22]

Fuzhao Xue, Aixin Sun, Hao Zhang, Jinjie Ni, and Eng Siong Chng. 2022. An Embarrassingly Simple Model for Dialogue Relation Extraction. arXiv preprint arXiv:2012.13873 (2022).

[23]

Hang Yan, Junqi Dai, Xipeng Qiu, Zheng Zhang, et al. 2021. A Unified Generative Framework for Aspect-Based Sentiment Analysis. arXiv preprint arXiv:2106.04300 (2021).

[24]

Bowen Yang, Cong Han, Yu Li, Lei Zuo, and Zhou Yu. 2021. Improving Conversational Recommendation Systems' Quality with Context-Aware Item Meta Information. arXiv preprint arXiv:2112.08140 (2021).

[25]

Tong Zhang, Yong Liu, Peixiang Zhong, Chen Zhang, Hao Wang, and Chunyan Miao. 2021. KECRS: Towards Knowledge-Enriched Conversational Recommendation System. arXiv preprint arXiv:2105.08261 (2021).

[26]

Yongfeng Zhang, Xu Chen, Qingyao Ai, Liu Yang, and W Bruce Croft. 2018. Towards conversational search and recommendation: System ask, user respond. In Proceedings of the 27th acm international conference on information and knowledge management. 177--186.

Digital Library

[27]

Jinfeng Zhou, Bo Wang, Ruifang He, and Yuexian Hou. 2021. CRFR: Improving Conversational Recommender Systems via Flexible Fragments Reasoning on Knowledge Graphs. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. 4324--4334.

[28]

Kun Zhou, Wayne Xin Zhao, Shuqing Bian, Yuanhang Zhou, Ji-Rong Wen, and Jingsong Yu. 2020. Improving conversational recommender systems via knowledge graph based semantic fusion. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1006--1014.

Digital Library

[29]

Kun Zhou, Wayne Xin Zhao, Hui Wang, Sirui Wang, Fuzheng Zhang, Zhongyuan Wang, and Ji-Rong Wen. 2020. Leveraging historical interaction data for improving conversational recommender system. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 2349--2352.

Digital Library

[30]

Kun Zhou, Yuanhang Zhou, Wayne Xin Zhao, Xiaoke Wang, and Ji-Rong Wen. 2020. Towards Topic-Guided Conversational Recommender System. In Proceedings of the 28th International Conference on Computational Linguistics. 4128--4139.

[31]

Jie Zou. 2022. Improving search and recommendation by asking clarifying questions. In ACM SIGIR Forum, Vol. 55. 1--2.

Digital Library

[32]

Jie Zou, Yifan Chen, and Evangelos Kanoulas. 2020. Towards question-based recommender systems. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 881--890.

Digital Library

[33]

Jie Zou and Evangelos Kanoulas. 2019. Learning to Ask: Question-Based Sequential Bayesian Product Search. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management (Beijing, China) (CIKM '19). 369--378.

Digital Library

[34]

Jie Zou and Evangelos Kanoulas. 2020. Towards Question-Based High-Recall Information Retrieval: Locating the Last Few Relevant Documents for TechnologyAssisted Reviews. ACM Trans. Inf. Syst. 38, 3, Article 27 (May 2020), 35 pages.

Digital Library

[35]

Jie Zou, Evangelos Kanoulas, and Yiqun Liu. 2020. An Empirical Study on Clarifying Question-Based Systems. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 2361--2364.

Digital Library

Cited By

Liu YPei JZhang WLi MChe Wde Rijke M(2025)Augmentation with Neighboring Information for Conversational RecommendationACM Transactions on Information Systems10.1145/371258843:3(1-49)Online publication date: 17-Jan-2025
https://dl.acm.org/doi/10.1145/3712588
He ZXie ZSteck HLiang DJha RKallus NMcAuley JNejdl WAuer SCha MMoens MNajork M(2025)Reindex-Then-Adapt: Improving Large Language Models for Conversational RecommendationProceedings of the Eighteenth ACM International Conference on Web Search and Data Mining10.1145/3701551.3703573(866-875)Online publication date: 10-Mar-2025
https://dl.acm.org/doi/10.1145/3701551.3703573
Liu YQi JYu Y(2025)A Review on Deep Learning for Sequential Recommender Systems: Key Technologies and DirectionsBig Data10.1007/978-981-96-1024-2_22(305-318)Online publication date: 24-Jan-2025
https://doi.org/10.1007/978-981-96-1024-2_22
Show More Cited By

Index Terms

Improving Conversational Recommender Systems via Transformer-based Sequential Modelling
1. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Question answering
      2. Recommender systems

Recommendations

Knowledge-Enhanced Conversational Recommendation via Transformer-Based Sequential Modeling
In conversational recommender systems (CRSs), conversations usually involve a set of items and item-related entities or attributes, e.g., director is a related entity of a movie. These items and item-related entities are often mentioned along the ...
Conversational Collaborative Recommendation --- An Experimental Analysis

Traditionally, collaborative recommender systems have been based on a single-shot model of recommendation where a single set of recommendations is generated based on a user's (past) stored preferences. However, content-based recommender system research ...
Customized Conversational Recommender Systems
Machine Learning and Knowledge Discovery in Databases
Abstract
Conversational recommender systems (CRS) aim to capture user’s current intentions and provide recommendations through real-time multi-turn conversational interactions. As a human-machine interactive system, it is essential for CRS to improve the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR '22: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 2022

3569 pages

ISBN:9781450387323

DOI:10.1145/3477495

General Chairs:
Enrique Amigo
UNED
,
Pablo Castells
UAM and Amazon
,
Julio Gonzalo
UNED
,
Program Chairs:
Ben Carterette
Spotify
,
J. Shane Culpepper
RMIT University
,
Gabriella Kazai
Waseda University

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 July 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

SIGIR '22

Sponsor:

SIGIR

SIGIR '22: The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 11 - 15, 2022

Madrid, Spain

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

25
Total Citations
View Citations
1,022
Total Downloads

Downloads (Last 12 months)179
Downloads (Last 6 weeks)19

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Liu YPei JZhang WLi MChe Wde Rijke M(2025)Augmentation with Neighboring Information for Conversational RecommendationACM Transactions on Information Systems10.1145/371258843:3(1-49)Online publication date: 17-Jan-2025
https://dl.acm.org/doi/10.1145/3712588
He ZXie ZSteck HLiang DJha RKallus NMcAuley JNejdl WAuer SCha MMoens MNajork M(2025)Reindex-Then-Adapt: Improving Large Language Models for Conversational RecommendationProceedings of the Eighteenth ACM International Conference on Web Search and Data Mining10.1145/3701551.3703573(866-875)Online publication date: 10-Mar-2025
https://dl.acm.org/doi/10.1145/3701551.3703573
Liu YQi JYu Y(2025)A Review on Deep Learning for Sequential Recommender Systems: Key Technologies and DirectionsBig Data10.1007/978-981-96-1024-2_22(305-318)Online publication date: 24-Jan-2025
https://doi.org/10.1007/978-981-96-1024-2_22
Zou JSun ALong CKanoulas E(2024)Knowledge-Enhanced Conversational Recommendation via Transformer-Based Sequential ModelingACM Transactions on Information Systems10.1145/367737642:6(1-27)Online publication date: 18-Oct-2024
https://dl.acm.org/doi/10.1145/3677376
Liu QFeng XGu TLiu X(2024)FairCRS: Towards User-oriented Fairness in Conversational Recommendation SystemsProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3688150(126-136)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3640457.3688150
Zhang SWang MZhao XGuo RZhao YZhuang CGu JZhang ZYin H(2024)DNS-Rec: Data-aware Neural Architecture Search for Recommender SystemsProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3688117(591-600)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3640457.3688117
Petruzzelli AMartina ASpillo GMusto CDe Gemmis MLops PSemeraro G(2024)Improving Transformer-based Sequential Conversational Recommendations through Knowledge Graph EmbeddingsProceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization10.1145/3627043.3659565(172-182)Online publication date: 22-Jun-2024
https://dl.acm.org/doi/10.1145/3627043.3659565
Dao HDeng YLe DLiao LHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Broadening the View: Demonstration-augmented Prompt Learning for Conversational RecommendationProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657755(785-795)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657755
Kim SLim JJang J(2024)Patient-Adaptive Beat-Wise Temporal Transformer for Atrial Fibrillation Classification in Continuous Long-Term Cardiac MonitoringIEEE Access10.1109/ACCESS.2024.349804312(172358-172367)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3498043
Ju WZhao YQin YYi SYuan JXiao ZLuo XYan XZhang M(2024)COOLInformation Fusion10.1016/j.inffus.2024.102341107:COnline publication date: 2-Jul-2024
https://dl.acm.org/doi/10.1016/j.inffus.2024.102341
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten