skip to main content
10.1145/3437963.3441733acmconferencesArticle/Chapter ViewAbstractPublication PageswsdmConference Proceedingsconference-collections
research-article

Real-time Relevant Recommendation Suggestion

Published: 08 March 2021 Publication History

Abstract

Users of recommendation systems usually focus on one topic at a time. When finishing reading an item, users may want to access more relevant items related to the last read one as extended reading. However, conventional recommendation systems are hard to provide the continuous extended reading function of these relevant items, since the main recommendation results should be diversified. In this paper, we propose a new task named recommendation suggestion, which aims to (1) predict whether users want extended reading, and (2) provide appropriate relevant items as suggestions. These recommended relevant items are arranged in a relevant box and instantly inserted below the clicked item in the main feed. The challenge of recommendation suggestion on relevant items is that it should further consider semantic relevance and information gain besides CTR-related factors. Moreover, the real-time relevant box insertion may also harm the overall performance when users do not want extended reading. To address these issues, we propose a novel Real-time relevant recommendation suggestion (R3S) framework, which consists of an Item recommender and a Box trigger. We extract features from multiple aspects including feature interaction, semantic similarity and information gain as different experts, and propose a new Multi-critic multi-gate mixture-of-experts (M3oE) strategy to jointly consider different experts with multi-head critics. In experiments, we conduct both offline and online evaluations on a real-world recommendation system with detailed ablation tests. The significant improvements in item/box related metrics verify the effectiveness of R3S. Moreover, we have deployed R3S on WeChat Top Stories, which affects millions of users. The source codes are in https://github.com/modriczhang/R3S.

References

[1]
Malak Al-Hassan, Haiyan Lu, and Jie Lu. 2015. A semantic enhanced hybrid recommendation approach: A case study of e-Government tourism service recommendation system. Decision Support Systems (2015).
[2]
Trapit Bansal, David Belanger, and Andrew McCallum. 2016. Ask the gru: Multi-task learning for deep text recommendations. In Proceedings of RecSys.
[3]
John S Breese, David Heckerman, and Carl Kadie. 1998. Empirical analysis of predictive algorithms for collaborative filtering. In Proceedings of UAI.
[4]
Michel Capelle, Flavius Frasincar, Marnix Moerland, and Frederik Hogenboom. 2012. Semantics-based news recommendation. In Proceedings of WIMS.
[5]
Wanyu Chen, Fei Cai, Honghui Chen, and Maarten De Rijke. 2020. Personalized query suggestion diversification in information retrieval. Frontiers of Computer Science (2020).
[6]
Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et al. 2016. Wide & deep learning for recommender systems. In Proceedings of the DLRS workshop.
[7]
Weiyu Cheng, Yanyan Shen, and Linpeng Huang. 2020. Adaptive Factorization Network: Learning Adaptive-Order Feature Interactions. In Proceedings of AAAI.
[8]
Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. 2017. DeepFM: a factorization-machine based neural network for CTR prediction. In Proceedings of IJCAI.
[9]
Xiangnan He and Tat-Seng Chua. 2017. Neural factorization machines for sparse predictive analytics. In Proceedings of SIGIR.
[10]
Baotian Hu, Zhengdong Lu, Hang Li, and Qingcai Chen. 2014. Convolutional neural network architectures for matching natural language sentences. In Proceedings of NIPS.
[11]
Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix factorization techniques for recommender systems. Computer (2009).
[12]
Ruirui Li, Liangda Li, Xian Wu, Yunhong Zhou, and Wei Wang. 2019. Click feedback-aware query recommendation using adversarial examples. In Proceedings of WWW.
[13]
Jianxun Lian, Xiaohuan Zhou, Fuzheng Zhang, Zhongxia Chen, Xing Xie, and Guangzhong Sun. 2018. xdeepfm: Combining explicit and implicit feature interactions for recommender systems. In Proceedings of KDD.
[14]
Jimmy Lin and Mark D Smucker. 2008. How do users find things with ? Towards automatic utility evaluation with user simulations. In Proceedings of SIGIR.
[15]
Bin Liu, Chenxu Zhu, Guilin Li, Weinan Zhang, Jincai Lai, Ruiming Tang, Xiuqiang He, Zhenguo Li, and Yong Yu. 2020 b. AutoFIS: Automatic Feature Interaction Selection in Factorization Models for Click-Through Rate Prediction. (2020).
[16]
Qi Liu, Ruobing Xie, Lei Chen, Shukai Liu, Ke Tu, Peng Cui, Bo Zhang, and Leyu Lin. 2020 a. Graph Neural Network for Tag Ranking in Tag-enhanced Video Recommendation. In Proceedings of CIKM.
[17]
Jiaqi Ma, Zhe Zhao, Xinyang Yi, Jilin Chen, Lichan Hong, and Ed H Chi. 2018. Modeling task relationships in multi-task learning with multi-gate mixture-of-experts. In Proceedings of KDD.
[18]
Steffen Rendle. 2010. Factorization machines. In Proceedings of ICDM.
[19]
Masoumeh Riyahi and Mohammad Karim Sohrabi. 2020. Providing effective recommendations in discussion groups using a new hybrid recommender system based on implicit ratings and semantic similarity. Electronic Commerce Research and Applications (2020).
[20]
Badrul Munir Sarwar, George Karypis, Joseph A Konstan, John Riedl, et al. 2001. Item-based collaborative filtering recommendation algorithms. In Proceedings of WWW.
[21]
Ozan Sener and Vladlen Koltun. 2018. Multi-task learning as multi-objective optimization. In Proceedings of NIPS.
[22]
Mark D Smucker and James Allan. 2006. Find-similar: similarity browsing as a search tool. In Proceedings of SIGIR.
[23]
Weiping Song, Chence Shi, Zhiping Xiao, Zhijian Duan, Yewen Xu, Ming Zhang, and Jian Tang. 2019. Autoint: Automatic feature interaction learning via self-attentive neural networks. In Proceedings of CIKM.
[24]
Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. 2019. BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer. In Proceedings of CIKM.
[25]
Ruoxi Wang, Bin Fu, Gang Fu, and Mingliang Wang. 2017. Deep & cross network for ad click predictions. In Proceedings of ADKDD.
[26]
Chuhan Wu, Fangzhao Wu, Mingxiao An, Jianqiang Huang, Yongfeng Huang, and Xing Xie. 2019 b. Npa: Neural news recommendation with personalized attention. In Proceedings of KDD.
[27]
Qiong Wu, Yong Liu, Chunyan Miao, Binqiang Zhao, Yin Zhao, and Lu Guan. 2019 a. PD-GAN: Adversarial Learning for Personalized Diversity-Promoting Recommendation. In Proceedings of IJCAI.
[28]
Jun Xiao, Hao Ye, Xiangnan He, Hanwang Zhang, Fei Wu, and Tat-Seng Chua. 2017. Attentional factorization machines: Learning the weight of feature interactions via attention networks. In Proceedings of IJCAI.
[29]
Ruobing Xie, Cheng Ling, Yalong Wang, Rui Wang, Feng Xia, and Leyu Lin. 2020 a. Deep Feedback Network for Recommendation. In Proceedings of IJCAI.
[30]
Ruobing Xie, Zhijie Qiu, Jun Rao, Yi Liu, Bo Zhang, and Leyu Lin. 2020 b. Internal and Contextual Attention Network for Cold-start Multi-channel Matching in Recommendation. In Proceedings of IJCAI.
[31]
Zhenghua Xu, Cheng Chen, Thomas Lukasiewicz, Yishu Miao, and Xiangwu Meng. 2016. Tag-aware personalized recommendation using a deep-semantic similarity model with negative sampling. In Proceedings of CIKM.
[32]
Ling Yan, Wu-Jun Li, Gui-Rong Xue, and Dingyi Han. 2014. Coupled group lasso for web-scale ctr prediction in display advertising. In Proceedings of ICML.
[33]
Xing Yi, Liangjie Hong, Erheng Zhong, Nanthan Nan Liu, and Suju Rajan. 2014. Beyond clicks: dwell time for personalization. In Proceedings of RecSys.
[34]
Mi Zhang and Neil Hurley. 2008. Avoiding monotony: improving the diversity of recommendation lists. In Proceedings of RecSys.
[35]
Weinan Zhang, Tianming Du, and Jun Wang. 2016. Deep learning over multi-field categorical data. In European conference on information retrieval.
[36]
Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018. Deep interest network for click-through rate prediction. In Proceedings of KDD.
[37]
Cai-Nicolas Ziegler, Sean M McNee, Joseph A Konstan, and Georg Lausen. 2005. Improving recommendation lists through topic diversification. In Proceedings of WWW.

Cited By

View all
  • (2024)Modeling User Intent Beyond Trigger: Incorporating Uncertainty for Trigger-Induced RecommendationProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3680065(4743-4751)Online publication date: 21-Oct-2024
  • (2024)Deep Evolutional Instant Interest Network for CTR Prediction in Trigger-Induced RecommendationProceedings of the 17th ACM International Conference on Web Search and Data Mining10.1145/3616855.3635829(846-854)Online publication date: 4-Mar-2024
  • (2024)Finding What Users Look for by Attribute-Aware Personalized Item Comparison in Relevant RecommendationCompanion Proceedings of the ACM Web Conference 202410.1145/3589335.3651508(549-552)Online publication date: 13-May-2024
  • Show More Cited By

Index Terms

  1. Real-time Relevant Recommendation Suggestion

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    WSDM '21: Proceedings of the 14th ACM International Conference on Web Search and Data Mining
    March 2021
    1192 pages
    ISBN:9781450382977
    DOI:10.1145/3437963
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 08 March 2021

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. recommendation suggestion
    2. recommender system
    3. relevant recommendation

    Qualifiers

    • Research-article

    Conference

    WSDM '21

    Acceptance Rates

    Overall Acceptance Rate 498 of 2,863 submissions, 17%

    Upcoming Conference

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)64
    • Downloads (Last 6 weeks)6
    Reflects downloads up to 18 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Modeling User Intent Beyond Trigger: Incorporating Uncertainty for Trigger-Induced RecommendationProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3680065(4743-4751)Online publication date: 21-Oct-2024
    • (2024)Deep Evolutional Instant Interest Network for CTR Prediction in Trigger-Induced RecommendationProceedings of the 17th ACM International Conference on Web Search and Data Mining10.1145/3616855.3635829(846-854)Online publication date: 4-Mar-2024
    • (2024)Finding What Users Look for by Attribute-Aware Personalized Item Comparison in Relevant RecommendationCompanion Proceedings of the ACM Web Conference 202410.1145/3589335.3651508(549-552)Online publication date: 13-May-2024
    • (2024)UID-Net: Enhancing Click-Through Rate Prediction in Trigger-Induced Recommendation Through User Interest DecompositionAdvanced Data Mining and Applications10.1007/978-981-96-0850-8_4(49-64)Online publication date: 24-Dec-2024
    • (2023)Generative Next-Basket RecommendationProceedings of the 17th ACM Conference on Recommender Systems10.1145/3604915.3608823(737-743)Online publication date: 14-Sep-2023
    • (2023)DPAN: Dynamic Preference-based and Attribute-aware Network for Relevant RecommendationsProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3615218(3838-3842)Online publication date: 21-Oct-2023
    • (2023)Deep Intention-Aware Network for Click-Through Rate PredictionCompanion Proceedings of the ACM Web Conference 202310.1145/3543873.3584661(533-537)Online publication date: 30-Apr-2023
    • (2023)On Efficient Processing of Queries for Live Multi-Streaming Soiree OrganizationIEEE Transactions on Services Computing10.1109/TSC.2023.324832116:4(2812-2826)Online publication date: 1-Jul-2023
    • (2023)Real-time Personalized Guest Experiences using Product Attribute Information on Vanilla Hardware2023 IEEE 39th International Conference on Data Engineering Workshops (ICDEW)10.1109/ICDEW58674.2023.00009(28-35)Online publication date: Apr-2023
    • (2023)When Alignment Makes a Difference: A Content-Based Variational Model for Cold-Start CTR PredictionAdvanced Data Mining and Applications10.1007/978-3-031-46661-8_48(724-739)Online publication date: 5-Nov-2023
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media