skip to main content
10.1145/3470482.3479621acmconferencesArticle/Chapter ViewAbstractPublication PageswebmediaConference Proceedingsconference-collections
research-article

A contextual approach to improve the user's experience in interactive recommendation systems

Published: 05 November 2021 Publication History

Abstract

Recommendation Systems have concerned about the online environment of real-world scenarios where the system should continually learn and predict new recommendations. Current works have handled it as a Multi-Armed Bandit (MAB) problem by proposing parametric bandit models based on the main recommendation concepts to handle the exploitation and exploration dilemma. However, recent works identified a new problem about the way these models handle the user cold-start. Due to the lack of information about the user, these models have intrinsically delivered naive non-personalized recommendations in their first recommendations until the system learns more about the user. The first recommendations of these bandit models are equivalent to a random selection around the items (i.e., a pure-exploration approach) or a biased selection by the most popular items in the system (i.e., a pure-exploitation approach). Thus, to mitigate this problem, we propose a new contextual approach to initialize the bandit models. This context is made by the information available about the items: their popularity and entropy. The idea is to address both exploration and exploitation goals since the first recommendations by mixing entropic and popular items. Indeed, this approach maximizes the user's satisfaction in the long-term run. By a strong experimental evaluation, comparing our proposal with seven state-of-the-art methods in three real datasets, we demonstrate this context achieves statistically significant improvements by outperforming all baselines.

References

[1]
Xavier Amatriain and Justin Basilico. 2015. Recommender systems in industry: A netflix case study. In Recommender systems handbook. Springer, 385--419.
[2]
Xavier Amatriain and Justin Basilico. 2016. Past, present, and future of recommender systems: An industry perspective. In Proceedings of the 10th ACM conference on recommender systems. 211--214.
[3]
Peter Auer. 2002. Using confidence bounds for exploitation-exploration trade-offs. Journal of Machine Learning Research 3, Nov (2002), 397--422.
[4]
Peter Auer, Nicolo Cesa-Bianchi, and Paul Fischer. 2002. Finite-time analysis of the multiarmed bandit problem. Machine learning 47, 2-3 (2002), 235--256.
[5]
Andrea Barraza-Urbina, Georgia Koutrika, Mathieu d'Aquin, and Conor Hayes. 2018. BEARS: Towards an evaluation framework for bandit-based interactive recommender systems. REVEAL 18, October 6-7, 2018, Vancouver, Canada (2018).
[6]
Jesús Bobadilla, Fernando Ortega, Antonio Hernando, and Abraham Gutiérrez. 2013. Recommender systems survey. Knowledge-Based Systems (2013).
[7]
Robin Burke. 2002. Hybrid recommender systems: Survey and experiments. User modeling and user-adapted interaction 12, 4 (2002), 331--370.
[8]
Olivier Chapelle and Lihong Li. 2011. An empirical evaluation of thompson sampling. In Advances in neural information processing systems. 2249--2257.
[9]
Mehdi Elahi, Francesco Ricci, and Neil Rubens. 2016. A survey of active learning in collaborative filtering recommender systems. Computer Science Review 20 (2016), 29--50.
[10]
Crícia Felício, Klérisson Paixão, Celia Barcelos, and Philippe Preux. 2017. A multi-armed bandit model selection for cold-start user recommendation. In Proceedings of the 25th Conference on User Modeling, Adaptation and Personalization.
[11]
Murat Göksedef and Şule Gündüz-Öğüdücü. 2010. Combination of Web page recommender systems. Expert Systems with Applications 37, 4 (2010), 2911--2922.
[12]
Negar Hariri, Bamshad Mobasher, and Robin Burke. 2014. Context adaptation in interactive recommender systems. In Proceedings of the 8th ACM Conference on Recommender systems. 41--48.
[13]
Antonio Hernando, Jesús Bobadilla, Fernando Ortega, and Abraham Gutiérrez. 2017. A probabilistic model for recommending to new cold-start non-registered users. Information Sciences 376 (2017), 216--232.
[14]
Steven Hoi, Doyen Sahoo, Jing Lu, and Peilin Zhao. 2018. Online Learning: A Comprehensive Survey. (02 2018).
[15]
Jaya Kawale, Hung H Bui, Branislav Kveton, Long T Thanh, and Sanjay Chawla. 2015. Efficient thompson sampling for online matrix-factorization recommendation. Advances in Neural Information Processing Systems 28 (2015), 1297--1305.
[16]
Neal Lathia, Stephen Hailes, Licia Capra, and Xavier Amatriain. 2010. Temporal diversity in recommender systems. In Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval. 210--217.
[17]
Hoyeop Lee, Jinbae Im, Seongwon Jang, Hyunsouk Cho, and Sehee Chung. 2019. MeLU: meta-learned user preference estimator for cold-start recommendation. In Proceedings of the 25th ACM SIGKDD. 1073--1082.
[18]
Lihong Li, Wei Chu, John Langford, and Robert E Schapire. 2010. A contextual-bandit approach to personalized news article recommendation. In Proceedings of the 19th international conference on Worldwide web. 661--670.
[19]
Trong T Nguyen and Hady W Lauw. 2014. Dynamic clustering of contextual multi-armed bandits. In Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management. 1959--1962.
[20]
Jorge Nocedal and Stephen J Wright. 2006. Sequential quadratic programming. Numerical optimization (2006), 529--562.
[21]
Lijing Qin, Shouyuan Chen, and Xiaoyan Zhu. 2014. Contextual combinatorial bandit and its application on diversified online recommendation. In Proceedings of the 2014 SIAM International Conference on Data Mining. SIAM, 461--469.
[22]
Chhavi Rana and Sanjay Kumar Jain. 2015. A study of the dynamic features of recommender systems. Artificial Intelligence Review 43, 1 (2015), 141--153.
[23]
Al Mamunur Rashid, Istvan Albert, Dan Cosley, Shyong K Lam, Sean M McNee, Joseph A Konstan, and John Riedl. 2002. Getting to know you: learning new user preferences in recommender systems. In Proceedings of the 7th international conference on Intelligent user interfaces. 127--134.
[24]
Javier Sanz-Cruzado, Pablo Castells, and Esther López. 2019. A simple multi-armed nearest-neighbor bandit for interactive recommendation. In Proceedings of the 13th ACM Conference on Recommender Systems. 358--362.
[25]
Nícollas Silva, Diego Carvalho, Adriano CM Pereira, Fernando Mourão, and Leonardo Rocha. 2019. The Pure Cold-Start Problem: A deep study about how to conquer first-time users in recommendations domains. Information Systems 80 (2019), 1--12.
[26]
Thiago Silva, Nícollas Silva, Heitor Werneck, Adriano CM Pereira, and Leonardo Rocha. 2020. The impact of first recommendations based on exploration or exploitation approaches in recommender systems' learning. In Proceedings of the Brazilian Symposium on Multimedia and the Web. 173--180.
[27]
Xiaoyuan Su and Taghi M Khoshgoftaar. 2009. A survey of collaborative filtering techniques. Advances in artificial intelligence 2009 (2009).
[28]
Richard S Sutton and Andrew G Barto. 2018. Reinforcement learning: An introduction. MIT press.
[29]
Huazheng Wang, Qingyun Wu, and Hongning Wang. 2016. Learning hidden features for contextual bandits. In Proceedings of the 25th ACM International on Conference on Information and Knowledge Management. 1633--1642.
[30]
Huazheng Wang, Qingyun Wu, and Hongning Wang. 2017. Factorization bandits for interactive recommendation. In AAAI Conference on Artificial Intelligence.
[31]
Lu Wang, Chengyu Wang, Keqiang Wang, and Xiaofeng He. 2017. Biucb: A contextual bandit algorithm for cold-start and diversified recommendation. In 2017 IEEE International Conference on Big Knowledge (ICBK). IEEE, 248--253.
[32]
Qing Wang, Chunqiu Zeng, Wubai Zhou, Tao Li, S Sitharama Iyengar, Larisa Shwartz, and Genady Ya Grabarnik. 2018. Online interactive collaborative filtering using multi-armed bandit with dependent arms. IEEE Transactions on Knowledge and Data Engineering 31, 8 (2018), 1569--1580.
[33]
Qingyun Wu, Naveen Iyer, and Hongning Wang. 2018. Learning contextual bandits in a non-stationary environment. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. 495--504.
[34]
Qingyun Wu, Huazheng Wang, Quanquan Gu, and Hongning Wang. 2016. Contextual bandits in a collaborative environment. In Proceedings of the 39th International ACM SIGIR Conference on Development in Information Retrieval.
[35]
Qitian Wu, Hengrui Zhang, Xiaofeng Gao, Peng He, Paul Weng, Han Gao, and Guihai Chen. 2019. Dual graph attention networks for deep latent representation of multifaceted social effects in recommender systems. In The World Wide Web Conference. 2091--2102. https://dl.acm.org/doi/abs/10.1145/3308558.3313442
[36]
Xiao Xu, Fang Dong, Yanghua Li, Shaojian He, and Xin Li. 2020. Contextual-Bandit Based Personalized Recommendation with Time-Varying User Interests. In AAAI. 6518--6525.
[37]
Chunqiu Zeng, Qing Wang, Shekoofeh Mokhtari, and Tao Li. 2016. Online context-aware recommendation with time varying multi-armed bandit. In Proceedings of the 22nd ACM SIGKDD. 2025--2034.
[38]
Xiangyu Zhao, Long Xia, Jiliang Tang, and Dawei Yin. 2019. Deep reinforcement learning for search, recommendation, and online advertising: a survey. ACM SIGWEB Newsletter Spring (2019), 1--15.
[39]
Xiaoxue Zhao, Weinan Zhang, and Jun Wang. 2013. Interactive collaborative filtering. In Proceedings of the 22nd ACM international conference on Information & Knowledge Management. 1411--1420.
[40]
Sijin Zhou, Xinyi Dai, Haokun Chen, Weinan Zhang, Kan Ren, Ruiming Tang, Xiuqiang He, and Yong Yu. 2020. Interactive recommender system via knowledge graph-enhanced reinforcement learning. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval.
[41]
Lixin Zou, Long Xia, Yulong Gu, Xiangyu Zhao, Weidong Liu, Jimmy Xiangji Huang, and Dawei Yin. 2020. Neural Interactive Collaborative Filtering. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 749--758.

Cited By

View all
  • (2023)A Complete Framework for Offline and Counterfactual Evaluations of Interactive Recommendation SystemsProceedings of the 29th Brazilian Symposium on Multimedia and the Web10.1145/3617023.3617049(193-197)Online publication date: 23-Oct-2023
  • (2022)Interactive POI Recommendation: applying a Multi-Armed Bandit framework to characterise and create new models for this scenarioProceedings of the Brazilian Symposium on Multimedia and the Web10.1145/3539637.3557060(211-221)Online publication date: 7-Nov-2022

Index Terms

  1. A contextual approach to improve the user's experience in interactive recommendation systems

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    WebMedia '21: Proceedings of the Brazilian Symposium on Multimedia and the Web
    November 2021
    271 pages
    ISBN:9781450386098
    DOI:10.1145/3470482
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    In-Cooperation

    • SBC: Brazilian Computer Society
    • CNPq: Conselho Nacional de Desenvolvimento Cientifico e Tecn
    • CAPES: Brazilian Higher Education Funding Council

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 05 November 2021

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Multi-Armed Bandits
    2. Online Learning
    3. Recommendation Systems

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Conference

    WebMedia '21
    Sponsor:
    WebMedia '21: Brazilian Symposium on Multimedia and the Web
    November 5 - 12, 2021
    Minas Gerais, Belo Horizonte, Brazil

    Acceptance Rates

    WebMedia '21 Paper Acceptance Rate 24 of 75 submissions, 32%;
    Overall Acceptance Rate 270 of 873 submissions, 31%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)21
    • Downloads (Last 6 weeks)3
    Reflects downloads up to 13 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2023)A Complete Framework for Offline and Counterfactual Evaluations of Interactive Recommendation SystemsProceedings of the 29th Brazilian Symposium on Multimedia and the Web10.1145/3617023.3617049(193-197)Online publication date: 23-Oct-2023
    • (2022)Interactive POI Recommendation: applying a Multi-Armed Bandit framework to characterise and create new models for this scenarioProceedings of the Brazilian Symposium on Multimedia and the Web10.1145/3539637.3557060(211-221)Online publication date: 7-Nov-2022

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media