skip to main content
10.1145/3132847.3133025acmconferencesArticle/Chapter ViewAbstractPublication PagescikmConference Proceedingsconference-collections
research-article
Public Access

Returning is Believing: Optimizing Long-term User Engagement in Recommender Systems

Published: 06 November 2017 Publication History

Abstract

In this work, we propose to improve long-term user engagement in a recommender system from the perspective of sequential decision optimization, where users' click and return behaviors are directly modeled for online optimization. A bandit-based solution is formulated to balance three competing factors during online learning, including exploitation for immediate click, exploitation for expected future clicks, and exploration of unknowns for model estimation. We rigorously prove that with a high probability our proposed solution achieves a sublinear upper regret bound in maximizing cumulative clicks from a population of users in a given period of time, while a linear regret is inevitable if a user's temporal return behavior is not considered when making the recommendations. Extensive experimentation on both simulations and a large-scale real-world dataset collected from Yahoo frontpage news recommendation log verified the effectiveness and significant improvement of our proposed algorithm compared with several state-of-the-art online learning baselines for recommendation.

References

[1]
Yasin Abbasi-yadkori, David Pal, and Csaba Szepesvari. 2011. Improved Algorithms for Linear Stochastic Bandits. In NIPS '11. 2312--2320.
[2]
Peter Auer. 2002. Using Confidence Bounds for Exploitation-Exploration Tradeo's. JMLR 3 (2002), 397--422.
[3]
Peter Auer, Nicolo Cesa-Bianchi, and Paul Fischer. 2002. Finite-time Analysis of ' the Multiarmed Bandit Problem. Mach. Learn. 47, 2--3 (May 2002), 235--256.
[4]
P. Auer, N. Cesa-Bianchi, Y. Freund, and R. E. Schapire. 1995. Gambling in a rigged casino: The adversarial multi-armed bandit problem. In 36th FOCS. 322--331.
[5]
Nicola Barbieri, Fabrizio Silvestri, and Mounia Lalmas. 2016. Improving PostClick User Engagement on Native Ads via Survival Analysis. In WWW '16. 761--770.
[6]
Nicolo Cesa-Bianchi, Claudio Gentile, and Giovanni Zappella. 2013. A gang of bandits. (2013), 737--745.
[7]
Olivier Chapelle. 2014. Modeling Delayed Feedback in Display Advertising. In KDD '14. ACM, New York, NY, USA, 1097--1105.
[8]
Abhinandan S. Das, Mayur Datar, Ashutosh Garg, and Shyam Rajaram. 2007. Google news personalization: scalable online collaborative filtering. In WWW '07. 271--280.
[9]
Nan Du, Hanjun Dai, Rakshit Trivedi, Utkarsh Upadhyay, Manuel GomezRodriguez, and Le Song. 2016. Recurrent marked temporal point processes: Embedding event history to vector. In KDD '16. ACM, 1555--1564.
[10]
Nan Du, Yichen Wang, Niao He, Jimeng Sun, and Le Song. 2015. Time-Sensitive Recommendation From Recurrent User Activities. In NIPS '15. 3492--3500.
[11]
Sarah Filippi, Olivier Cappe, Aurelien Garivier, and Csaba Szepesvari. 2010. Parametric bandits: The generalized linear case. In NIPS. 586--594.
[12]
J. Giffins, K. Glazebrook, and R. Weber. 2011. Multi-armed Bandit Allocation Indices. John Wiley & Sons.
[13]
L.P. Kaelbling, M.L. Littman, and A.W. Moore. 1996. Reinforcement learning: A survey. Journal of Artificial Intelligence 4, 1 (1996), 237--285.
[14]
Komal Kapoor, Karthik Subbian, Jaideep Srivastava, and Paul Schrater. 2015. Just in Time Recommendations: Modeling the Dynamics of Boredom in Activity Streams. In WSDM '15. ACM, New York, NY, USA, 233--242.
[15]
Komal Kapoor, Mingxuan Sun, Jaideep Srivastava, and Tao Ye. 2014. A Hazard Based Approach to User Return Time Prediction. In KDD '14. 1719--1728.
[16]
Y. Koren, R. Bell, and C. Volinsky. 2009. Matrix factorization techniques for recommender systems. Computer 42, 8 (2009), 30--37.
[17]
Mounia Lalmas, Janette Lehmann, Guy Shaked, Fabrizio Silvestri, and Gabriele Tolomei. 2015. Promoting Positive Post-Click Experience for In-Stream Yahoo Gemini Users. In KDD '15. ACM, New York, NY, USA, 1929--1938.
[18]
Mounia Lalmas, Heather O'Brien, and Elad Yom-Tov. 2014. Measuring user engagement. Synthesis Lectures on Information Concepts, Retrieval, and Services 6, 4 (2014), 1--132.
[19]
Lihong Li, Wei Chu, John Langford, and Robert E Schapire. 2010. A contextual bandit approach to personalized news article recommendation. In WWW '10. ACM, 661--670.
[20]
Lihong Li, Wei Chu, John Langford, and Xuanhui Wang. 2011. Unbiased Offine Evaluation of Contextual-bandit-based News Article Recommendation Algorithms. In WSDM '11. ACM, New York, NY, USA, 297--306.
[21]
Lihong Li, Jin Young Kim, and Imed Zitouni. 2015. Toward predicting the outcome of an A/B experiment for search relevance. In WSDM. 37--46.
[22]
Peter McCullagh. 1984. Generalized Linear Models. European Journal of Operational Research 16, 3 (1984), 285--292.
[23]
Heather L O'Brien and Elaine G Toms. 2008. What is user engagement? A conceptual framework for defining user engagement with technology. JASIST 59, 6 (2008), 938--955.
[24]
Badrul Sarwar, George Karypis, Joseph Konstan, and John Riedl. 2001. Item-based collaborative filtering recommendation algorithms. In WWW '01. 285--295.
[25]
Alex Strehl, John Langford, Lihong Li, and Sham M Kakade. 2010. Learning from logged implicit exploration data. In NIPS. 2217--2225.
[26]
X. Su and T.M. Khoshgoftaar. 2009. A survey of collaborative filtering techniques. AAI 2009 (2009), 4.
[27]
Qingyun Wu, Huazheng Wang, Quanquan Gu, and Hongning Wang. 2016. Contextual Bandits in a Collaborative Environment. In SIGIR '16. 529--538.
[28]
Xing Yi, Liangjie Hong, Erheng Zhong, Nanthan Nan Liu, and Suju Rajan. 2014. Beyond Clicks: Dwell Time for Personalization. In RecSys '14. 113--120.

Cited By

View all
  • (2024)Embedding Democratic Values into Social Media AIs via Societal Objective FunctionsProceedings of the ACM on Human-Computer Interaction10.1145/36410028:CSCW1(1-36)Online publication date: 26-Apr-2024
  • (2024)Modeling User Retention through Generative Flow NetworksProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671531(5497-5508)Online publication date: 25-Aug-2024
  • (2024)System-2 Recommenders: Disentangling Utility and Engagement in Recommendation Systems via Temporal Point-ProcessesProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency10.1145/3630106.3659004(1763-1773)Online publication date: 3-Jun-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CIKM '17: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management
November 2017
2604 pages
ISBN:9781450349185
DOI:10.1145/3132847
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 November 2017

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. content recommendation
  2. contextual bandit algorithm
  3. user long-term engagement modeling

Qualifiers

  • Research-article

Funding Sources

Conference

CIKM '17
Sponsor:

Acceptance Rates

CIKM '17 Paper Acceptance Rate 171 of 855 submissions, 20%;
Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)355
  • Downloads (Last 6 weeks)67
Reflects downloads up to 28 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Embedding Democratic Values into Social Media AIs via Societal Objective FunctionsProceedings of the ACM on Human-Computer Interaction10.1145/36410028:CSCW1(1-36)Online publication date: 26-Apr-2024
  • (2024)Modeling User Retention through Generative Flow NetworksProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671531(5497-5508)Online publication date: 25-Aug-2024
  • (2024)System-2 Recommenders: Disentangling Utility and Engagement in Recommendation Systems via Temporal Point-ProcessesProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency10.1145/3630106.3659004(1763-1773)Online publication date: 3-Jun-2024
  • (2024)Sequential Recommendation for Optimizing Both Immediate Feedback and Long-term RetentionProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657829(1872-1882)Online publication date: 10-Jul-2024
  • (2024)Large Language Models are Learnable Planners for Long-Term RecommendationProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657683(1893-1903)Online publication date: 10-Jul-2024
  • (2024)Practical Bandits: An Industry PerspectiveProceedings of the 17th ACM International Conference on Web Search and Data Mining10.1145/3616855.3636449(1132-1135)Online publication date: 4-Mar-2024
  • (2024)Towards Reliable and Efficient Long-Term Recommendation with Large Foundation ModelsCompanion Proceedings of the ACM Web Conference 202410.1145/3589335.3651258(1190-1193)Online publication date: 13-May-2024
  • (2024)State of art and emerging trends on group recommender system: a comprehensive reviewInternational Journal of Multimedia Information Retrieval10.1007/s13735-024-00329-513:2Online publication date: 2-May-2024
  • (2024)Determining the Most Important Features for Designing a Smart Recommender Framework in E-Learning Systems: An Investigation Using the Delphi Study MethodData Science and Big Data Analytics10.1007/978-981-99-9179-2_40(535-548)Online publication date: 17-Mar-2024
  • (2023)Challenging social media threats using collective well-being-aware recommendation algorithms and an educational virtual companionFrontiers in Artificial Intelligence10.3389/frai.2022.6549305Online publication date: 9-Jan-2023
  • Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media