short-paper

PyRecGym: a reinforcement learning gym for recommender systems

Authors:

Bichen Shi,

Makbule Gulcin Ozsoy,

Aonghus LawlorAuthors Info & Claims

RecSys '19: Proceedings of the 13th ACM Conference on Recommender Systems

Pages 491 - 495

https://doi.org/10.1145/3298689.3346981

Published: 10 September 2019 Publication History

Get Access

Abstract

Recommender systems (RS) share many features and objectives with reinforcement learning (RL) systems. The former aim to maximise user satisfaction by recommending the right items to the right users at the right time, the latter maximise future rewards by selecting state-changing actions in some environment. The concept of an RL gym has become increasingly important when it comes to supporting the development of RL models. A gym provides a simulation environment in which to test and develop RL agents, providing a state model, actions, rewards/penalties etc. In this paper we describe and demonstrate the PyRecGym gym, which is specifically designed for the needs of recommender systems research, by supporting standard test datasets (MovieLens, Yelp etc.), common input types (text, numeric etc.), and thereby offering researchers a reproducible research environment to accelerate experimentation and development of RL in RS.

References

[1]

2019. A Beginner's Guide to Deep Reinforcement Learning. https://skymind.ai/wiki/deep-reinforcement-learning

Google Scholar

[2]

2019. Reinforcement Learning (Part 1) The Mario Bros Example. https://cai.tools.sap/blog/the-future-with-reinforcement-learning-part-1/

Google Scholar

[3]

Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. 2016. OpenAI Gym. CoRR abs/1606.01540 (2016). arXiv:1606.01540

Google Scholar

[4]

Shi-Yong Chen, Yang Yu, Qing Da, Jun Tan, Hai-Kuan Huang, and Hai-Hong Tang. 2018. Stabilizing Reinforcement Learning in Dynamic Environment with Application to Online Recommendation. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD '18). ACM, 1187--1196.

Digital Library

Google Scholar

[5]

Sungwoon Choi, Heonseok Ha, Uiwon Hwang, Chanju Kim, Jung-Woo Ha, and Sungroh Yoon. 2018. Reinforcement Learning based Recommender System using Biclustering Technique. CoRR abs/1801.05532 (2018). arXiv:1801.05532

Google Scholar

[6]

David Cortes. 2018. Adapting multi-armed bandits policies to contextual bandits scenarios. CoRR abs/1811.04383 (2018). arXiv:1811.04383

Google Scholar

[7]

Lihong Li, Wei Chu, John Langford, and Robert E Schapire. 2010. A contextual-bandit approach to personalized news article recommendation. In Proceedings of the 19th international conference on World wide web. ACM, 661--670.

Digital Library

Google Scholar

[8]

David Rohde, Stephen Bonner, Travis Dunlop, Flavian Vasile, and Alexandros Karatzoglou. 2018. RecoGym: A Reinforcement Learning Environment for the problem of Product Recommendation in Online Advertising. CoRR abs/1808.00720 (2018). arXiv:1808.00720

Google Scholar

[9]

Yingfei Wang, Hua Ouyang, Chu Wang, Jianhui Chen, Tsvetan Asamov, and Yi Chang. 2017. Efficient Ordered Combinatorial Semi-Bandits for Whole-Page Recommendation.

Digital Library

Google Scholar

[10]

Qingyun Wu, Naveen Iyer, and Hongning Wang. 2018. Learning Contextual Bandits in a Non-stationary Environment. SIGIR âĂ&Zacute;18 The 41st International ACM SIGIR Conference on Research Development in Information Retrieval (Jul 2018).

Digital Library

Google Scholar

[11]

Xiangyu Zhao, Long Xia, Jiliang Tang, and Dawei Yin. 2018. Reinforcement Learning for Online Information Seeking. CoRR abs/1812.07127 (2018). arXiv:1812.07127

Google Scholar

[12]

Xiangyu Zhao, Long Xia, Yihong Zhao, Dawei Yin, and Jiliang Tang. 2019. Model-Based Reinforcement Learning for Whole-Chain Recommendations. arXiv:cs.IR/1902.03987

Google Scholar

[13]

Guanjie Zheng, Fuzheng Zhang, Zihan Zheng, Yang Xiang, Nicholas Jing Yuan, Xing Xie, and Zhenhui Li. 2018. Drn: A deep reinforcement learning framework for news recommendation. In Proceedings of the 2018 World Wide Web Conference on World Wide Web. International World Wide Web Conferences Steering Committee, 167--176.

Digital Library

Google Scholar

Cited By

View all

Pang GXie JHao F(2025)An adaptive meta-imitation learning-based recommendation environment simulator: A case study on ship-cargo matchingInformation Fusion10.1016/j.inffus.2024.102740115(102740)Online publication date: Mar-2025
https://doi.org/10.1016/j.inffus.2024.102740
Lin YLiu YLin FZou LWu PZeng WChen HMiao C(2024)A Survey on Reinforcement Learning for Recommender SystemsIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2023.328016135:10(13164-13184)Online publication date: Oct-2024
https://doi.org/10.1109/TNNLS.2023.3280161
Miehling ENair RDaly ERamamurthy KRedmond ROh ANaumann TGloberson ASaenko KHardt MLevine S(2023)Cookie consent has disparate impact on estimation accuracyProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3667610(34308-34328)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3666122.3667610
Show More Cited By

Index Terms

PyRecGym: a reinforcement learning gym for recommender systems
1. Information systems
  1. Information retrieval
    1. Evaluation of retrieval results
    2. Retrieval tasks and goals
      1. Recommender systems
2. Theory of computation
  1. Theory and algorithms for application domains
    1. Machine learning theory
      1. Reinforcement learning
        Multi-agent reinforcement learning

Recommendations

Reinforcement Learning based Recommender Systems: A Survey
Recommender systems (RSs) have become an inseparable part of our everyday lives. They help us find our favorite items to purchase, our friends on social networks, and our favorite movies to watch. Traditionally, the recommendation problem was considered ...
Reinforcement learning for addressing the cold-user problem in recommender systems
Abstract
Recommender systems are widely used in webshops because of their ability to provide users with personalized recommendations. However, the cold-user problem (i.e., recommending items to new users) is an important issue many webshops face. With the ...
Usage-based web recommendations: a reinforcement learning approach
RecSys '07: Proceedings of the 2007 ACM conference on Recommender systems

Information overload is no longer news; the explosive growth of the Internet has made this issue increasingly serious for Web users. Users are very often overwhelmed by the huge amount of information and are faced with a big challenge to find the most ...

Comments

Information & Contributors

Information

Published In

RecSys '19: Proceedings of the 13th ACM Conference on Recommender Systems

September 2019

635 pages

ISBN:9781450362436

DOI:10.1145/3298689

General Chairs:
Toine Bogers
Aalborg University Copenhagen, Denmark
,
Alan Said
University of Gothenburg, Sweden
,
Program Chairs:
Peter Brusilovsky
University of Pittsburgh
,
Domonkos Tikk
Gravity R&D, Hungary

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 September 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

Science Foundation Ireland

Conference

RecSys '19

RecSys '19: Thirteenth ACM Conference on Recommender Systems

September 16 - 20, 2019

Copenhagen, Denmark

Acceptance Rates

RecSys '19 Paper Acceptance Rate 36 of 189 submissions, 19%;

Overall Acceptance Rate 254 of 1,295 submissions, 20%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
1,128
Total Downloads

Downloads (Last 12 months)72
Downloads (Last 6 weeks)5

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Pang GXie JHao F(2025)An adaptive meta-imitation learning-based recommendation environment simulator: A case study on ship-cargo matchingInformation Fusion10.1016/j.inffus.2024.102740115(102740)Online publication date: Mar-2025
https://doi.org/10.1016/j.inffus.2024.102740
Lin YLiu YLin FZou LWu PZeng WChen HMiao C(2024)A Survey on Reinforcement Learning for Recommender SystemsIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2023.328016135:10(13164-13184)Online publication date: Oct-2024
https://doi.org/10.1109/TNNLS.2023.3280161
Miehling ENair RDaly ERamamurthy KRedmond ROh ANaumann TGloberson ASaenko KHardt MLevine S(2023)Cookie consent has disparate impact on estimation accuracyProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3667610(34308-34328)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3666122.3667610
Tragos EO'Reilly-Morgan DGeraci JShi BSmyth BDoherty CLawlor AHurley NElkind E(2023)Keeping people active and healthy at home using a reinforcement learning-based fitness recommendation frameworkProceedings of the Thirty-Second International Joint Conference on Artificial Intelligence10.24963/ijcai.2023/692(6237-6245)Online publication date: 19-Aug-2023
https://dl.acm.org/doi/10.24963/ijcai.2023/692
Pawłowska JRydzewska KWierzbicki A(2023)Using Cognitive Models to Understand and Counteract the Effect of Self-Induced Bias on Recommendation AlgorithmsJournal of Artificial Intelligence and Soft Computing Research10.2478/jaiscr-2023-000813:2(73-94)Online publication date: 11-Mar-2023
https://doi.org/10.2478/jaiscr-2023-0008
Zhu ZQin RHuang JDai XYu YYu YZhang W(2023)Understanding or Manipulation: Rethinking Online Performance Gains of Modern Recommender SystemsACM Transactions on Information Systems10.1145/363786942:4(1-32)Online publication date: 15-Dec-2023
https://dl.acm.org/doi/10.1145/3637869
Afzali JDrzewiecki ABalog KZhang SChua TLauw HSi LTerzi ETsaparas P(2023)UserSimCRS: A User Simulation Toolkit for Evaluating Conversational Recommender SystemsProceedings of the Sixteenth ACM International Conference on Web Search and Data Mining10.1145/3539597.3573029(1160-1163)Online publication date: 27-Feb-2023
https://dl.acm.org/doi/10.1145/3539597.3573029
Chen XYao LMcAuley JZhou GWang X(2023)Deep reinforcement learning in recommender systems: A survey and new perspectivesKnowledge-Based Systems10.1016/j.knosys.2023.110335264(110335)Online publication date: Mar-2023
https://doi.org/10.1016/j.knosys.2023.110335
Afsar MCrump TFar B(2022)Reinforcement Learning based Recommender Systems: A SurveyACM Computing Surveys10.1145/354384655:7(1-38)Online publication date: 15-Dec-2022
https://dl.acm.org/doi/10.1145/3543846
Huang JOosterhuis HCetinkaya BRood Tde Rijke MAmigo ECastells PGonzalo JCarterette BCulpepper JKazai G(2022)State Encoders in Reinforcement Learning for RecommendationProceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3477495.3531716(2738-2748)Online publication date: 7-Jul-2022
https://doi.org/10.1145/3477495.3531716
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Reinforcement Learning based Recommender Systems: A Survey

Reinforcement learning for addressing the cold-user problem in recommender systems

Usage-based web recommendations: a reinforcement learning approach

Comments

Information

Published In

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Funding Sources

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations