skip to main content
10.1145/3523227.3551472acmotherconferencesArticle/Chapter ViewAbstractPublication PagesrecsysConference Proceedingsconference-collections
demonstration

RecPack: An(other) Experimentation Toolkit for Top-N Recommendation using Implicit Feedback Data

Published: 13 September 2022 Publication History

Abstract

RecPack is an easy-to-use, flexible and extensible toolkit for top-N recommendation with implicit feedback data. Its goal is to support researchers with the development of their recommendation algorithms, from similarity-based to deep learning algorithms, and allow for correct, reproducible and reusable experimentation. In this demo, we give an overview of the package and show how researchers can use it to their advantage when developing recommendation algorithms.

Supplementary Material

MP4 File (RecPack An(other) Experimentation Toolkit for Top-N Recommendation using Implicit Feedback Data.mp4)
Demo video

References

[1]
2021. But is the code (re)usable?Nature Computational Science 1, 7 (01 Jul 2021), 449–449. https://doi.org/10.1038/s43588-021-00109-9
[2]
Charu C. Aggarwal. 2016. Recommender Systems: The Textbook. Springer International Publishing, Cham. https://doi.org/10.1007/978-3-319-29659-3
[3]
Vito Walter Anelli, Alejandro Bellogin, Antonio Ferrara, Daniele Malitesta, Felice Antonio Merra, Claudio Pomo, Francesco Maria Donini, and Tommaso Di Noia. 2021. Elliot: A Comprehensive and Rigorous Framework for Reproducible Recommender Systems Evaluation. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (Virtual Event, Canada) (SIGIR ’21). Association for Computing Machinery, New York, NY, USA, 2405–2414. https://doi.org/10.1145/3404835.3463245
[4]
David Ben-Shimon, Alexander Tsikinovsky, Michael Friedmann, Bracha Shapira, Lior Rokach, and Johannes Hoerle. 2015. RecSys Challenge 2015 and the YOOCHOOSE Dataset. In Proceedings of the 9th ACM Conference on Recommender Systems (Vienna, Austria) (RecSys ’15). Association for Computing Machinery, New York, NY, USA, 357–358. https://doi.org/10.1145/2792838.2798723
[5]
James Bennett and Stan Lanning. 2007. The Netflix Prize. In Proceedings of KDD Cup and Workshop 2007 (San Jose, California, USA) (KDDCup ’07). Association for Computing Machinery, New York, NY, USA, 4 pages. https://www.cs.uic.edu/~liub/KDD-cup-2007/NetflixPrize-description.pdf
[6]
Pedro G. Campos, Fernando Díez, and Iván Cantador. 2014. Time-Aware Recommender Systems: A Comprehensive Survey and Analysis of Existing Evaluation Protocols. User Modeling and User-Adapted Interaction 24, 1–2 (feb 2014), 67–119. https://doi.org/10.1007/s11257-012-9136-x
[7]
Mukund Deshpande and George Karypis. 2004. Item-Based Top-N Recommendation Algorithms. ACM Trans. Inf. Syst. 22, 1 (jan 2004), 143–177. https://doi.org/10.1145/963770.963776
[8]
Kim Falk. 2019. Practical Recommender Systems. Manning. https://www.manning.com/books/practical-recommender-systems
[9]
Free Software Foundation. 2016. GNU Affero General Public License Version 3 (AGPL-3.0). Accessed 26 July 2022.
[10]
Diksha Garg, Priyanka Gupta, Pankaj Malhotra, Lovekesh Vig, and Gautam Shroff. 2019. Sequence and Time Aware Neighborhood for Session-Based Recommendations: STAN(SIGIR’19). Association for Computing Machinery, New York, NY, USA, 1069–1072. https://doi.org/10.1145/3331184.3331322
[11]
Mouzhi Ge, Carla Delgado-Battenfeld, and Dietmar Jannach. 2010. Beyond Accuracy: Evaluating Recommender Systems by Coverage and Serendipity. In Proceedings of the Fourth ACM Conference on Recommender Systems (Barcelona, Spain) (RecSys ’10). Association for Computing Machinery, New York, NY, USA, 257–260. https://doi.org/10.1145/1864708.1864761
[12]
Mihajlo Grbovic, Vladan Radosavljevic, Nemanja Djuric, Narayan Bhamidipati, Jaikit Savla, Varun Bhagwan, and Doug Sharp. 2015. E-Commerce in Your Inbox: Product Recommendations at Scale. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (Sydney, NSW, Australia) (KDD ’15). Association for Computing Machinery, New York, NY, USA, 1809–1818. https://doi.org/10.1145/2783258.2788627
[13]
Jon Atle Gulla, Lemei Zhang, Peng Liu, Özlem Özgöbek, and Xiaomeng Su. 2017. The Adressa Dataset for News Recommendation. In Proceedings of the International Conference on Web Intelligence (Leipzig, Germany) (WI ’17). Association for Computing Machinery, New York, NY, USA, 1042–1048. https://doi.org/10.1145/3106426.3109436
[14]
F. Maxwell Harper and Joseph A. Konstan. 2015. The MovieLens Datasets: History and Context. ACM Trans. Interact. Intell. Syst. 5, 4, Article 19 (dec 2015), 19 pages. https://doi.org/10.1145/2827872
[15]
Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural Collaborative Filtering. In Proceedings of the 26th International Conference on World Wide Web (Perth, Australia) (WWW ’17). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 173–182. https://doi.org/10.1145/3038912.3052569
[16]
Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. 2016. Session-based Recommendations with Recurrent Neural Networks. In 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings (San Juan, Puerto Rico), Yoshua Bengio and Yann LeCun (Eds.). https://doi.org/10.48550/ARXIV.1511.06939
[17]
Yifan Hu, Yehuda Koren, and Chris Volinsky. 2008. Collaborative Filtering for Implicit Feedback Datasets. In 2008 Eighth IEEE International Conference on Data Mining. 263–272. https://doi.org/10.1109/ICDM.2008.22
[18]
Michael Kechinov. 2020. CosmeticsShop E-commerce Dataset. https://www.kaggle.com/mkechinov/ecommerce-events-history-in-cosmetics-shop Accessed: 2022-07-26.
[19]
Dawen Liang, Rahul G. Krishnan, Matthew D. Hoffman, and Tony Jebara. 2018. Variational Autoencoders for Collaborative Filtering. In Proceedings of the 2018 World Wide Web Conference (Lyon, France) (WWW ’18). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 689–698. https://doi.org/10.1145/3178876.3186150
[20]
Nathan N. Liu, Min Zhao, Evan Xiang, and Qiang Yang. 2010. Online Evolutionary Collaborative Filtering. In Proceedings of the Fourth ACM Conference on Recommender Systems (Barcelona, Spain) (RecSys ’10). Association for Computing Machinery, New York, NY, USA, 95–102. https://doi.org/10.1145/1864708.1864729
[21]
Netflix. 2019. Netflix Prize Data. https://www.kaggle.com/datasets/netflix-inc/netflix-prize-data Accessed: 2022-07-26.
[22]
Xia Ning and George Karypis. 2011. SLIM: Sparse Linear Methods for Top-N Recommender Systems. In 2011 IEEE 11th International Conference on Data Mining. 497–506. https://doi.org/10.1109/ICDM.2011.134
[23]
Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian Personalized Ranking from Implicit Feedback. In Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence (Montreal, Quebec, Canada) (UAI ’09). AUAI Press, Arlington, Virginia, USA, 452–461.
[24]
Retailrocket. 2017. Retailrocket Recommender System Dataset. https://www.kaggle.com/datasets/retailrocket/ecommerce-dataset Accessed: 2022-07-26.
[25]
Francesco Ricci, Lior Rokach, Bracha Shapira, and Paul B. Kantor. 2011. Recommender Systems Handbook. Springer US, Boston, MA. https://doi.org/10.1007/978-0-387-85820-3
[26]
Badrul Sarwar, George Karypis, Joseph Konstan, and John Riedl. 2000. Analysis of Recommendation Algorithms for E-Commerce. In Proceedings of the 2nd ACM Conference on Electronic Commerce (Minneapolis, Minnesota, USA) (EC ’00). Association for Computing Machinery, New York, NY, USA, 158–167. https://doi.org/10.1145/352871.352887
[27]
Ilya Shenbin, Anton Alekseev, Elena Tutubalina, Valentin Malykh, and Sergey I. Nikolenko. 2020. RecVAE: A New Variational Autoencoder for Top-N Recommendations with Implicit Feedback. Association for Computing Machinery, New York, NY, USA, 528–536. https://doi.org/10.1145/3336191.3371831
[28]
Harald Steck. 2019. Embarrassingly Shallow Autoencoders for Sparse Data. In The World Wide Web Conference (San Francisco, CA, USA) (WWW ’19). Association for Computing Machinery, New York, NY, USA, 3251–3257. https://doi.org/10.1145/3308558.3313710
[29]
Zhu Sun, Di Yu, Hui Fang, Jie Yang, Xinghua Qu, Jie Zhang, and Cong Geng. 2020. Are We Evaluating Rigorously? Benchmarking Recommendation for Reproducible Evaluation and Fair Comparison. Association for Computing Machinery, New York, NY, USA, 23–32. https://doi.org/10.1145/3383313.3412489
[30]
Ana Trisovic, Matthew K. Lau, Thomas Pasquier, and Mercè Crosas. 2022. A large-scale study on research code quality and execution. Scientific Data 9, 1 (21 Feb 2022), 60. https://doi.org/10.1038/s41597-022-01143-6
[31]
Koen Verstrepen and Bart Goethals. 2014. Unifying Nearest Neighbors Collaborative Filtering. In Proceedings of the 8th ACM Conference on Recommender Systems (Foster City, Silicon Valley, California, USA) (RecSys ’14). Association for Computing Machinery, New York, NY, USA, 177–184. https://doi.org/10.1145/2645710.2645731
[32]
Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, Stéfan J. van der Walt, Matthew Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. J. Nelson, Eric Jones, Robert Kern, Eric Larson, C J Carey, İlhan Polat, Yu Feng, Eric W. Moore, Jake VanderPlas, Denis Laxalde, Josef Perktold, Robert Cimrman, Ian Henriksen, E. A. Quintero, Charles R. Harris, Anne M. Archibald, Antônio H. Ribeiro, Fabian Pedregosa, Paul van Mulbregt, and SciPy 1.0 Contributors. 2020. SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python. Nature Methods 17(2020), 261–272. https://doi.org/10.1038/s41592-019-0686-2
[33]
Hao Wang, Binyi Chen, and Wu-Jun Li. 2013. Collaborative Topic Regression with Social Regularization for Tag Recommendation. In Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence (Beijing, China) (IJCAI ’13). AAAI Press, 2719–2725.
[34]
Wes McKinney. 2010. Data Structures for Statistical Computing in Python. In Proceedings of the 9th Python in Science Conference, Stéfan van der Walt and Jarrod Millman (Eds.). 56 – 61. https://doi.org/10.25080/Majora-92bf1922-00a
[35]
Wayne Xin Zhao, Junhua Chen, Pengfei Wang, Qi Gu, and Ji-Rong Wen. 2020. Revisiting Alternative Experimental Settings for Evaluating Top-N Item Recommendation Algorithms. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management (Virtual Event, Ireland) (CIKM ’20). Association for Computing Machinery, New York, NY, USA, 2329–2332. https://doi.org/10.1145/3340531.3412095

Cited By

View all
  • (2025)A Reproducible Analysis of Sequential Recommender SystemsIEEE Access10.1109/ACCESS.2024.352204913(5762-5772)Online publication date: 2025
  • (2024)Recommender Systems Algorithm Selection for Ranking Prediction on Implicit Feedback DatasetsProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3691718(1163-1167)Online publication date: 8-Oct-2024
  • (2024)beeFormer: Bridging the Gap Between Semantic and Interaction Similarity in Recommender SystemsProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3691707(1102-1107)Online publication date: 8-Oct-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
RecSys '22: Proceedings of the 16th ACM Conference on Recommender Systems
September 2022
743 pages
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 September 2022

Check for updates

Author Tags

  1. Python
  2. evaluation
  3. implicit feedback data
  4. open-source framework
  5. top-N recommendation

Qualifiers

  • Demonstration
  • Research
  • Refereed limited

Funding Sources

Conference

Acceptance Rates

Overall Acceptance Rate 254 of 1,295 submissions, 20%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)117
  • Downloads (Last 6 weeks)7
Reflects downloads up to 15 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2025)A Reproducible Analysis of Sequential Recommender SystemsIEEE Access10.1109/ACCESS.2024.352204913(5762-5772)Online publication date: 2025
  • (2024)Recommender Systems Algorithm Selection for Ranking Prediction on Implicit Feedback DatasetsProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3691718(1163-1167)Online publication date: 8-Oct-2024
  • (2024)beeFormer: Bridging the Gap Between Semantic and Interaction Similarity in Recommender SystemsProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3691707(1102-1107)Online publication date: 8-Oct-2024
  • (2024)RePlay: a Recommendation Framework for Experimentation and Production UseProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3691701(1191-1194)Online publication date: 8-Oct-2024
  • (2024)ReChorus2.0: A Modular and Task-Flexible Recommendation LibraryProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3688076(454-464)Online publication date: 8-Oct-2024
  • (2024)From Clicks to Carbon: The Environmental Toll of Recommender SystemsProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3688074(580-590)Online publication date: 8-Oct-2024
  • (2024)On (Normalised) Discounted Cumulative Gain as an Off-Policy Evaluation Metric for Top-n RecommendationProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671687(1222-1233)Online publication date: 25-Aug-2024
  • (2024)Optimizing Neighborhoods for Fair Top-N RecommendationProceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization10.1145/3627043.3659539(57-66)Online publication date: 22-Jun-2024
  • (2024)A Framework and Toolkit for Testing the Correctness of Recommendation AlgorithmsACM Transactions on Recommender Systems10.1145/35911092:1(1-45)Online publication date: 7-Mar-2024
  • (2023)Reproducibility Analysis of Recommender Systems relying on Visual Features: traps, pitfalls, and countermeasuresProceedings of the 17th ACM Conference on Recommender Systems10.1145/3604915.3609492(554-564)Online publication date: 14-Sep-2023
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media