Skip to main content
SpringerLink
Log in

Replication and Reproduction in Recommender Systems Research - Evidence from a Case-Study with the rrecsys Library

  • Conference paper
  • First Online:
Advances in Artificial Intelligence: From Theory to Practice (IEA/AIE 2017)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 10350))

Abstract

Recommender systems (RS) are a real-world application domain for Artificial Intelligence standing at the core of massively used e-commerce and social-media platforms like Amazon, Netflix, Spotify and many more. The research field of recommendation systems now has already a more than 20 years long tradition and issues like replication of results and reproducibility of algorithms become more important. Therefore this work is oriented towards better understanding the underlying challenges of reproducibility of offline measurements of recommendation techniques. We therefore introduce rrecsys, an open-source package in R, that implements many popular RS algorithms, expansion capabilities and has an integrated offline evaluation mechanism following an accepted methodology. In addition, we present a case study on the usability of the library along with results of benchmarking the provided algorithms with other open-source implementations.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://rival.recommenders.net/.

  2. 2.

    https://cran.r-project.org/web/packages/rrecsys/vignettes/d1_extend.html.

  3. 3.

    http://grouplens.org/datasets/movielens/latest/.

  4. 4.

    http://grouplens.org/datasets/movielens/1m/.

  5. 5.

    http://sifter.org/~simon/journal/20061211.html.

  6. 6.

    https://CRAN.R-project.org/package=recommenderlab.

  7. 7.

    https://CRAN.R-project.org/package=irlba.

  8. 8.

    http://grouplens.org/datasets/movielens/100k/.

  9. 9.

    https://github.com/ludovikcoba/iea2017.

  10. 10.

    http://validation.scienceexchange.com/.

References

  1. Beel, J., Breitinger, C., Langer, S., Lommatzsch, A., Gipp, B.: Towards reproducibility in recommender-systems research. User Model. User Adap. Inter. 26(1), 69–101 (2016)

    Article  Google Scholar 

  2. Çoba, L., Zanker, M.: rrecsys: an r-package for prototyping recommendation algorithms. In: Guy, I., Sharma, A. (eds.) Poster Track of the 10th ACM Conference on Recommender Systems (RecSys 2016) (RecSysPosters). No. 1688 in CEUR Workshop Proceedings, Aachen (2016). http://ceur-ws.org/Vol-1688/#paper-12

  3. Eddelbuettel, D., François, R., Allaire, J., Chambers, J., Bates, D., Ushey, K.: Rcpp: Seamless r and C++ integration. J. Stat. Softw. 40(8), 1–18 (2011)

    Article  Google Scholar 

  4. Ekstrand, M.D., Ludwig, M., Konstan, J.A., Riedl, J.T.: Rethinking the recommender research ecosystem: reproducibility, openness, and lenskit. In: Proceedings of the Fifth ACM Conference on Recommender Systems, pp. 133–140. ACM (2011)

    Google Scholar 

  5. Harper, F.M., Konstan, J.A.: The movielens datasets: History and context. ACM Trans. Interact. Intell. Syst. (TiiS) 5(4), 19 (2016)

    Google Scholar 

  6. Jannach, D., Lerche, L., Gedikli, F., Bonnin, G.: What recommenders recommend – an analysis of accuracy, popularity, and sales diversity effects. In: Carberry, S., Weibelzahl, S., Micarelli, A., Semeraro, G. (eds.) UMAP 2013. LNCS, vol. 7899, pp. 25–37. Springer, Heidelberg (2013). doi:10.1007/978-3-642-38844-6_3

    Chapter  Google Scholar 

  7. Jannach, D., Resnick, P., Tuzhilin, A., Zanker, M.: Recommender systems—: beyond matrix completion. Commun. ACM 59(11), 94–102 (2016)

    Article  Google Scholar 

  8. Koren, Y., Bell, R., Volinsky, C., et al.: Matrix factorization techniques for recommender systems. Computer 42(8), 30–37 (2009)

    Article  Google Scholar 

  9. Lemire, D., Maclachlan, A.: Slope one predictors for online rating-based collaborative filtering. In: SDM, vol. 5, pp. 1–5. SIAM (2005)

    Google Scholar 

  10. Lops, P., De Gemmis, M., Semeraro, G.: Content-based recommender systems: state of the art and trends. In: Ricci, F., Rokach, L., Shapira, B., Kantor, P.B. (eds.) Recommender Systems Handbook, pp. 73–105. Springer, New York (2011)

    Google Scholar 

  11. Pan, R., Zhou, Y., Cao, B., Liu, N.N., Lukose, R., Scholz, M., Yang, Q.: One-class collaborative filtering. In: 2008 Eighth IEEE International Conference on Data Mining, pp. 502–511. IEEE (2008)

    Google Scholar 

  12. Ricci, F., Rokach, L., Shapira, B.: Introduction to recommender systems handbook. Springer, New York (2011)

    Google Scholar 

  13. Rossetti, M., Stella, F., Zanker, M.: Contrasting offline and online results when evaluating recommendation algorithms. In: Proceedings of the 10th ACM Conference on Recommender Systems, pp. 31–34. ACM (2016)

    Google Scholar 

  14. Said, A., Bellogín, A.: Comparative recommender system evaluation: benchmarking recommendation frameworks. In: Proceedings of the 8th ACM Conference on Recommender systems, pp. 129–136. ACM (2014)

    Google Scholar 

  15. Sarwar, B., Karypis, G., Konstan, J., Riedl, J.: Item-based collaborative filtering recommendation algorithms. In: Proceedings of the 10th International Conference on World Wide Web, pp. 285–295. ACM (2001)

    Google Scholar 

  16. Shani, G., Gunawardana, A.: Evaluating recommendation systems. In: Ricci, F., Rokach, L., Shapira, B., Kantor, P.B. (eds.) Recommender Systems Handbook, pp. 257–297. Springer, New York (2011)

    Google Scholar 

  17. Zanker, M., Jessenitschnig, M., Jannach, D., Gordea, S.: Comparing recommendation strategies in a commercial context. IEEE Intell. Syst. 22(3), 69–73 (2007)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Free University of Bozen-Bolzano, 39100, Bozen-Bolzano, Italy

    Ludovik Çoba & Markus Zanker

Authors
  1. Ludovik Çoba
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Markus Zanker
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ludovik Çoba .

Editor information

Editors and Affiliations

  1. Artois University, Lens, France

    Salem Benferhat

  2. Artois University, Lens, France

    Karim Tabia

  3. Texas State University, San Marcos, Texas, USA

    Moonis Ali

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Çoba, L., Zanker, M. (2017). Replication and Reproduction in Recommender Systems Research - Evidence from a Case-Study with the rrecsys Library. In: Benferhat, S., Tabia, K., Ali, M. (eds) Advances in Artificial Intelligence: From Theory to Practice. IEA/AIE 2017. Lecture Notes in Computer Science(), vol 10350. Springer, Cham. https://doi.org/10.1007/978-3-319-60042-0_36

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-60042-0_36

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-60041-3

  • Online ISBN: 978-3-319-60042-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation