Skip to main content
Springer Nature Link
Log in

Advances in Collaborative Filtering

  • Chapter
  • First Online:
Recommender Systems Handbook

  • 11k Accesses

Abstract

Collaborative filtering (CF) methods produce recommendations based on usage patterns without the need of exogenous information about items or users. CF algorithms have shown great prediction quality both in academic research and in industrial applications. This chapter surveys core methods in the field. Matrix factorization techniques, which became a first choice for implementing CF, are described together with other innovations. We also describe several extensions that bring competitive accuracy into neighborhood methods, which used to dominate the field. The chapter demonstrates how to utilize temporal models and implicit feedback to extend model accuracy. In passing, we illustrate the use of CF algorithms on the Netflix Prize competition. The CF methods discussed in this chapter have been proposed a decade ago but still show state-of-the art accuracy in recent studies. The modeling patterns identified in this chapter are applicable to a variety of recommender problems such as item recommendation, rating prediction, cold start recommendation and context-aware recommenders.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    Robert Bell is retired.

  2. 2.

    This article includes copyrighted materials, which were reproduced with permission of ACM and IEEE. The original articles are:

    R. Bell and Y. Koren [3], Ⓒ 2007 IEEE. Reprinted by permission.

    Y. Koren [26], Ⓒ 2008 ACM, Inc. Reprinted by permission. http://doi.acm.org/10.1145/1401890.1401944

    Y. Koren [27], Ⓒ 2009 ACM, Inc. Reprinted by permission. http://doi.acm.org/10.1145/1557019.1557072

  3. 3.

    Recall that the dot product between two vectors \(\mathbf {x},\mathbf {y} \in \mathbb {R}^f\) is defined as: \({\mathbf {x}}^T\mathbf {y} = \langle \mathbf {x}, \mathbf {y}\rangle = \sum _{k=1}^f x_k \cdot y_k\).

  4. 4.

    The item i should be excluded from the summation over R(u). To simplify notation, we omit this detail in the remainder of this section.

  5. 5.

    Notational clarification: With other neighborhood models it was beneficial to use Sk(i; u), which denotes the k items most similar to i among those rated by u. Hence, if u rated at least k items, we will always have |Sk(i; u)| = k, regardless of how similar those items are to i. However, |Rk(i; u)| is typically smaller than k, as some of those items most similar to i were not rated by u.

References

  1. D. Agarwal, B.-C. Chen, Regression-based latent factor models, in Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD’09 (Association for Computing Machinery, New York, 2009)

    MATH  Google Scholar 

  2. I. Bayer, X. He, B. Kanagal, S. Rendle, A generic coordinate descent framework for learning from implicit feedback, in Proceedings of the 26th International Conference on World Wide Web, WWW’17 (2017), pp. 1341–1350

    Google Scholar 

  3. R.M. Bell, Y. Koren, Scalable collaborative filtering with jointly derived neighborhood interpolation weights, in Seventh IEEE International Conference on Data Mining (ICDM 2007) (2007), pp. 43–52

    Google Scholar 

  4. R.M. Bell, Y. Koren, Lessons from the netflix prize challenge. SIGKDD Explor. Newsl. 9(2), 75–79 (2007)

    Article  MATH  Google Scholar 

  5. R. Bell, Y. Koren, C. Volinsky, Modeling relationships at multiple scales to improve accuracy of large recommender systems, in Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD’07 (Association for Computing Machinery, New York, 2007), pp. 95–104

    Google Scholar 

  6. J. Bennett, S. Lanning, The netflix prize, in In KDD Cup and Workshop in Conjunction with KDD (2007)

    Google Scholar 

  7. D.M. Blei, A.Y. Ng, M.I. Jordan, Latent dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)

    MATH  Google Scholar 

  8. J. Canny, Collaborative filtering with privacy via factor analysis, in Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR’02 (Association for Computing Machinery, New York, 2002), pp. 238–245

    MATH  Google Scholar 

  9. C. Chen, D. Li, Y. Zhao, Q. Lv, L. Shang, WEMAREC: accurate and scalable recommendation through weighted and ensemble matrix approximation, in Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR’15 (ACM, New York, 2015), pp. 303–312

    Google Scholar 

  10. C. Chen, D. Li, Q. Lv, J. Yan, S.M. Chu, L. Shang, MPMA: mixture probabilistic matrix approximation for collaborative filtering, in Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI’16 (AAAI Press, Palo Alto, 2016), pp. 1382–1388

    Google Scholar 

  11. C. Chen, D. Li, Q. Lv, J. Yan, L. Shang, S. Chu, GLOMA: embedding global information in local matrix approximation models for collaborative filtering, in AAAI Conference on Artificial Intelligence (2017)

    Google Scholar 

  12. P. Covington, J. Adams, E. Sargin, Deep neural networks for youtube recommendations, in Proceedings of the 10th ACM Conference on Recommender Systems, RecSys’16 (Association for Computing Machinery, New York, 2016), pp. 191–198

    Google Scholar 

  13. P. Cremonesi, Y. Koren, R. Turrin, Performance of recommender algorithms on top-n recommendation tasks, in Proceedings of the Fourth ACM Conference on Recommender Systems, RecSys’10 (Association for Computing Machinery, New York, 2010), pp. 39–46

    Book  MATH  Google Scholar 

  14. M.F. Dacrema, S. Boglio, P. Cremonesi, D. Jannach, A troubling analysis of reproducibility and progress in recommender systems research. ACM Trans. Inf. Syst. 39(2), Article 20 (2021). https://doi.org/10.1145/3434185

  15. A.S. Das, M. Datar, A. Garg, S. Rajaram, Google news personalization: scalable online collaborative filtering, in Proceedings of the 16th International Conference on World Wide Web, WWW’07 (Association for Computing Machinery, New York, 2007), pp. 271–280

    Google Scholar 

  16. S. Deerwester, S.T. Dumais, G.W. Furnas, T.K. Landauer, R. Harshman, Indexing by latent semantic analysis. J. Am. Soc. Inf. Sci. 41(6), 391–407 (1990)

    Article  Google Scholar 

  17. C. Freudenthaler, L. Schmidt-Thieme, S. Rendle, Bayesian factorization machines, in Proceedings of the NIPS Workshop on Sparse Representation and Low-rank Approximation (2011)

    Google Scholar 

  18. S. Funk, Netflix update: try this at home (2006). http://sifter.org/~simon/journal/20061211.html

  19. A. Gelman, J.B. Carlin, H.S. Stern, D.B. Rubin, Bayesian Data Analysis (Chapman and Hall, London, 1995)

    Book  MATH  Google Scholar 

  20. J.L. Herlocker, J.A. Konstan, A. Borchers, J. Riedl, An algorithmic framework for performing collaborative filtering, in Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR’99 (Association for Computing Machinery, New York, 1999), pp. 230–237

    Google Scholar 

  21. J.L. Herlocker, J.A. Konstan, J. Riedl, Explaining collaborative filtering recommendations, in Proceedings of the 2000 ACM Conference on Computer Supported Cooperative Work, CSCW’00 (Association for Computing Machinery, New York, 2000), pp. 241–250

    MATH  Google Scholar 

  22. S. Hochreiter, J. Schmidhuber, Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  MATH  Google Scholar 

  23. T. Hofmann, Latent semantic models for collaborative filtering. ACM Trans. Inf. Syst. 22(1), 89–115 (2004)

    Article  MATH  Google Scholar 

  24. Y. Hu, Y. Koren, C. Volinsky, Collaborative filtering for implicit feedback datasets, in Proceedings of the 2008 Eighth IEEE International Conference on Data Mining, ICDM’08 (2008), pp. 263–272

    Google Scholar 

  25. D. Kim, B.-J. Yum, Collaborative filtering based on iterative principal component analysis. Expert Syst. Appl. 28(4), 823–830 (2005)

    Article  MATH  Google Scholar 

  26. Y. Koren, Factorization meets the neighborhood: a multifaceted collaborative filtering model, in Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD’08 (Association for Computing Machinery, New York, 2008), pp. 426–434

    MATH  Google Scholar 

  27. Y. Koren, Collaborative filtering with temporal dynamics, in Proceedings of 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (ACM, New York, 2009), pp. 447–456

    MATH  Google Scholar 

  28. Y. Koren, Factor in the neighbors: scalable and accurate collaborative filtering. ACM Trans. Knowl. Discov. Data 4(1), 1–24 (2010)

    Article  MATH  Google Scholar 

  29. J. Lee, S. Kim, G. Lebanon, Y. Singer, Local low-rank matrix approximation, in Proceedings of the 30th International Conference on International Conference on Machine Learning - Volume 28, ICM’13. JMLR.org. (2013), pp. II–82–II–90

    Google Scholar 

  30. D. Li, C. Chen, W. Liu, T. Lu, N. Gu, S. Chu, Mixture-rank matrix approximation for collaborative filtering, in Advances in Neural Information Processing Systems 30, ed. by I. Guyon, U.V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, R. Garnett (Curran Associates, Red Hook, 2017), pp. 477–485

    MATH  Google Scholar 

  31. D. Li, C. Chen, Q. Lv, J. Yan, L. Shang, S.M. Chu, Low-rank matrix approximation with stability, in Proceedings of the 33rd International Conference on Machine Learning - Volume 48, ICML’16. JMLR.org (2016), pp. 295–303

    Google Scholar 

  32. D. Li, C. Chen, Q. Lv, L. Shang, S. Chu, H. Zha, ERMMA: expected risk minimization for matrix approximation-based recommender systems, in AAAI Conference on Artificial Intelligence (2017)

    Google Scholar 

  33. D. Li, C. Chen, Q. Lv, H. Gu, T. Lu, L. Shang, N. Gu, S.M. Chu, AdaError: an adaptive learning rate method for matrix approximation-based collaborative filtering, in Proceedings of the 2018 World Wide Web Conference, WWW’18, Republic and Canton of Geneva, Switzerland. International World Wide Web Conferences Steering Committee (2018), pp. 741–751

    Book  MATH  Google Scholar 

  34. D. Liang, R.G. Krishnan, M.D. Hoffman, T. Jebara, Variational autoencoders for collaborative filtering, in Proceedings of the 2018 World Wide Web Conference, WWW’18, Republic and Canton of Geneva, CHE. International World Wide Web Conferences Steering Committee (2018)

    MATH  Google Scholar 

  35. G. Linden, B. Smith, J. York, Amazon.com recommendations: item-to-item collaborative filtering. IEEE Internet Comput. 7(1), 76–80 (2003)

    Google Scholar 

  36. B.M. Marlin, R.S. Zemel, S. Roweis, M. Slaney, Collaborative filtering and the missing at random assumption, in Proceedings of the Twenty-Third Conference on Uncertainty in Artificial Intelligence, UAI’07 (AUAI Press, Arlington, 2007), pp. 267–275

    Google Scholar 

  37. X. Ning, G. Karypis, SLIM: Sparse linear methods for top-n recommender systems, in Proceedings of the 2011 IEEE 11th International Conference on Data Mining, ICDM’11 (IEEE Computer Society, Washington, 2011), pp. 497–506

    Google Scholar 

  38. D.W. Oard, J. Kim, Implicit feedback for recommender systems, in Proceedings of 5th DELOS Workshop on Filtering and Collaborative Filtering (1998), pp. 31–36

    Google Scholar 

  39. A. Paterek, Improving regularized singular value decomposition for collaborative filtering, in Proceedings of KDD Cup and Workshop (2007)

    Google Scholar 

  40. S. Rendle, Factorization machines, in Proceedings of the 2010 IEEE International Conference on Data Mining, ICDM’10 (IEEE Computer Society, Washington, 2010), pp. 995–1000

    Google Scholar 

  41. S. Rendle, Factorization machines with libFM. ACM Trans. Intell. Syst. Technol. 3(3), 57:1–57:22 (2012)

    Google Scholar 

  42. S. Rendle, Scaling factorization machines to relational data, in Proceedings of the 39th International Conference on Very Large Data Bases, PVLDB’13. VLDB Endowment (2013), pp. 337–348

    Google Scholar 

  43. S. Rendle, C. Freudenthaler, Z. Gantner, L. Schmidt-Thieme, BPR: Bayesian personalized ranking from implicit feedback, in Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence, UAI’09 (AUAI Press, Arlington, 2009), pp. 452–461

    Google Scholar 

  44. S. Rendle, L. Zhang, Y. Koren, On the difficulty of evaluating baselines: a study on recommender systems. CoRR, abs/1905.01395 (2019)

    Google Scholar 

  45. R. Salakhutdinov, A. Mnih, Probabilistic matrix factorization, in Proceedings of the 20th International Conference on Neural Information Processing Systems, NIPS’07 (Curran Associates, Red Hook, 2007), pp. 1257–1264

    Google Scholar 

  46. R. Salakhutdinov, A. Mnih, G. Hinton, Restricted boltzmann machines for collaborative filtering, in Proceedings of the 24th International Conference on Machine Learning, ICML’07 (Association for Computing Machinery, New York, 2007), pp. 791–798

    MATH  Google Scholar 

  47. B.M. Sarwar, G. Karypis, J.A. Konstan, J.T. Riedl, Application of dimensionality reduction in recommender system – a case study, in WEBKDD’2000 (2000)

    Google Scholar 

  48. B. M. Sarwar, G. Karypis, J. Konstan, J. Riedl, Item-based collaborative filtering recommendation algorithms, in Proceedings of the 10th International Conference on World Wide Web, WWW’01 (Association for Computing Machinery, New York, 2001), pp. 285–295

    MATH  Google Scholar 

  49. S. Sedhain, A.K. Menon, S. Sanner, L. Xie, AutoRec: autoencoders meet collaborative filtering, in Proceedings of the 24th International Conference on World Wide Web, WWW’15 Companion (ACM, New York, 2015), pp. 111–112

    Google Scholar 

  50. H. Steck, Embarrassingly shallow autoencoders for sparse data, in The World Wide Web Conference, WWW’19 (Association for Computing Machinery, New York, 2019), pp. 3251–3257

    Book  MATH  Google Scholar 

  51. F. Strub, J. Mary, R. Gaudel, Hybrid recommender system based on autoencoders. CoRR, abs/1606.07659 (2016)

    Google Scholar 

  52. G. Takács, I. Pilászy, B. Németh, D. Tikk, Major components of the gravity recommendation system. SIGKDD Explor. Newsl. 9(2), 80–83 (2007)

    Article  MATH  Google Scholar 

  53. G. Takács, I. Pilászy, B. Németh, D. Tikk, Matrix factorization and neighbor based algorithms for the netflix prize problem, in Proceedings of the 2008 ACM Conference on Recommender Systems, RecSys’08 (Association for Computing Machinery, New York, 2008), pp. 267–274

    Google Scholar 

  54. N. Tintarev, J. Masthoff, A survey of explanations in recommender systems, in Proceedings of the 2007 IEEE 23rd International Conference on Data Engineering Workshop, ICDEW’07 (IEEE Computer Society, Washington, 2007), pp. 801–810

    Google Scholar 

  55. A. Töscher, M. Jahrer, R. Legenstein, Improved neighborhood-based algorithms for large-scale recommender systems, in Proceedings of the 2nd KDD Workshop on Large-Scale Recommender Systems and the Netflix Prize Competition, NETFLIX’08 (Association for Computing Machinery, New York, 2008)

    MATH  Google Scholar 

  56. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A.N. Gomez, Ł. Kaiser, I. Polosukhin, Attention is all you need, in Advances in Neural Information Processing Systems, 30, ed. by I. Guyon, U.V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, R. Garnett. (Curran Associates, Red Hook, 2017), pp. 5998–6008

    Google Scholar 

  57. J. Wang, A.P. de Vries, M.J.T. Reinders, Unifying user-based and item-based collaborative filtering approaches by similarity fusion, in Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR’06 (Association for Computing Machinery, New York, 2006), pp. 501–508

    Google Scholar 

  58. Y. Zheng, B. Tang, W. Ding, H. Zhou, A neural autoregressive approach to collaborative filtering, in Proceedings of the 33rd International Conference on Machine Learning - Volume 48, ICML’16, JMLR.org (2016). pp. 764–773

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Google, Israel, Israel

    Yehuda Koren

  2. Google Research, Mountain View, CA, USA

    Steffen Rendle

  3. Berkeley Heights, NJ, USA

    Robert Bell

Authors
  1. Yehuda Koren
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Steffen Rendle
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Robert Bell
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yehuda Koren .

Editor information

Editors and Affiliations

  1. Faculty of Computer Science, Free University of Bozen-Bolzano, Bozen-Bolzano, Italy

    Francesco Ricci

  2. Software and Information Systems Engineering, Ben-Gurion University of the Negev, Beer-Sheva, Israel

    Lior Rokach

  3. Software and Information Systems Engineering, Ben-Gurion University of the Negev, Beer-Sheva, Israel

    Bracha Shapira

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Science+Business Media, LLC, part of Springer Nature

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Koren, Y., Rendle, S., Bell, R. (2022). Advances in Collaborative Filtering. In: Ricci, F., Rokach, L., Shapira, B. (eds) Recommender Systems Handbook. Springer, New York, NY. https://doi.org/10.1007/978-1-0716-2197-4_3

Download citation

  • DOI: https://doi.org/10.1007/978-1-0716-2197-4_3

  • Published:

  • Publisher Name: Springer, New York, NY

  • Print ISBN: 978-1-0716-2196-7

  • Online ISBN: 978-1-0716-2197-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics