Skip to main content
SpringerLink
Log in

An optimization model for collaborative recommendation using a covariance-based regularizer

  • Published:
Data Mining and Knowledge Discovery Aims and scope Submit manuscript

Abstract

This paper suggests a convex regularized optimization model to produce recommendations, which is adaptable, fast, and scalable—while remaining very competitive to state-of-the-art methods in terms of accuracy. We introduce a regularizer based on the covariance matrix such that the model minimizes two measures ensuring that the recommendations provided to a user are guided by both the preferences of the other users in the system and the known preferences of the user being processed. It is adaptable since (1) it can be viewed from both user and item perspectives (allowing to choose, depending on the task, the formulation with fewer decision variables) and (2) multiple constraints depending on the context (and not only based on the accuracy, but also on the utility of personalized recommendations) can easily be added, as shown in this paper through two examples. Since our regularizer is based on the covariance matrix, this paper also describes how to improve computational and space complexities by using matrix factorization techniques in the optimization model, leading to a fast and scalable model. To illustrate all these concepts, experiments were conducted on four real datasets of different sizes (i.e., FilmTrust, Ciao, MovieLens, and Netflix) and comparisons with state-of-the-art methods are provided, showing that our context-sensitive approach is very competitive in terms of accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. https://sites.google.com/site/nicolasgillis/code.

  2. http://grouplens.org/datasets/movielens/.

References

  • Adomavicius G, Tuzhilin A (2005) Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Trans Knowl Data Eng 17(6):734–749

    Article  Google Scholar 

  • Agarwal D, Chen BC, Elango P, Wang X (2011) Click shaping to optimize multiple objectives. In: Proceedings of the 17th ACM SIGKDD international conference on knowledge discovery and data mining (KDD’11), pp 132–140

  • Armstrong J (2001) Principles of forecasting, a handbook for researchers and practitioners. Kluwer Academic, Norwell

    Book  Google Scholar 

  • Bennett J, Lanning S (2007) The Netflix prize. In: KDD Cup and Workshop in conjunction with KDD

  • Boyd S, Vandenberghe L (2004) Convex optimization. Cambridge University Press, Cambridge

    Book  MATH  Google Scholar 

  • Cremonesi P, Koren Y, Turrin R (2010) Performance of recommender algorithms on top-n recommendation tasks. In: Proceedings of the fourth ACM conference on recommender systems (RecSys ’10), pp 39–46

  • Devooght R, Kourtellis N, Mantrach A (2015) Dynamic matrix factorization with priors on unknown values. In: Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pp 189–198

  • Deza M, Deza E (2014) Encyclopedia of distances, 3rd edn. Springer, Heidelberg

    MATH  Google Scholar 

  • Feldman S (2012) Multi-task averaging: theory and practice. PhD thesis, University of Washington

  • Fouss F, Francoisse K, Yen L, Pirotte A, Saerens M (2012) An experimental investigation of kernels on graphs for collaborative recommendation and semi-supervised classification. Neural Netw 31:53–72

    Article  MATH  Google Scholar 

  • Fouss F, Saerens M, Shimbo M (2016) Algorithms and models for network data and link analysis. Cambridge University Press, Cambridge

    Book  Google Scholar 

  • Gantner Z, Rendle S, Freudenthaler C, Schmidt-Thieme L (2011) MyMediaLite: a free recommender system library. In: Proceedings of the 5th ACM conference on recommender systems (RecSys 2011)

  • Gill PE, Murray W, Wright MH (1981) Practical optimization. Academic Press, London

    MATH  Google Scholar 

  • Gillis N (2011) Nonnegative matrix factorization: complexity, algorithms and applications. PhD thesis, Université catholique de Louvain

  • Gillis N (2014) The why and how of nonnegative matrix factorization. In: Suykens JAK, Signoretto M, Argyriou A (eds) Regularization, optimization, kernels, and support vector machines. Chapman & Hall/CRC, Machine Learning and Pattern Recognition Series, pp 257–291

    Google Scholar 

  • Gillis N, Glineur F (2012) Accelerated multiplicative updates and hierarchical Als algorithms for nonnegative matrix factorization. Neural Comput 24(4):1085–1105

    Article  MathSciNet  Google Scholar 

  • Gu Q, Zhou J, Ding C (2010) Collaborative filtering: weighted nonnegative matrix factorization incorporating user and item graphs. In: Proceedings of the 2010 SIAM international conference on data mining, pp 199–210

  • Guo G, Zhang J, Yorke-Smith N (2013) A novel bayesian similarity measure for recommender systems. In: Proceedings of the 23rd international joint conference on artificial intelligence (IJCAI), pp 2619–2625

  • Herlocker J, Konstan J, Riedl J (2002) An empirical analysis of design choices in neighborhood-based collaborative filtering algorithms. Inf Retr 5:287–310

    Article  Google Scholar 

  • Hill W, Stead L, Rosenstein M, Furnas G (1995) Recommending and evaluating choices in a virtual community of use. In: Proceedings of ACM CHI’95 conference on human factors in computing systems, pp 194–201

  • Hsieh CJ, Olsen P (2014) Nuclear norm minimization via active subspace selection. In: Proceedings of the 31st international conference on machine learning (ICML-14), pp 575–583

  • Hu Y, Koren Y, Volinsky C (2008) Collaborative filtering for implicit feedback datasets. In: Proceedings of the Eighth IEEE international conference on data mining (ICDM), pp 263–272

  • Jambor T, Wang J (2010) Optimizing multiple objectives in collaborative filtering. In: Proceedings of the fourth ACM conference on recommender systems (RecSys ’10), pp 55–62

  • Järvelin K, Kekäläinen J (2002) Cumulated gain-based evaluation of ir techniques. ACM Trans Inf Syst 20(4):422–446

    Article  Google Scholar 

  • Johnson R, Wichern D (2002) Applied multivariate statistical analysis, 5th edn. Prentice Hall, Englewood Cliffs

    MATH  Google Scholar 

  • Kannan R, Ishteva M, Park H (2012) Bounded matrix low rank approximation. In: Proceedings of the 12th IEEE international conference on data mining (ICDM), pp 319–328

  • Kim D, Yum BJ (2005) Collaborative filtering based on iterative principal component analysis. Expert Syst Appl 28(4):823–830

    Article  Google Scholar 

  • Koren Y (2008) Factorization meets the neighborhood: a multifaceted collaborative filtering model. In: Proceedings of the 14th ACM SIGKDD international conference on knowledge discovery and data mining (KDD’08), pp 426–434

  • Koren Y (2009) Collaborative filtering with temporal dynamics. In: Proceedings of the 15th ACM SIGKDD international conference on knowledge discovery and data mining (KDD ’09), pp 447–456

  • Koren Y, Bell R, Volinsky C (2009) Matrix factorization techniques for recommender systems. Computer 42(8):30–37

    Article  Google Scholar 

  • Lilien G, Smith B, Moorthy K (1992) Marketing models. Prentice Hall, Englewood Cliffs

    Google Scholar 

  • Lü L, Medo M, Yeung CH, Zhang YC, Zhang ZK, Zhou T (2012) Recommender systems. Phys Rep 519:1–49

    Article  Google Scholar 

  • Luo X, Zhou M, Xia Y, Zhu Q (2014) An efficient non-negative matrix-factorization-based approach to collaborative filtering for recommender systems. IEEE Trans Ind Inf 10(2):1273–1284

    Article  Google Scholar 

  • Murthi B, Sarkar S (2003) The role of the management sciences in research on personalization. Manag Sci 49(10):1344–1362

    Article  Google Scholar 

  • Ning X, Karypis G (2011) SLIM: sparse linear methods for top-n recommender systems. In: Proceedings of the IEEE international conference on data mining (ICDM), pp 497–506

  • Ning X, Desrosiers C, Karypis G (2015) A comprehensive survey of neighborhood-based recommendation methods. Springer, Heidelberg, pp 37–76

    Google Scholar 

  • Paterek A (2007) Improving regularized singular value decomposition for collaborative filtering. In: Proceedings of KDD cup and workshop, pp 39–42

  • Rao N, Yu H-F, Ravikumar P, Dhillon IS (2015) Collaborative filtering with graph information: consistency and scalable methods. In: Cortes C, Lee DD, Sugiyama M, Garnett R (eds) Proceedings of the 28th international conference on neural information processing systems, vol 2, MIT Press, Cambridge, MA, pp 2107–2115

  • Rendle S, Freudenthaler C, Gantner Z, Schmidt-Thieme L (2009) BPR: Bayesian personalized ranking from implicit feedback. In: Proceedings of the twenty-fifth conference on uncertainty in artificial intelligence (AUAI), pp 452–461

  • Resnick P, Neophytos I, Mitesh S, Bergstrom P, Riedl J (1994) GroupLens: an open architecture for collaborative filtering of netnews. In: Proceedings of the conference on computer supported cooperative work, pp 175–186

  • Rich E (1979) User modeling via stereotypes. Cogn Sci 3(4):329–354

    Article  Google Scholar 

  • Rodriguez M, Posse C, Zhang E (2012) Multiple objective optimization in recommender systems. In: Proceedings of the Sixth ACM conference on recommender systems (RecSys ’12), pp 11–18

  • Salakhutdinov R, Mnih A (2008) Bayesian probabilistic matrix factorization using Markov chain Monte Carlo. In: Proceedings of the 25th international conference on machine learning (ICML ’08), pp 880–887

  • Salton G (1989) Automatic text processing. Addison-Wesley, Reading

    Google Scholar 

  • Shan H, Banerjee A (2010) Generalized probabilistic matrix factorizations for collaborative filtering. In: Proceedings of the 10th IEEE international conference on data mining (ICDM), pp 1025–1030

  • Shani G, Gunawardana A (2011) Evaluating recommendation systems. In: Ricci F, Rokach L, Shapira B, Kantor P (eds) Recommender systems handbook. Springer, Boston, MA, pp 257–297

    Chapter  Google Scholar 

  • Shardanand U, Maes P (1995) Social information filtering: algorithms for automating ’word of mouth’. In: Proceedings of the conference on human factors in computing systems, pp 210–217

  • Smola AJ, Kondor R (2003) Kernels and regularization on graphs. Learning theory and kernel machines, vol 2777. Lecture notes in computer science. Springer, Berlin, pp 144–158

    MATH  Google Scholar 

  • Tang J, Gao H, Liu H (2012) mTrust: discerning multi-faceted trust in a connected world. In: Proceedings of the fifth ACM international conference on web search and data mining, pp 93–102

  • Yu K, Zhu S, Lafferty J, Gong Y (2009) Fast nonparametric matrix factorization for large-scale collaborative filtering. In: Proceedings of the 32nd international ACM SIGIR conference on research and development in information retrieval (SIGIR ’09), pp 211–218

  • Zhang S, Wang W, Ford J, Makedon F (2006) Learning from incomplete ratings using non-negative matrix factorization. In: Proceedings of the 2006 SIAM international conference on data mining, pp 549–553

  • Zhang Y, Chen W, Yin Z (2013) Collaborative filtering with social regularization for TV program recommendation. Knowl Based Syst 54:310–317

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank Marco Saerens, Nicolas Gillis, and Arnaud Vandaele for insightful comments on this work.

Author information

Authors and Affiliations

  1. Department of Engineering Innovation Management, Faculty of Engineering, University of Mons, Mons, Belgium

    Fabian Lecron

  2. Louvain School of Management (LSM), Louvain Research Institute in Management and Organizations (LouRIM), Université catholique de Louvain (UCL), Mons, Belgium

    François Fouss

Authors
  1. Fabian Lecron
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. François Fouss
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fabian Lecron.

Additional information

Responsible editor: Chih-Jen Lin.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lecron, F., Fouss, F. An optimization model for collaborative recommendation using a covariance-based regularizer. Data Min Knowl Disc 32, 651–674 (2018). https://doi.org/10.1007/s10618-018-0552-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10618-018-0552-3

Keywords

Search

Navigation