Suhrid Balakrishnan
ABSTRACT
Typical recommender systems use the root mean squared error (RMSE) between the predicted and actual ratings as the evaluation metric. We argue that RMSE is not an optimal choice for this task, especially when we will only recommend a few (top) items to any user. Instead, we propose using a ranking metric, namely normalized discounted cumulative gain (NDCG), as a better evaluation metric for this task. Borrowing ideas from the learning to rank community for web search, we propose novel models which approximately optimize NDCG for the recommendation task. Our models are essentially variations on matrix factorization models where we also additionally learn the features associated with the users and the items for the ranking task. Experimental results on a number of standard collaborative filtering data sets validate our claims. The results also show the accuracy and efficiency of our models and the benefits of learning features for ranking.
- Y. Bengio, R. Ducharme, P. Vincent, and C. Janvin. A neural probabilistic language model. Journal of Machine Learning Research, 3:1137--1155, 2003. Google Scholar
Digital Library
- L. Breiman. Random forests. In Machine Learning, volume 45(1), 2001. Google ScholarDigital Library
- C. Burges. From ranknet to lambdarank to lambdamart: An overview. In Microsoft Research Technical Report MSR-TR-2010-82, 2010.Google Scholar
- R. Collobert and J. Weston. A unified architecture for natural language processing: deep neural networks with multitask learning. In ICML '08: Proceedings of the 25th international conference on Machine learning, pages 160--167, New York, NY, USA, 2008. ACM. Google ScholarDigital Library
- D. Cossock and T. Zhang. Statistical analysis of bayes optimal subset ranking. IEEE Transactions on Information Theory, 54(11):5140--5154, 2008. Google ScholarDigital Library
- P. Cremonesi, Y. Koren, and R. Turrin. Performance of recommender algorithms on top-n recommendation tasks. In Proceedings of the fourth ACM conference on Recommender systems, RecSys '10, pages 39--46, New York, NY, USA, 2010. ACM. Google ScholarDigital Library
- P. Donmez, K. M. Svore, and C. J. Burges. On the local optimality of lambdarank. In Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval, SIGIR '09, pages 460--467, New York, NY, USA, 2009. ACM. Google ScholarDigital Library
- Y. Freund, R. Iyer, R. E. Schapire, and Y. Singer. An efficient boosting algorithm for combining preferences. J. Mach. Learn. Res., 4:933--969, 2003. Google ScholarDigital Library
- J. H. Friedman. Greedy function approximation: A gradient boosting machine. Annals of Statistics, 29:1189--1232, 2000.Google ScholarCross Ref
- S. Hacker and L. von Ahn. Matchin: eliciting user preferences with an online game. In CHI '09: Proceedings of the 27th international conference on Human factors in computing systems, pages 1207--1216, New York, NY, USA, 2009. ACM. Google ScholarDigital Library
- R. Herbrich, T. Graepel, and K. Obermayer. Large margin rank boundaries for ordinal regression. In P. J. Bartlett, B. Schölkopf, D. Schuurmans, and A. J. Smola, editors, Advances in Large Margin Classifiers, pages 115--132. MIT Press, 2000.Google Scholar
- K. Järvelin and J. Kekäläinen. Cumulated gain-based evaluation of ir techniques. ACM Trans. Inf. Syst., 20:422--446, October 2002. Google ScholarDigital Library
- T. Joachims. Optimizing search engines using clickthrough data. In Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, KDD '02, pages 133--142, New York, NY, USA, 2002. ACM. Google ScholarDigital Library
- Y. Koren, R. Bell, and C. Volinsky. Matrix Factorization Techniques for Recommender Systems. Computer, 42(8):30--37, 2009. Google ScholarDigital Library
- P. Li, C. J. C. Burges, and Q. Wu. Mcrank: Learning to rank using multiple classification and gradient boosting, 2007.Google Scholar
- B. M. Marlin and R. S. Zemel. Collaborative prediction and ranking with non-random missing data. Proceedings of the third ACM conference on Recommender systems (RecSys '09), 2009. Google ScholarDigital Library
- P. Melville and V. Sindhwani. Recommender Systems. 2010.Google Scholar
- A. Mohan, Z. Chen, and K. Q. Weinberger. Web-search ranking with initialized gradient boosted regression trees. Journal of Machine Learning Research, Workshop and Conference Proceedings, 14:77--89, 2011.Google Scholar
- M. Richardson, E. Dominowska, and R. Ragno. Predicting clicks: estimating the click-through rate for new ads. In Proceedings of the 16th international conference on World Wide Web, WWW '07, pages 521--530, New York, NY, USA, 2007. ACM. Google ScholarDigital Library
- R. Salakhutdinov and A. Mnih. Probabilistic matrix factorization. In Neural Information Processing Systems (NIPS), 2007.Google Scholar
- H. Steck and R. S. Zemel. A generalized probabilistic framework and its variants for training top-k recommender systems. Workshop on the Practical Use of Recommender Systems, Algorithms and Technologies (PRSAT 2010) in conjunction with RecSys 2010, 2010.Google Scholar
- M. Weimer, A. Karatzoglou, Q. V. Le, and A. Smola. Cofi-rank: Maximum margin matrix factorization for collaborative ranking. In Neural Information Processing Systems (NIPS), 2007.Google Scholar
- J. Xu and H. Li. Adarank: a boosting algorithm for information retrieval. In Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, SIGIR '07, pages 391--398, New York, NY, USA, 2007. ACM. Google ScholarDigital Library
- J. Xu, T.-Y. Liu, M. Lu, H. Li, and W.-Y. Ma. Directly optimizing evaluation measures in learning to rank. In Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval, SIGIR '08, pages 107--114, New York, NY, USA, 2008. ACM. Google ScholarDigital Library
Index Terms
- Collaborative ranking
Recommendations
Predicting User Engagement in Twitter with Collaborative Ranking
RecSysChallenge '14: Proceedings of the 2014 Recommender Systems ChallengeCollaborative Filtering (CF) is a core component of popular web-based services such as Amazon, YouTube, Netflix, and Twitter. Most applications use CF to recommend a small set of items to the user. For instance, YouTube presents to a user a list of top-...
Item-based collaborative ranking
PreNIt is the first item-based approach for neighborhood collaborative ranking.PreNit models the users choice contexts as bipartite networks with labeled edges.PreNit uses a novel ranking algorithm in bipartite networks, called MemoRank.MemoRank models ...
Graph-based collaborative ranking
GRank is a novel framework, designed for recommendation based on rank data.GRank handles the sparsity problem of neighbor-based collaborative ranking.GRank uses the novel TPG graph structure to model users' choice context.GRank directly ranks itemsfor a ...
Comments