Michael D. Ekstrand
Daniel Kluver
F. Maxwell Harper
Joseph A. Konstan
ABSTRACT
Recommender systems are not one-size-fits-all; different algorithms and data sources have different strengths, making them a better or worse fit for different users and use cases. As one way of taking advantage of the relative merits of different algorithms, we gave users the ability to change the algorithm providing their movie recommendations and studied how they make use of this power. We conducted our study with the launch of a new version of the MovieLens movie recommender that supports multiple recommender algorithms and allows users to choose the algorithm they want to provide their recommendations. We examine log data from user interactions with this new feature to under-stand whether and how users switch among recommender algorithms, and select a final algorithm to use. We also look at the properties of the algorithms as they were experienced by users and examine their relationships to user behavior. We found that a substantial portion of our user base (25%) used the recommender-switching feature. The majority of users who used the control only switched algorithms a few times, trying a few out and settling down on an algorithm that they would leave alone. The largest number of users prefer a matrix factorization algorithm, followed closely by item-item collaborative filtering; users selected both of these algorithms much more often than they chose a non-personalized mean recommender. The algorithms did produce measurably different recommender lists for the users in the study, but these differences were not directly predictive of user choice.
Supplemental Material
Available for Download
This archive contains the configurations and additional components for the LensKit configurations used in the paper. - These files are licensed under the GNU Lesser General Public License, version 2.1 or later, the same terms governing LensKit itself.
- Balabanović, M. and Shoham, Y. 1997. Fab: content-based, collaborative recommendation. Commun. ACM. 40, 3 (1997), 66--72. Google Scholar
Digital Library
- Burke, R. 2002. Hybrid Recommender Systems: Survey and Experiments. User Modeling and User-Adapted Interaction. 12, 4 (Nov. 2002), 331--370. Google ScholarDigital Library
- Chang, S., Harper, F.M. and Terveen, L. 2015. Using Groups of Items for Preference Elicitation in Recommender Systems. In Proc. ACM CSCW '15 (2015), 1258--1269. Google ScholarDigital Library
- Cook, R. and Kay, J. 1994. The justified user model: a viewable, explained user model. Proceedings of the Fourth International Conference on User Modelling (1994).Google Scholar
- Cremonesi, P., Garzottto, F. and Turrin, R. 2012. User Effort vs. Accuracy in Rating-based Elicitation. In Proc. RecSys 2012 (2012), 27--34. Google ScholarDigital Library
- Dooms, S. 2014. Dynamic Generation of Personalized Hybrid Recommender Systems. Ph.D thesis, Universiteit Gent.Google Scholar
- Ekstrand, M.D., Harper, F.M., Willemsen, M.C. and Konstan, J.A. 2014. User Perception of Differences in Recommender Algorithms. In Proc. RecSys '14 (2014). Google ScholarDigital Library
- Ekstrand, M., Ludwig, M., Konstan, J.A. and Riedl, J. 2011. Rethinking the Recommender Research Ecosystem: Reproducibility, Openness, and LensKit. In Proc. RecSys '11 (2011), 133--140. Google ScholarDigital Library
- Ekstrand, M. and Riedl, J. 2012. When recommenders fail: predicting recommender failure for algorithm selection and combination. In Proc. RecSys '12 (2012), 233--236. Google ScholarDigital Library
- Goldberg, D., Nichols, D., Oki, B.M. and Terry, D. 1992. Using collaborative filtering to weave an information tapes-try. Commun. ACM. 35, 12 (1992), 61--70. Google ScholarDigital Library
- Halfaker, A., Keyes, O., Kluver, D., Thebault-Spieker, J., Nguyen, T., Shores, K., Uduwage, A. and Warncke-Wang, M. 2014. User Session Identification Based on Strong Regularities in Inter-activity Time. arXiv:1411.2878 {cs}. (Nov. 2014).Google Scholar
- Kay, J. 2006. Scrutable Adaptation: Because We Can and Must. Adaptive Hypermedia and Adaptive Web-Based Systems. V.P. Wade, H. Ashman, and B. Smyth, eds. Springer Berlin Heidelberg. 11--19. Google ScholarDigital Library
- Kelly, D. and Belkin, N.J. 2001. Reading Time, Scrolling and Interaction: Exploring Implicit Sources of User Preferences for Relevance Feedback. In Proc. SIGIR '01 (2001), 408--409. Google ScholarDigital Library
- Knijnenburg, B., Willemsen, M., Gantner, Z., Soncu, H. and Newell, C. 2012. Explaining the user experience of recommender systems. User Modeling and User-Adapted Interaction. 22, 4--5 (Oct. 2012), 441--504. Google ScholarDigital Library
- Kohavi, R., Longbotham, R., Sommerfield, D. and Henne, R.M. 2008. Controlled experiments on the web: survey and practical guide. Data Mining and Knowledge Discovery. 18, 1 (Jul. 2008), 140--181. Google ScholarDigital Library
- Kramer, A.D.I., Guillory, J.E. and Hancock, J.T. 2014. Experimental evidence of massive-scale emotional contagion through social networks. Proceedings of the National Academy of Sciences. 111, 24 (Jun. 2014), 8788--8790.Google ScholarCross Ref
- Levi, A., Mokryn, O., Diot, C. and Taft, N. 2012. Finding a Needle in a Haystack of Reviews: Cold Start Context-based Hotel Recommender System. In Proc. RecSys '12 (2012), 115--122. Google ScholarDigital Library
- McNee, S., Kapoor, N. and Konstan, J.A. 2006. Don't Look Stupid: Avoiding Pitfalls When Recommending Research Papers. In Proc. CSCW '06 (2006), 171. Google ScholarDigital Library
- Netflix Update: Try This at Home: 2006. http://sifter.org/~simon/journal/20061211.html. Accessed: 2010-04-08.Google Scholar
- Nguyen, T.T., Hui, P.-M., Harper, F.M., Terveen, L. and Konstan, J.A. 2014. Exploring the Filter Bubble: The Effect of Using Recommender Systems on Content Diversity. In Proc. WWW '14 (2014), 677--686. Google ScholarDigital Library
- Pariser, E. 2011. The Filter Bubble: How the New Personalized Web Is Changing What We Read and How We Think. Penguin. Google ScholarDigital Library
- Paterek, A. 2007. Improving regularized singular value decomposition for collaborative filtering. In Proc. KDD Cup and Workshop 2007 (Aug. 2007).Google Scholar
- Resnick, P., Iacovou, N., Suchak, M., Bergstrom, P. and Riedl, J. 1994. GroupLens: an open architecture for collaborative filtering of netnews. In Proc. ACM CSCW'94 (1994), 175--186. Google ScholarDigital Library
- Rich, E. 1979. User modeling via stereotypes. Cognitive Science. 3, 4 (Oct. 1979), 329--354.Google ScholarCross Ref
- Said, A., Fields, B., Jain, B.J. and Albayrak, S. 2013. User-centric Evaluation of a K-furthest Neighbor Collaborative Filtering Recommender Algorithm. In Proc. ACM CSCW '13 (2013), 1399--1408. Google ScholarDigital Library
- Sarwar, B., Karypis, G., Konstan, J. and Reidl, J. 2001. Item-based collaborative filtering recommendation algorithms. In Proc. WWW '01 (2001), 285--295. Google ScholarDigital Library
- Sill, J., Takacs, G., Mackey, L. and Lin, D. 2009. Feature-Weighted Linear Stacking. arXiv:0911.0460. (Nov. 2009).Google Scholar
- Tintarev, N. 2007. Explanations of recommendations. In Proc. RecSys '07 (2007), 203--206. Google ScholarDigital Library
- Vig, J., Sen, S. and Riedl, J. 2012. The Tag Genome: En-coding Community Knowledge to Support Novel Interaction. ACM Trans. Interact. Intell. Syst. 2, 3 (Sep. 2012), 13:1--13:44. Google ScholarDigital Library
- Ziegler, C.-N., McNee, S., Konstan, J.A. and Lausen, G. 2005. Improving Recommendation Lists through Topic Diversification. In Proc. WWW '05 (2005), 22--32. Google ScholarDigital Library
Index Terms
- Letting Users Choose Recommender Algorithms: An Experimental Study
Recommendations
A Scalable, Accurate Hybrid Recommender System
WKDD '10: Proceedings of the 2010 Third International Conference on Knowledge Discovery and Data MiningRecommender systems apply machine learning techniques for filtering unseen information and can predict whether a user would like a given resource. There are three main types of recommender systems: collaborative filtering, content-based filtering, and ...
Acquiring User Information Needs for Recommender Systems
WI-IAT '13: Proceedings of the 2013 IEEE/WIC/ACM International Joint Conferences on Web Intelligence (WI) and Intelligent Agent Technologies (IAT) - Volume 03Most recommender systems attempt to use collaborative filtering, content-based filtering or hybrid approach to recommend items to new users. Collaborative filtering recommends items to new users based on their similar neighbours, and content-based ...
Leveraging clustering approaches to solve the gray-sheep users problem in recommender systems
We provide detailed analysis of gray-sheep users problem in recommender systems.We show how conventional collaborative filtering fail for gray-sheep users problem.We use K-means clustering to separate these users from rest of the users.We propose ...
Comments