Abstract
Providing useful recommendations is an important challenge for user-centric media systems. Whereas current recommender systems research mainly focuses on predictive accuracy, we contend that a truly user-centric approach to media recommendations requires the inclusion of user experience measurement. For a good experience, predictive accuracy is not enough. What users like and dislike about our systems is also determined by usage context and individual user characteristics. We therefore propose a generic framework for evaluating the user experience using both subjective and objective measures of user experience. We envision the framework, which will be tested and validated in the large-scale field trials of the FP7 MyMedia project, to be a fundamental step beyond accuracy of algorithms, towards usability of recommender systems.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Davis, F.D., Bagozzi, R.P., Warshaw, P.R.: User Acceptance of Computer Technology: A Comparison of two theoretical models. Management Science 35(8), 982–1003 (1989)
Hassenzahl, M.: User Experience (UX): Towards an experiential perspective on product quality. In: IHM 2008, Metz, France, September 2-5, pp. 11–15 (2008)
Herlocker, J.L., Konstan, J.A., Terveen, L.G., Riedl, J.T.: Evaluating collaborative filtering recommender systems. ACM Transactions on Information Systems 22(1), 5–53 (2004)
Knijnenburg, B.P., Willemsen, M.C.: Understanding the Effect of Adaptive Preference Elicitation Methods on User Satisfaction of a Recommender System. In: ACM Conference on Recommender Systems 2009, New York (2009)
McNee, S.M., Riedl, J., Konstan, J.A.: Making recommendations better: An analytic model for human-recommender interaction. In: CHI 2006, Montreal Canada (2006)
McNee, S.M., Riedl, J., Konstan, J.A.: Accurate is not Always Good: How Accuracy Metrics have hurt Recommender Systems. In: CHI 2006, Montreal, Canada (2006)
Meesters, L., Marrow, P., Knijnenburg, B., Bouwhuis, D., Glancy, M.: Deliverable 1.5 End-user recommendation evaluation metrics. FP7 MyMedia project, www.mymediaproject.org
Spiekermann, S.: Online Information Search with Electronic Agents: Drivers, Impediments, and Privacy Issues. PhD Thesis, Humboldt University, Berlin (2001)
Xiao, B., Benbasat, I.: E-Commerce Product Recommendation Agents: Use, Characteristics, and Impact. MIS Quarterly 31(1), 137–209 (2007)
Zins, A., Bauernfeind, U.: Explaining Online Purchase Planning Experiences with Recommender Websites. In: Information and Communication Technologies in Tourism 2005, Innsbruck, Austria, pp. 137–148 (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 ICST Institute for Computer Science, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Knijnenburg, B., Meesters, L., Marrow, P., Bouwhuis, D. (2010). User-Centric Evaluation Framework for Multimedia Recommender Systems. In: Daras, P., Ibarra, O.M. (eds) User Centric Media. UCMEDIA 2009. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 40. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-12630-7_47
Download citation
DOI: https://doi.org/10.1007/978-3-642-12630-7_47
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-12629-1
Online ISBN: 978-3-642-12630-7
eBook Packages: Computer ScienceComputer Science (R0)