Abstract
This paper analyzes the main barriers that user model developers have to face when evaluating a statistical user model. Main techniques used to evaluate statistical user models, mostly borrowed from the areas of Machine Learning and Information Retrieval, are examined. Then an evaluation methodology for statistical user models is proposed together with a set of metrics to specifically evaluate statistical user models. Finally, a benchmark for statistical user models is proposed, thus making possible to compare and replicate the evaluations. Thus, main contribution of this paper is to enable that several user model evaluations were comparable.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Amento, B., Terveen, L., Hix, D., Ju, P.: An empirical evaluation of user interfaces for topic management of web sites. In: Proceedings of the Conference on Human Factors in Computing Systems (CHI 1999), pp. 552–559. ACM, New York (1999)
Baeza-Yates, R., Ribeiro-Neto, B.: Modern Information Retrieval. Addison-Wesley (1999)
Balabanovic, M., Shoham, Y.: Fab: Content-based, collaborative recommendation. Commun. ACM 40, 66–72 (1997)
Basu, C., Hirsh, H., Cohen, W.: Recommendation as Classification: Using Social and Content-Based Information in Recommendation. In: Proceedings of the 15th National Conference on Artificial Intelligence, Madison, WI, pp. 714–720 (1998)
Breese, J., Heckerman, D., Kadie, C.: Empirical analysis of predictive algorithms for collaborative clustering. In: Proc. of the Fourteenth Annual Conference on Uncertainty in Artificial Intelligence, San Francisco, vol. 43 (1998)
Bollen, D., Knijnenburg, B.P., Willemsen, M.C., Graus, M.: Understanding choice overload in recommender systems. In: Proceedings of the Fourth ACM Conference on Recommender Systems, pp. 63–70. ACM, New York (2010), http://doi.acm.org/10.1145/1864708.1864724 , doi:10.1145/1864708.1864724
Cosley, D., Lam, S.K., Albert, I., Konstan, J.A., Riedl, J.: Is seeing believing? How recommender interfaces affect users’ opinions. CHI Lett. 5 (2003)
Heckerman, D., Chickering, D.M., Meek, C., Rounthwaite, R., Kadie, C.: Dependency networks for inference, collaborative filtering, and data visualization. J. Mach. Learn. Res. 1, 49–75 (2000)
Herlocker, J.L., Konstan, J.A., Riedl, J.: An empirical analysis of design choices in neighborhood-based collaborative filtering algorithms. Inf. Retr. 5, 287–310 (2002)
Herlocker, J.L., Konstan, J.A., Terveen, L.G., Riedl, J.T.: Evaluating collaborative filtering recommender systems. ACM Transactions on Information Systems 22(1), 5–53 (2004)
Hernández, F.O., Gaudioso, E.: Evaluation of recommender systems: A new approach. Expert Syst. 35(3), 790–804 (2008), http://dx.doi.org/10.1016/j.eswa.2007.07.047 , doi:10.1016/j.eswa.2007.07.047
Larson, H.J.: Introduction to Probability Theory and Statistical Inference. Wiley International Edition (1969)
Miller, B., Albert, I., Lam, S.K., Konstan, J., Riedl, J.: MovieLens Unplugged: Experiences with a Recommender System on Four Mobile Devices. In: Proceedings of the 17th Annual Human-Computer Interaction Conference (HCI). British HCI Group (September 2003)
Pu, P., Chen, L., Hu, R.: A user-centric evaluation framework for recommender systems. In: Proceedings of the Fifth ACM Conference on Recommender Systems, pp. 157–164. ACM, New York (2011), http://doi.acm.org/10.1145/2043932.2043962 , doi:10.1145/2043932.2043962
Wade, W.: A grocery cart that holds bread, butter and preferences. NY Times (January 16, 2003)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer International Publishing Switzerland
About this paper
Cite this paper
Calle, J., Castaño, L., Castro, E., Cuadra, D. (2013). Evaluation Framework for Statistical User Models. In: Omatu, S., Neves, J., Rodriguez, J., Paz Santana, J., Gonzalez, S. (eds) Distributed Computing and Artificial Intelligence. Advances in Intelligent Systems and Computing, vol 217. Springer, Cham. https://doi.org/10.1007/978-3-319-00551-5_54
Download citation
DOI: https://doi.org/10.1007/978-3-319-00551-5_54
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-00550-8
Online ISBN: 978-3-319-00551-5
eBook Packages: EngineeringEngineering (R0)