Authors:
Andre Paulino de Lima
and
Sarajane Marques Peres
Affiliation:
School of Arts, Sciences and Humanities, University of São Paulo and Brazil
Keyword(s):
Recommender Systems, Surprise Metric, Unexpectedness, Serendipity, Item Representation, Item Comparison, Off-line Evaluation.
Related
Ontology
Subjects/Areas/Topics:
Artificial Intelligence
;
Enterprise Information Systems
;
Human Factors
;
Human-Computer Interaction
;
Information Systems Analysis and Specification
;
Knowledge Discovery and Information Retrieval
;
Knowledge-Based Systems
;
Physiological Computing Systems
;
Software Metrics and Measurement
;
Symbolic Systems
;
User Profiling and Recommender Systems
Abstract:
Surprise is a property of recommender systems that has been receiving increasing attention owing to its links to serendipity. Most of the metrics for surprise poorly agree with definitions employed in research areas that conceptualise surprise as a human factor, and because of this, their use in the task of evaluating recommendations may not produce the desired effect. We argue that metrics with the characteristics that are presumed by models of surprise from the Cognitive Science may be more successful in that task. Moreover, we show that a metric for surprise is sensitive to the choices of how items are represented and compared by the recommender. In this paper, we review metrics for surprise in recommender systems, and analyse to which extent they align to two competing cognitive models of surprise. For that metric with the highest agreement, we conducted an off-line experiment to estimate the effect exerted on surprise by choices of item representation and comparison. We explore
56 recommenders that vary in recommendation algorithms, and item representation and comparison. The results show a large interaction between item representation and item comparison, which suggests that new distance functions can be explored to promote serendipity in recommendations.
(More)