Balancing user satisfaction and cognitive load in coverage-optimised retrieval

Abstract

Coverage-optimised retrieval (CORE) is a new case-based reasoning (CBR) approach to product recommendation which ensures that for any case that is acceptable to the user, one of the recommended cases is at least as good in an objective sense and so also likely to be acceptable. Similarity to the users query, the standard criterion on which recommendations are based in CBR, is only one of several preference criteria according to which a given case may be considered at least as good as another in CORE. We present a detailed analysis of retrieval in CORE and the trade-off between user satisfaction and cognitive load in the approach.

Introduction

An advantage of case-based reasoning (CBR) as an approach to product recommendation is that if none of the available products exactly matches the user's query, she can be shown the cases that are most similar to her query [1]. A basic premise in the approach is that one of the recommended cases may be acceptable to the user even though it fails to satisfy one or more of her requirements. However, several authors have questioned the assumption that the most similar product is the one that is most acceptable to the user [2], [3], [4]. For example, a case that satisfies only two of the user's requirements may be more similar than one that satisfies three of her requirements; or the most similar case may fail to satisfy a requirement that the user considers to be essential. The k-NN strategy of retrieving the k most similar cases, rather than a single case, only partially compensates for this limitation, as the number of cases that can be presented to the user is necessarily restricted in practice [4], [5].

Thus, the existence of a case in the case library that would be acceptable to the user does not guarantee that it will be retrieved. While retrieving every case in the case library is seldom feasible in practice, we argue that the next best thing is to ensure that for any case that is acceptable to the user, the retrieval set contains a case that is as at least as good in an objective sense and so also likely to be acceptable. This is the aim of a new approach to retrieval in recommender systems which we refer to as coverage-optimised retrieval (CORE) [6].

The CORE retrieval set provides full coverage of the case library in the sense that for any case that is acceptable to the user, one of the recommended cases is at least as good according to a given preference criterion. Similarity to the target query is the weakest of several preference criteria according to which a given case may be considered at least as good as another in CORE. A basic premise in the approach is that the stronger the preference criterion on which a recommendation is based, the more likely it is to be acceptable to the user. However, the size of the retrieval sets that a retrieval algorithm produces is an important factor in its ability to address the trade-off between user satisfaction and cognitive load to which Branting [5] refers. The strength of the preference criterion on which retrieval is based in CORE must, therefore, be balanced against the size of the retrieval set required to provide full coverage of the case library. Here we present a detailed analysis of retrieval in CORE that aims to increase our understanding of the trade-off between user satisfaction and cognitive load in the approach.

In Section 2, we examine four different preference criteria on which retrieval may be based in CORE. We also show that k-NN provides only limited coverage of the case library with respect to preference criteria that may be more predictive of user satisfaction than similarity to the target query. In Section 3, we describe how the retrieval set is constructed in CORE and examine factors that affect the trade-off between user satisfaction and cognitive load. Related work is discussed in Section 4 and our conclusions are presented in Section 5.