Using Raw Accuracy to Estimate Classifier Fitness in XCS

Abstract

In XCS classifier fitness is based on the relative accuracy of the classifier prediction [3]. A classifier is more fit if its prediction of the expected payoff is more accurate than the prediction given by the other classifiers that are applied in the same situations. The use of relative accuracy has two major implications. First, because the evaluation of fitness is based on the relevance that classifiers have in some situations, classifiers that are the only ones applying in a certain situation have a high fitness, even if they are inaccurate. As a consequence, inaccurate classifiers might be able to reproduce so to cause reduced performance; as already noted by Wilson (personal communication reported in [1]). In addition, because the computation of classifier fitness is based both (i) on the classifier accuracy and (ii) on the classifier relevance in situations in which it applies, in XCS, classifier fitness does not provide information about the problem solution, but rather an indication of the classifier relevance in the encountered situations. Accordingly, it is not generally possible to tell whether a classifier with a high fitness is accurate or not, just looking at the fitness. To have this kind of information, we need the prediction error ε which provides an indication of the raw classifier accuracy.