The complexity of batch approaches to reduced error rule set induction

Abstract

Cohen [3] introduced a rule set improvement method, Grow, that is used in classifier learning in a similar way to standard reduced error pruning methods, but is based on “reduced error rule set re-growth”. Here we follow Cohen's suggestion that order of magnitude analysis of the time complexity of such reduced error methods on random data provides insight into their behaviour on real data sets that are noisy. We consider the growth of rule sets produced for such data by these methods, and suggest that the size of the final rule set is roughly of order n, for n training items, whereas Cohen assumed it was roughly constant. This leads to increased estimates of the relevant time complexities. We propose a simple improvement to the implementation to reduce the order of the time complexities by about n. We give experimental results in support of our rough order of magnitude claims.