Repechage bootstrap aggregating for misclassification cost reduction

Abstract

This paper examines the use of bootstrap aggregating (bagging) with classifier learning methods based upon hold-out pruning (or growing) for misclassification cost reduction. Both decision tree and rule set classifiers are used. The paper introduces a “repechange” variation of bagging, that uses, as the hold-out data for cost reduction, the “out of bag” items, which would be unused in standard bagging. The paper presents experimental evidence that, when used with the hold-out cost reduction methods, the repechage, method can achieve better misclassification cost results than the straightforward use of standard bagging used with the same hold-out cost reduction method. Superior results for the repechange method on some problems with previously defined cost matrices are shown for a cost reduction decision tree method and two cost reduction rule set methods.