An Incremental Bit Allocation Strategy for Supervised Feature Discretization

Abstract

Feature discretization (FD) is a necessary pre-processing step for many machine learning tasks. Its use often yields compact and robust data representations, leading to more accurate classifiers and lower training times. In this paper, we propose an incremental supervised FD technique based on recursive bit allocation. The proposed algorithm starts with a pool of bits and, at each stage, if there are still bits left in the pool, allocates the next bit to the most promising feature, i.e., the one which, after discretization, has the highest mutual information with the class label. Since it may happen that one (or more) feature(s) receives no bits at all, this FD procedure has a built-in feature selection effect. The experimental evaluation on public domain benchmark datasets shows that the proposed method obtains similar or better results, both in terms of classification accuracy and number of discretization intervals, as compared to other state-of-the-art supervised FD techniques.