Partitioner Trees for Classification: A New Ensemble Method

Abstract

Two major types of ensemble methods exist: while the first type is based on the idea of varying the training data (Boosting, Bagging), the second type tries to exploit information on each classifier’s area of expertise (Grading, Delegating, Arbitrating). This paper presents a new ensemble method called partitioner trees that combines both approaches. Information on misclassifications is used to train meta classifiers called partitioners and to split training data into disjoint subsets. On these subsets succeeding specialised classifiers are constructed. This process yields a binary decision tree with partitioners on the inner nodes to perform the splitting and specialised local classifiers on the leaves for final classification. Partitioner trees are compared to five other ensemble methods in experiments on four different datasets. The results show that on large datasets partitioner trees have a similar classification accuracy as AdaBoost , and are superior to those of other meta classifier based ensemble methods. In addition, partitioner trees outperform most other ensemble methods in regard to training time and allow for adaptively tuning parameters.