Boosted Geometry-Based Ensembles

Abstract

Geometry-based ensembles is a newly proposed algorithm based on the concept of characterizing boundary points. These points are found from the geometry of the data set and belong to the optimal boundary between classes under a certain notion of robustness. The characterizing boundary points can be used to build a classifier. Based on these points, a set of locally robust linear classifiers is defined and assembled in an additive model to create a final decision rule. As a result a strong classifier able to compete with nowadays state-of-the-art classifiers is obtained. The main drawback of the original proposal comes from the fact that the complexity of the created model can be arbitrarily high and depends on the data set. Moreover, outliers and noise may increase this number. In this article, small complexity models with strong generalization capability are explored. Two incremental non-parametric additive building algorithms are considered: boosting and least squared residual fitting approaches. Moreover, the last method is extended to deal with incremental L2 penalized solutions (which implicitly combines the advantages of sparse models and smooth ones due to the complexity limit). The validation of the approach on the UCI database achieves very promising results assessing the validity of CBP based classifiers ensembles.