Progressive Boosting for Classifier Committee Learning

Abstract

Most applications of artificial intelligence to tasks of practical importance are based on constructing a model of the knowledge used by a human expert. In a classification model, the connection between classes and properties can be defined by something as simple as a flowchart or as complex and unstructured as a procedures manual. Classifier committee learning methods generate multiple classifiers to form a committee by repeated application of a single base learning algorithm. The committee members vote to decide the final classification. Two such methods are bagging and boosting for improving the predictive power of classifier learning systems. This paper studies a different approach progressive boosting of decision trees. Instead of sampling the same number of data points at each boosting iteration t, our progressive boosting algorithm draws n _t data according to the sampling schedule. an empirical evaluation of a variant of this method shows that the progressive boosting can significantly reduce the error rate of decision tree learning. On average this is more accurate than bagging and boosting.