Zhipeng Xie
ABSTRACT
This paper proposes an algorithm to integrate AdaBoost.M1 with the decision trees that output confidence-rated predictions, which is done by transforming decision trees from "expert" models to "specialist" models that may abstain when the confidence is less than 1/2. The confidence is used to update the instance weights during the boosting process, and it is also used to determine the vote weights of base classifiers in decision process. This makes the algorithm a "dynamic" one, in that: (1) for a given test instance, only those whose confidences are higher than 1/2 can vote on the decision making; and (2) the vote weight of each base classifier is dependent on the confidence that the classifier has on the target instance. Experimental results with C4.5 decision tree learner as the base learning algorithm have shown that this algorithm has significantly outperformed both the base algorithm and the AdaBoost.M1 of the C4.5 decision trees with simple predictions.
- Blake, C., Keogh, E., & Merz, C. J. UCI Repository of Machine Learning Databases. http://www.ics.uci.edu/~mlearn/MLRepository.html. Irvine, CA: University of California, Department of Information and Computer Science, 1998Google Scholar
- Blum, A. Empirical support for winnow and weighted-majority based algorithms: results on a calendar scheduling domain. Machine Learning, 1997, 26: 5--23 Google ScholarDigital Library
- Freund, Y. and Schapire, R. E. Experiments with a new boosting algorithm. In Machine Learning: Proceedings of the Thirteenth International Conference, 148--156, 1996Google Scholar
- Freund, Y. and Schapire, R. E. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 1997, 55(1): 119--139 Google ScholarDigital Library
- Quinlan, J. R. C4.5: Programs for machine learning. Morgan Kaufmann, 1993 Google ScholarDigital Library
- Quinlan, J. R. Bagging, Boosting, and C4.5. Proceedings of the Thirteenth National Conference on Artificial Intelligence, pp. 725--730, 1996. Google ScholarDigital Library
- Schapire, R. E. and Singer, Y. Improved boosting algorithms using confidence-rated predictions, Machine Learning, 1999, 37: 297--336 Google ScholarDigital Library
- Schapire, R. E. The boosting approach to machine learning: An overview. In D. D. Denison, M. H. Hansen, C. Holmes, B. Mallick, B. Yu, editors, Nonlinear Estimation and Classification. Springer, 2003.Google Scholar
Index Terms
- Improved AdaBoost.M1 of decision trees with confidence-rated predictions
Recommendations
Improved Boosting Algorithms Using Confidence-rated Predictions
The Eleventh Annual Conference on computational Learning TheoryWe describe several improvements to Freund and Schapire‘s AdaBoost boosting algorithm, particularly in a setting in which hypotheses may assign confidences to each of their predictions. We give a simplified analysis of AdaBoost in this setting, and we ...
Ensembles of decision trees for imbalanced data
MCS'11: Proceedings of the 10th international conference on Multiple classifier systemsEnsembles of decision trees are considered for imbalanced datasets. Conventional decision trees (C4.5) and trees for imbalanced data (CCPDT: Class Confidence Proportion Decision Tree) are used as base classifiers. Ensemble methods, based on ...
An Efficient Rule-Based Classification of Diabetes Using ID3, C4.5, & CART Ensembles
FIT '14: Proceedings of the 2014 12th International Conference on Frontiers of Information TechnologyConventional techniques for clinical decision support systems are based on a single classifier or simple combination of these classifiers used for disease diagnosis and prediction. Recently much attention has been paid on improving the performance of ...
Comments