Nitin Indurkhya
ABSTRACT
We describe a lightweight learning method that induces an ensemble of decision-rule solutions for regression problems. Instead of direct prediction of a continuous output variable, the method discretizes the variable by k-means clustering and solves the resultant classification problem. Predictions on new examples are made by averaging the mean values of classes with votes that are close in number to the most likely class. We provide experimental evidence that this indirect approach can often yield strong results for many applications, generally outperforming direct approaches such as regression trees and rivaling bagged regression trees.
- 1.E. Bauer and R. Kohavi. An empirical comparison of voting classification algorithms: Bagging, boosting and variants. Machine Learning, 36(1):105-139, 1999. Google Scholar
Digital Library
- 2.L. Breiman. Bagging predictors. Machine Learning, 24:123-140, 1996. Google ScholarDigital Library
- 3.L. Breiman, J. Friedman, R. Olshen, and C. Stone. Classification and Regression Trees. Wadsworth, Monterrey, CA., 1984.Google Scholar
- 4.R. Camacho. Inducing Models of Human Control Skills using Machine Learning Algorithms. Ph.d. thesis, University of Porto, 2000.Google Scholar
- 5.W. Cohen and Y. Singer. A simple, fast, and effective rule learner. In Proceedings of Annual Conference of American Association for Artificial Intelligence, pages 335-342, 1999. Google ScholarDigital Library
- 6.J. Dougherty, R. Kohavi, and M. Sahami. Supervised and unsupervised discretization of continuous features. In Proceedings of the 12th Int'l Conference on Machine Learning, pages 194-202, 1995.Google ScholarCross Ref
- 7.U. Fayyad and K. Irani. Multi-interval discretization of continuous-valued attributes for classification learning. In Proceedings of the 13th Int'l Joint Conference on Artificial Intelligence, pages 1022-1027, 1993.Google Scholar
- 8.J. Friedman. Multivariate adaptive regression splines. Annals of Statistics, 19(1):1-141, 1991.Google ScholarCross Ref
- 9.J. Friedman, T. Hastie, and R. Tibshirani. Additive logistic regression: A statistical view of boosting. Technical report, Stanford University Statistics Department, 1998. www.stat-stanford.edu/~tibs.Google Scholar
- 10.J. Hartigan and M. Wong. A k-means clustering algorithm, ALGORITHM AS 136. Applied Statistics, 28(1), 1979.Google Scholar
- 11.R. Schapire, Y. Freund, P. Bartlett, and W. Lee. Boosting the margin: A new explanation for the effectiveness of voting methods. In Proceedings of the Fourteenth Int'l Conference on Machine Learning, pages 322-330. Morgan Kanhnann, 1998. Google ScholarDigital Library
- 12.L. Torgo and J. Gama. Regression using classification algorithms. Intelligent Data Analysis, 1(4), 1997.Google Scholar
- 13.S. Weiss and N. Indurkhya. Rule-based machine learning methods for functional prediction. Journal of Artificial Intelligence Research, 3:383-403, 1995. Google ScholarDigital Library
- 14.S. Weiss and N. Indurkhya. Lightweight rule induction. In Proceedings of the Seventeenth International Conference on Machine Learning, pages 1135-1142, 2000. Google ScholarDigital Library
Index Terms
- Solving regression problems with rule-based ensemble classifiers
Recommendations
On the Interpretation of Ensemble Classifiers in Terms of Bayes Classifiers
Many of the best classifiers are ensemble methods such as bagging, random forests, boosting, and Bayes model averaging. We give conditions under which each of these four classifiers can be regarded as a Bayes classifier. We also give conditions under ...
Learning regression problems by using classifiers
Special Section: Recent Advances in Machine Learning and Soft ComputingRegression via Classification (RvC) is a process to solve a regression problem by using a classifier. An ensemble consists of many models, in which the final result is the combination of the results of these individual models. In this paper, two RvC ...
Ensemble-based classifiers
The idea of ensemble methodology is to build a predictive model by integrating multiple models. It is well-known that ensemble methods can be used for improving prediction performance. Researchers from various disciplines such as statistics and AI ...
Comments