Abstract
Error-correcting output codes (ECOCs) represent classes with a set of output bits, where each bit encodes a binary classification task corresponding to a unique partition of the classes. Algorithms that use ECOCs learn the function corresponding to each bit, and combine them to generate class predictions. ECOCs can reduce both variance and bias errors for multiclass classification tasks when the errors made at the output bits are not correlated. They work well with algorithms that eagerly induce global classifiers (e.g., C4.5) but do not assist simple local classifiers (e.g., nearest neighbor), which yield correlated predictions across the output bits. We show that the output bit predictions of local learners can be decorrelated by selecting different features for each bit. We present promising empirical results for this combination of ECOCs, nearest neighbor, and feature selection.
Chapter PDF
References
D. W. Aha and R. L. Bankert. Cloud classification using error-correcting output codes. Artificial Intelligence Applications: Natural Science, Agriculture, and Environmental Science, 11:13–28, 1997.
D. W. Aha, D. Kibler, and M. K. Albert. Instance-based learning algorithms. Machine Learning, 6:37–66, 1991.
D. W. Aha. Tolerating noisy, irrelevant, and novel attributes in instancebased learning algorithms. International Journal of Man-Machine Studies, 36:267–287, 1992.
L. Bottou and V. Vapnik. Local learning algorithms. Neural Computation, 4:888–900, 1992.
L. Breiman. Bias, variance, and arcing classifiers. Technical Report 460, University of California, Berkeley, April 1996.
T. G. Dietterich and G. Bakiri. Solving multiclass learning problems via error-correcting output codes. Journal of Artificial Intelligence Research, 2:263–286, 1995.
R. Kohavi and D. H. Wolpert. Bias plus variance decomposition for zero-one loss function. In Proceeding of the Thirteenth International Conference on Machine Learning, pages 275–283, Bari, Italy, 1996. Morgan Kaufmann.
E. B. Kong and T. G Dietterich. Error-correcting output coding corrects bias and variance. In Proceedings of the Twelfth International Conference on Machine Learning, pages 313–321, Tahoe City, CA, 1995. Morgan Kaufmann.
O. Maron and A. W. Moore. The racing algorithm: Model selection for lazy learners. Artificial Intelligence Review, pages 193–225, 1997.
C. Merz and P. M. Murphy.UCI repository of machine learning databases. [http://www.ics.uci.edu/≈mlearn/MLRepository.html], 1996.
M. P. Perrone and L. N. Cooper. When networks disagree: Ensemble methods for hybrid neural networks. In R. J. Mammone, editor, Neural Networks for Speech and Image Processing. Chapman and Hall, Philadelphia, PA, 1993.
J.R. Quinlan. C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo, CA, 1993.
F. Ricci and D. W. Aha. Bias, variance, and error correcting output codes for local learners. Technical Report 9711-10, IRST, 1997.
F. Ricci and D. W. Aha. Extending local learners with error-correcting output codes. Technical Report 9701-08, IRST, 1997.
D. E. Rumelhart, G. E. Hinton, and R. J. Williams. Learning internal representations by error propagation. In D. E. Rumelhart and J. L. McClelland, editors, Parallel Distributed Processing: Explorations in the Microstructures of Cognition. MIT Press, Cambridge, MA, 1986.
C. Stanfill and D. Waltz. Toward memory-based reasoning. Communication of ACM, 29:1213–1229, 1986.
X. Zhang, J. Mesirov, and D. Waltz. Hybrid system for protein structure prediction. Journal of Molecular Biology, 225:1049–1063, 1992.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1998 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ricci, F., Aha, D.W. (1998). Error-correcting output codes for local learners. In: Nédellec, C., Rouveirol, C. (eds) Machine Learning: ECML-98. ECML 1998. Lecture Notes in Computer Science, vol 1398. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0026698
Download citation
DOI: https://doi.org/10.1007/BFb0026698
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-64417-0
Online ISBN: 978-3-540-69781-7
eBook Packages: Springer Book Archive