Abstract
A major topic in machine learning is to determine good upper bounds on the true error rates of learned hypotheses based upon their empirical performance on training data. In this paper, we demonstrate new adaptive bounds designed for learning algorithms that operate by making a sequence of choices. These bounds, which we call Microchoice bounds, are similar to Occam-style bounds and can be used to make learning algorithms self-bounding in the style of Freund (1998). We then show how to combine these bounds with Freund's query-tree approach producing a version of Freund's query-tree structure that can be implemented with much more algorithmic efficiency.
Article PDF
Similar content being viewed by others
References
Blumer, A., Ehrenfeucht, A., Haussler, D., & Warmuth, M. K. (1987). Occam's razor. Information Processing Letters, 24, 377–380.
Domingos, P. (1998). A process-oriented heuristic for model selection. In Machine Learning Proceedings of the Fifteenth International Conference (pp. 127–135). San Francisco, CA: Morgan Kaufmann Publishers.
Freund, Y. (1998). Self bounding learning algorithms. In Proceedings of the 11th Annual Conference on Computational Learning Theory (pp. 247–258). New York, NY: ACM Press.
Kearns, M. (1993). Efficient noise-tolerant learning from statistical queries. In Proceedings of the Twenty-Fifth Annual ACM Symposium on the Theory of Computing (pp. 392–401). New York, NY: ACM Press.
McAllester, D. A. (1998). Some PAC-Bayesian theorems. In Proceedings of the 11th Annual Conference on Computational Learning Theory (pp. 230–234). New York, NY: ACM Press.
Mingers, J. (1989). An empirical comparison of pruning methods for decision tree induction. Machine Learning 4, 227–243.
Quinlan, J. L. (1986). Induction of decision trees. Machine Learning, 1, 81–106.
Rivest, R. L. (1987). Learning decision lists. Machine Learning, 2:3, 229–246.
Shawe-Taylor, J., Bartlett, P., Williamson, R., & Anthony, M. (1996).A framework for structural risk minimization. In Proceedings of the 9th Annual Conference on Computational Learning Theory (pp. 68–88). New York, NY: ACM Press
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Langford, J., Blum, A. Microchoice Bounds and Self Bounding Learning Algorithms. Machine Learning 51, 165–179 (2003). https://doi.org/10.1023/A:1022806918936
Issue Date:
DOI: https://doi.org/10.1023/A:1022806918936