Abstract
Racing algorithms have recently been proposed as a general-purpose method for performing model selection in machine learning algorithms. In this paper, we present an empirical study of the Hoeffding racing algorithm for selecting the k parameter in a simple k-nearest neighbor classifier. Fifteen widely-used classification datasets from UCI are used and experiments conducted across different confidence levels for racing. The results reveal a significant amount of sensitivity of thek -nn classifier to its model parameter value. The Hoeffding racing algorithm also varies widely in its performance, in terms of the computational savings gained over an exhaustive evaluation. While in some cases the savings gained are quite small, the racing algorithm proved to be highly robust to the possibility of erroneously eliminating the optimal models. All results were strongly dependent on the datasets used.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Maron, O., Moore, A.W.: Hoeffding Races: Accelerating Model Selection Search for Classification and Function Approximation. In: Cowan, J.D., Tesauro, G., Alspector, J. (eds.) Advances in Neural Information Processing System. Morgan Kaufmann, San Francisco (1994)
Maron, O., Moore, A.W.: The Racing Algorithm: Model Selection for Lazy Learners. Artificial Intelligence Review, 193–225 (1997)
Birattari, M., Stuetzle, T., Paquete, L., Varrentrapp, K.: A Racing Algorithm for Configuring Metaheuristics. In: Genetic and Evolutionary Computation Conference 2002, pp. 11–18. Morgan Kaufmann Publishers, New York (2002)
Yuan, B., Gallagher, M.: Statistical Racing Techniques for Improved Empirical Evaluation of Evolutionary Algorithms. In: Yao, X., Burke, E.K., Lozano, J.A., Smith, J., Merelo-Guervós, J.J., Bullinaria, J.A., Rowe, J.E., Tiňo, P., Kabán, A., Schwefel, H.-P. (eds.) PPSN 2004. LNCS, vol. 3242, pp. 172–181. Springer, Heidelberg (2004)
Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Infererence and Prediction. Springer series in statistics. Springer, New York (2001)
Atkeson, C.G., Moore, A.W., Schaal, S.: Locally Weighted Learning. Artificial Intelligence Review 10 (1996)
Mitchell, T.M.: Machine Learning. The McGraw-Hill Companies, Inc., New York (1997)
Hittich, S., Blake, C.L., Merz, C.J.: UCI Repository of Machine Learning Databases. University of California Irvine, USA (1998), http://www.ics.uci.edu/~learn/MLRepository.html
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Yeh, F.YH., Gallagher, M. (2005). An Empirical Study of Hoeffding Racing for Model Selection in k-Nearest Neighbor Classification. In: Gallagher, M., Hogan, J.P., Maire, F. (eds) Intelligent Data Engineering and Automated Learning - IDEAL 2005. IDEAL 2005. Lecture Notes in Computer Science, vol 3578. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11508069_29
Download citation
DOI: https://doi.org/10.1007/11508069_29
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-26972-4
Online ISBN: 978-3-540-31693-0
eBook Packages: Computer ScienceComputer Science (R0)