Abstract
In empirical studies of Evolutionary Algorithms, it is usually desirable to evaluate and compare algorithms using as many different parameter settings and test problems as possible, in order to have a clear and detailed picture of their performance. Unfortunately, the total number of experiments required may be very large, which often makes such research work computationally prohibitive. In this paper, the application of a statistical method called racing is proposed as a general-purpose tool to reduce the computational requirements of large-scale experimental studies in evolutionary algorithms. Experimental results are presented that show that racing typically requires only a small fraction of the cost of an exhaustive experimental study.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Whitley, D., Mathias, K., Rana, S., Dzubera, J.: Evaluating Evolutionary Algorithms. Artificial Intelligence 85(1-2), 245–276 (1996)
De Jong, K.A., Potter, M.A., Spears, W.M.: Using Problem Generators to Explore the Effects of Epistasis. In: Bäck, T. (ed.) Seventh International Conference on Genetic Algorithms, pp. 338–345. Morgan Kauffman, San Francisco (1997)
Eiben, A.E., Jelasity, M.: A Critical Note on Experimental Research Methodology in EC. In: Congress on Evolutionary Computation, Hawaii, pp. 582–587. IEEE, Los Alamitos (2002)
Maron, O., Moore, A.W.: Hoeffding Races: Accelerating Model Selection Search for Classification and Function Approximation. In: Cowan, J.D., et al. (eds.) Advances in Neural Information Processing Systems, vol. 6, pp. 59–66 (1994)
Maron, O., Moore, A.W.: The Racing Algorithm: Model Selection for Lazy Learners. Artificial Intelligence Review 11, 193–225 (1997)
Hoeffding, W.: Probability Inequalities for Sums of Bounded Random Variables. Journal of the American Statistical Association 58(301), 13–30 (1963)
Conover, W.J.: Practical Nonparametric Statistics, 3rd edn. John Wiley & Sons, Inc., Chichester (1999)
Box, G.E.P., Hunter, W.G., Hunter, J.S.: Statistics for Experimenters. Wiley, Chichester (1978)
Birattari, M., Stutzle, T., Paquete, L., Varrentrapp, K.: A Racing Algorithm for Configuring Metaheuristics. In: Genetic and Evolutionary Computation Conference (GECCO 2002), pp. 11–18 (2002)
Mühlenbein, H., Paaß, G.: From Recombination of Genes to the Estimation of Distributions: I. Binary Parameters. In: Voigt, H.-M., et al. (eds.) Parallel Problem Solving from Nature IV, pp. 178–187 (1996)
Bishop, C.M.: Neural Networks for Pattern Recognition. Oxford University Press, Oxford (1995)
Gallagher, M.: Multi-Layer Perceptron Error Surfaces: Visualization, Structure and Modelling. PhD Thesis, The University of Queensland (2000)
Yuan, B., Gallagher, M.: On Building a Principled Framework for Evaluating and Testing Evolutionary Algorithms: A Continuous Landscape Generator. In: The 2003 Congress on Evolutionary Computation, pp. 451–458 (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Yuan, B., Gallagher, M. (2004). Statistical Racing Techniques for Improved Empirical Evaluation of Evolutionary Algorithms. In: Yao, X., et al. Parallel Problem Solving from Nature - PPSN VIII. PPSN 2004. Lecture Notes in Computer Science, vol 3242. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30217-9_18
Download citation
DOI: https://doi.org/10.1007/978-3-540-30217-9_18
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-23092-2
Online ISBN: 978-3-540-30217-9
eBook Packages: Springer Book Archive