Abstract
We are using bandit-based adaptive operator selection while autotuning parallel computer programs. The autotuning, which uses evolutionary algorithm-based stochastic sampling, takes place over an extended duration and occurs in situ as programs execute. The environment or context during tuning is either largely static in one scenario or dynamic in another. We rely upon adaptive operator selection to dynamically generate worthy test configurations of the program. In this paper, we study how the choice of hyperparameters, which control the trade-off between exploration and exploitation, affects the effectiveness of adaptive operator selection which in turn affects the performance of the autotuner. We show that while the optimal assignment of hyperparameters varies greatly between different benchmarks, there exists a single assignment, for a context, of hyperparameters that performs well regardless of the program being tuned.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Ansel, J., Chan, C., Wong, Y.L., Olszewski, M., Zhao, Q., Edelman, A., Amarasinghe, S.: Petabricks: A language and compiler for algorithmic choice. In: ACM SIGPLAN Conference on Programming Language Design and Implementation, Dublin, Ireland (June 2009)
Ansel, J., Wong, Y.L., Chan, C., Olszewski, M., Edelman, A., Amarasinghe, S.: Language and compiler support for auto-tuning variable-accuracy algorithms. In: International Symposium on Code Generation and Optimization, Chamonix, France (April 2011)
Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Mach. Learn. 47, 235–256 (2002)
Eiben, A., Hinterding, R., Michalewicz, Z.: Parameter control in evolutionary algorithms. IEEE Transactions on Evolutionary Computation 3(2), 124–141 (1999)
Fialho, Á.: Adaptive Operator Selection for Optimization. PhD thesis, Université Paris-Sud XI, Orsay, France (December 2010)
Fialho, Á., Da Costa, L., Schoenauer, M., Sebag, M.: Analyzing bandit-based adaptive operator selection mechanisms. Annals of Mathematics and Artificial Intelligence – Special Issue on Learning and Intelligent Optimization (2010)
Fialho, Á., Ros, R., Schoenauer, M., Sebag, M.: Comparison-Based Adaptive Strategy Selection with Bandits in Differential Evolution. In: Schaefer, R., Cotta, C., Kołodziej, J., Rudolph, G. (eds.) PPSN XI. LNCS, vol. 6238, pp. 194–203. Springer, Heidelberg (2010)
Maturana, J., Fialho, Á., Saubion, F., Schoenauer, M., Sebag, M.: Extreme com-pass and dynamic multi-armed bandits for adaptive operator selection. In: CEC 2009: Proc. IEEE International Conference on Evolutionary Computation, pp. 365–372. IEEE Press (May 2009)
Pacula, M.: Evolutionary algorithms for compiler-enabled program autotuning. Master’s thesis, Massachusetts Institute of Technology, Cambridge, MA (2011)
Price, K., Storn, R.M., Lampinen, J.A.: Differential Evolution: A Practical Approach to Global Optimization. Natural Computing Series. Springer-Verlag New York, Inc., Secaucus (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Pacula, M., Ansel, J., Amarasinghe, S., O’Reilly, UM. (2012). Hyperparameter Tuning in Bandit-Based Adaptive Operator Selection. In: Di Chio, C., et al. Applications of Evolutionary Computation. EvoApplications 2012. Lecture Notes in Computer Science, vol 7248. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-29178-4_8
Download citation
DOI: https://doi.org/10.1007/978-3-642-29178-4_8
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-29177-7
Online ISBN: 978-3-642-29178-4
eBook Packages: Computer ScienceComputer Science (R0)