Thomas Stützle
Manuel López-Ibáñez
ABSTRACT
Most optimization algorithms, including evolutionary algorithms and metaheuristics, and general-purpose solvers for integer or constraint programming, have often many parameters that need to be properly configured (i.e., tuned) for obtaining the best results on a particular problem. Automatic (offline) algorithm configuration methods help algorithm users to determine the parameter settings that optimize the performance of the algorithm before the algorithm is actually deployed. Moreover, automatic algorithm configuration methods may potentially lead to a paradigm shift in algorithm design and configuration because they enable algorithm designers to explore much larger design spaces than by traditional trial-and-error and experimental design procedures. Thus, algorithm designers can focus on inventing new algorithmic components, combine them in flexible algorithm frameworks, and let final algorithm design decisions be taken by automatic algorithm configuration techniques for specific application contexts.
This tutorial will be divided in two parts. The first part will give an overview of the algorithm configuration problem, review recent methods for automatic algorithm configuration, and illustrate the potential of these techniques using recent, notable applications from the presenters' and other researchers work. The second part of the tutorial will focus on a detailed discussion of more complex scenarios, including multi-objective problems, anytime algorithms, heterogeneous problem instances, and the automatic generation of algorithms from algorithm frameworks. The focus of this second part of the tutorial is, hence, on practical but challenging applications of automatic algorithm configuration. The second part of the tutorial will demonstrate how to tackle these configuration tasks using our irace software (http://iridia.ulb.ac.be/irace), which implements the iterated racing procedure for automatic algorithm configuration. We will provide a practical step-by-step guide on using irace for the typical algorithm configuration scenario.
- T. Achterberg. SCIP: Solving constraint integer programs. Mathematical Programming Computation, 1 (1): 1--41, July 2009.Google Scholar
Cross Ref
- B. Adenso-Díaz and M. Laguna. Fine-tuning of algorithms using fractional experimental design and local search. Operations Research, 54 (1): 99--114, 2006. Google ScholarDigital Library
- C. Ansótegui, M. Sellmann, and K. Tierney. A gender-based genetic algorithm for the automatic configuration of algorithms. In I. P. Gent, editor, Principles and Practice of Constraint Programming, CP 2009, volume 5732 of Lecture Notes in Computer Science, pages 142--157. Springer, Heidelberg, Germany, 2009. 10.1007/978-3-642-04244-7_14. Google ScholarDigital Library
- C. Audet and D. Orban. Finding optimal algorithmic parameters using derivative-free optimization. SIAM Journal on Optimization, 17 (3): 642--664, 2006. Google ScholarDigital Library
- C. Audet, C.-K. Dang, and D. Orban. Algorithmic parameter optimization of the DFO method with the OPAL framework. In K. Naono, K. Teranishi, J. Cavazos, and R. Suda, editors, Software Automatic Tuning: From Concepts to State-of-the-Art Results, pages 255--274. Springer, 2010.Google Scholar
- P. Balaprakash, M. Birattari, and T. Stützle. Improvement strategies for the F-race algorithm: Sampling design and iterative refinement. In T. Bartz-Beielstein, M. J. Blesa, C. Blum, B. Naujoks, A. Roli, G. Rudolph, and M. Sampels, editors, Hybrid Metaheuristics, volume 4771 of Lecture Notes in Computer Science, pages 108--122. Springer, Heidelberg, Germany, 2007. Google ScholarDigital Library
- T. Bartz-Beielstein, C. Lasarczyk, and M. Preuss. Sequential parameter optimization. In Proceedings of the 2005 Congress on Evolutionary Computation (CEC 2005), pages 773--780, Piscataway, NJ, Sept. 2005. IEEE Press.Google ScholarCross Ref
- T. Bartz-Beielstein, C. Lasarczyk, and M. Preuss. The sequential parameter optimization toolbox. In T. Bartz-Beielstein, M. Chiarandini, L. Paquete, and M. Preuss, editors, Experimental Methods for the Analysis of Optimization Algorithms, pages 337--360. Springer, Berlin, Germany, 2010.Google ScholarCross Ref
- S. Becker, J. Gottlieb, and T. Stützle. Applications of racing algorithms: An industrial perspective. In E.-G. Talbi, P. Liardet, P. Collet, E. Lutton, and M. Schoenauer, editors, Artificial Evolution, volume 3871 of Lecture Notes in Computer Science, pages 271--283. Springer, Heidelberg, Germany, 2005. Google ScholarDigital Library
- M. Birattari. The Problem of Tuning Metaheuristics as Seen from a Machine Learning Perspective. PhD thesis, IRIDIA, École polytechnique, Université Libre de Bruxelles, Belgium, 2004.Google Scholar
- M. Birattari, T. Stützle, L. Paquete, and K. Varrentrapp. A racing algorithm for configuring metaheuristics. In W. B. Langdon et al., editors, Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2002, pages 11--18. Morgan Kaufmann Publishers, San Francisco, CA, 2002.Google ScholarDigital Library
- M. Birattari, P. Balaprakash, and M. Dorigo. The ACO/F-RACE algorithm for combinatorial optimization under uncertainty. In K. F. Doerner, M. Gendreau, P. Greistorfer, W. J. Gutjahr, R. F. Hartl, and M. Reimann, editors, Metaheuristics -- Progress in Complex Systems Optimization, volume 39 of Operations Research/Computer Science Interfaces Series, pages 189--203. Springer, New York, NY, 2006.Google Scholar
- M. Birattari, Z. Yuan, P. Balaprakash, and T. Stützle. F-race and iterated F-race: An overview. In T. Bartz-Beielstein, M. Chiarandini, L. Paquete, and M. Preuss, editors, Experimental Methods for the Analysis of Optimization Algorithms, pages 311--336. Springer, Berlin, Germany, 2010.Google ScholarCross Ref
- E. K. Burke, M. R. Hyde, and G. Kendall. Grammatical evolution of local search heuristics. IEEE Transactions on Evolutionary Computation, 16 (7): 406--417, 2012. 10.1109/TEVC.2011.2160401. Google ScholarDigital Library
- M. Chiarandini, M. Birattari, K. Socha, and O. Rossi-Doria. An effective hybrid algorithm for university course timetabling. Journal of Scheduling, 9 (5): 403--432, Oct. 2006. 10.1007/s10951-006-8495-8. Google ScholarDigital Library
- W. J. Conover. Practical Nonparametric Statistics. John Wiley & Sons, New York, NY, third edition, 1999.Google Scholar
- S. P. Coy, B. L. Golden, G. C. Runger, and E. A. Wasil. Using experimental design to find effective parameter settings for heuristics. Journal of Heuristics, 7 (1): 77--97, 2001. Google ScholarDigital Library
- T. Dean and M. S. Boddy. An analysis of time-dependent planning. In Proceedings of the 7th National Conference on Artificial Intelligence, AAAI-88, pages 49--54. AAAI Press, 1988. Google ScholarDigital Library
- J. Dubois-Lacoste, M. López-Ibáñez, and T. Stützle. Automatic configuration of state-of-the-art multi-objective optimizers using the TP+PLS framework. In N. Krasnogor and P. L. Lanzi, editors, Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2011, pages 2019--2026. ACM Press, New York, NY, 2011. ISBN 978-1-4503-0557-0. 10.1145/2001576.2001847. Google ScholarDigital Library
- A. S. Fukunaga. Automated discovery of local search heuristics for satisfiability testing. Evolutionary Computation, 16 (1): 31--61, Mar. 2008. 10.1162/evco.2008.16.1.31. Google ScholarDigital Library
- J. J. Grefenstette. Optimization of control parameters for genetic algorithms. IEEE Transactions on Systems, Man, and Cybernetics, 16 (1): 122--128, 1986. Google ScholarDigital Library
- F. Hutter, D. Babić, H. H. Hoos, and A. J. Hu. Boosting verification by automatic tuning of decision procedures. In FMCAD'07: Proceedings of the 7th International Conference Formal Methods in Computer Aided Design, pages 27--34, Austin, Texas, USA, 2007. IEEE Computer Society, Washington, DC, USA. Google ScholarDigital Library
- F. Hutter, H. H. Hoos, and T. Stützle. Automatic algorithm configuration based on local search. In Proc. of the Twenty-Second Conference on Artifical Intelligence (AAAI '07), pages 1152--1157. AAAI Press / MIT Press, Menlo Park, CA, 2007. Google ScholarDigital Library
- F. Hutter, H. H. Hoos, K. Leyton-Brown, and T. Stützle. ParamILS: an automatic algorithm configuration framework. Journal of Artificial Intelligence Research, 36: 267--306, Oct. 2009. Google ScholarCross Ref
- F. Hutter, H. H. Hoos, and K. Leyton-Brown. Automated configuration of mixed integer programming solvers. In A. Lodi, M. Milano, and P. Toth, editors, Integration of AI and OR Techniques in Constraint Programming for Combinatorial Optimization Problems, 7th International Conference, CPAIOR 2010, volume 6140 of Lecture Notes in Computer Science, pages 186--202. Springer, Heidelberg, Germany, 2010. Google ScholarDigital Library
- F. Hutter, H. H. Hoos, and K. Leyton-Brown. Sequential model-based optimization for general algorithm configuration. In C. A. Coello Coello, editor, Learning and Intelligent Optimization, 5th International Conference, LION 5, volume 6683 of Lecture Notes in Computer Science, pages 507--523. Springer, Heidelberg, Germany, 2011. Google ScholarDigital Library
- A. R. KhudaBukhsh, L. Xu, H. H. Hoos, and K. Leyton-Brown. SATenstein: Automatically building local search SAT solvers from components. In C. Boutilier, editor, Proceedings of the Twenty-First International Joint Conference on Artificial Intelligence (IJCAI-09), pages 517--524. AAAI Press, Menlo Park, CA, 2009. Google ScholarDigital Library
- M. Lang, H. Kotthaus, Marwedel, C. Weihs, J. Rahnenführer, and B. Bischl. Automatic model selection for high-dimensional survival analysis. Journal of Statistical Computation and Simulation, 85 (1): 62--76, 2014. 10.1080/00949655.2014.929131.Google ScholarCross Ref
- K. Leyton-Brown, M. Pearson, and Y. Shoham. Towards a universal test suite for combinatorial auction algorithms. In A. Jhingran et al., editors, ACM Conference on Electronic Commerce (EC-00), pages 66--76. ACM Press, New York, NY, 2000. 10.1145/352871.352879. Google ScholarDigital Library
- T. Liao, M. A. Montes de Oca, and T. Stützle. Computational results for an automatically tuned CMA-ES with increasing population size on the CEC'05 benchmark set. Soft Computing, 17 (6): 1031--1046, 2013. 0.1007/s00500-012-0946-x. Google ScholarDigital Library
- M. López-Ibáñez and T. Stützle. The automatic design of multi-objective ant colony optimization algorithms. IEEE Transactions on Evolutionary Computation, 16 (6): 861--875, 2012. 10.1109/TEVC.2011.2182651. Google ScholarDigital Library
- M. López-Ibáñez, J. Dubois-Lacoste, T. Stützle, and M. Birattari. TheRpackageirace package, iterated race for automatic algorithm configuration. Technical Report TR/IRIDIA/2011-004, IRIDIA, Université Libre de Bruxelles, Belgium, 2011. URL http://iridia.ulb.ac.be/IridiaTrSeries/IridiaTr2011-004.pdf.Google Scholar
- F. Mascia, M. López-Ibáñez, J. Dubois-Lacoste, and T. Stützle. Grammar-based generation of stochastic local search heuristics through automatic algorithm configuration tools. Computers & Operations Research, 51: 190--199, 2014. 10.1016/j.cor.2014.05.020. Google ScholarDigital Library
- M. A. Montes de Oca, D. Aydín, and T. Stützle. An incremental particle swarm for large-scale continuous optimization problems: An example of tuning-in-the-loop (re)design of optimization algorithms. Soft Computing, 15 (11): 2233--2255, 2011. 10.1007/s00500-010-0649-0. Google ScholarDigital Library
- V. Nannen and A. E. Eiben. A method for parameter calibration and relevance estimation in evolutionary algorithms. In M. Cattolico et al., editors, Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2006, pages 183--190. ACM Press, New York, NY, 2006. 10.1145/1143997.1144029. Google ScholarDigital Library
- M. Oltean. Evolving evoluionary algorithms using linear genetic programming. Evolutionary Computation, 13 (3): 387--410, 2005. Google ScholarDigital Library
- P. Pellegrini, F. Mascia, T. Stützle, and M. Birattari. On the sensitivity of reactive tabu search to its meta-parameters. Soft Computing, 18 (11): 2177--2190, 2014. 10.1007/s00500-013--1192--6. Google ScholarDigital Library
- E. Ridge and D. Kudenko. Tuning the performance of the MMAS heuristic. In T. Stützle, M. Birattari, and H. H. Hoos, editors, International Workshop on Engineering Stochastic Local Search Algorithms (SLS 2007), volume 4638 of Lecture Notes in Computer Science, pages 46--60. Springer, Heidelberg, Germany, 2007. Google ScholarDigital Library
- R. Ruiz and C. Maroto. A comprehensive review and evaluation of permutation flow-shop heuristics. European Journal of Operational Research, 165 (2): 479--494, 2005.Google ScholarCross Ref
- S. K. Smit and A. E. Eiben. Comparing parameter tuning methods for evolutionary algorithms. In Proceedings of the 2009 Congress on Evolutionary Computation (CEC 2009), pages 399--406. IEEE Press, Piscataway, NJ, 2009. Google ScholarDigital Library
- S. K. Smit and A. E. Eiben. Beating the 'world champion' evolutionary algorithm via REVAC tuning. In H. Ishibuchi et al., editors, Proceedings of the 2010 Congress on Evolutionary Computation (CEC 2010), pages 1--8. IEEE Press, Piscataway, NJ, 2010. 10.1109/CEC.2010.5586026.Google Scholar
- J. A. Vázquez-Rodríguez and G. Ochoa. On the automatic discovery of variants of the NEH procedure for flow shop scheduling using genetic programming. Journal of the Operational Research Society, 62 (2): 381--396, 2010.Google ScholarCross Ref
- S. Wessing, N. Beume, G. Rudolph, and B. Naujoks. Parameter tuning boosts performance of variation operators in multiobjective optimization. In R. Schaefer, C. Cotta, J. Kolodziej, and G. Rudolph, editors, Parallel Problem Solving from Nature, PPSN XI, volume 6238 of phLecture Notes in Computer Science, pages 728--737. Springer, Heidelberg, Germany, 2010. 10.1007/978-3-642-15844-5_73. Google ScholarDigital Library
- Z. Yuan, M. A. Montes de Oca, T. Stützle, and M. Birattari. Continuous optimization algorithms for tuning real and integer algorithm parameters of swarm intelligence algorithms. Swarm Intelligence, 6 (1): 49--75, 2012.Google ScholarCross Ref
- Z. Yuan, M. A. Montes de Oca, T. Stützle, H. C. Lau, and M. Birattari. An analysis of post-selection in automatic configuration. In C. Blum and E. Alba, editors, Proceedings of GECCO 2013, pages 1557--1564. ACM Press, New York, NY, 2013. Google ScholarDigital Library
- S. Zilberstein. Using anytime algorithms in intelligent systems. AI Magazine, 17 (3): 73--83, 1996.Google ScholarDigital Library
Index Terms
- Automatic (Offline) Configuration of Algorithms
Recommendations
Automatic (Offline) Configuration of Algorithms
GECCO Companion '15: Proceedings of the Companion Publication of the 2015 Annual Conference on Genetic and Evolutionary ComputationMost optimization algorithms, including evolutionary algorithms and metaheuristics, and general-purpose solvers for integer or constraint programming, have often many parameters that need to be properly configured (i.e., tuned) for obtaining the best ...
Comments