Skip to main content

Implementation Effort and Performance

A Comparison of Custom and Out-of-the-Box Metaheuristics on the Vehicle Routing Problem with Stochastic Demand

  • Conference paper
Engineering Stochastic Local Search Algorithms. Designing, Implementing and Analyzing Effective Heuristics (SLS 2007)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 4638))

Abstract

In practical applications, one can take advantage of metaheuristics in different ways: To simplify, we can say that metaheuristics can be either used out-of-the-box or a custom version can be developed. The former way requires a rather low effort, and in general allows to obtain fairly good results. The latter implies a larger investment in the design, implementation, and fine-tuning, and can often produce state-of-the-art results.

Unfortunately, most of the research works proposing an empirical analysis of metaheuristics do not even try to quantify the development effort devoted to the algorithms under consideration. In other words, they do not make clear whether they considered out-of-the-box or custom implementations of the metaheuristics under analysis. The lack of this information seriously undermines the generality and utility of these works.

The aim of the paper is to stress that results obtained with out-of-the-box implementations cannot be always generalized to custom ones, and vice versa. As a case study, we focus on the vehicle routing problem with stochastic demand and on five among the most successful metaheuristics—namely, tabu search, simulated annealing, genetic algorithm, iterated local search, and ant colony optimization. We show that the relative performance of these algorithms strongly varies whether one considers out-of-the-box implementations or custom ones, in which the parameters are accurately fine-tuned.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Glover, F.: Future paths for integer programming and links to artificial intelligence. Computers & Operations Research 13, 533–549 (1986)

    Article  MATH  MathSciNet  Google Scholar 

  2. Dorigo, M., Stützle, T.: Ant Colony Optimization. MIT Press, Cambridge (2004)

    MATH  Google Scholar 

  3. Bianchi, L., Birattari, M., Chiarandini, M., Manfrin, M., Mastrolilli, M., Paquete, L., Rossi-Doria, O., Schiavinotto, T.: Hybrid metaheuristics for the vehicle routing problem with stochastic demands. Journal of Mathematical Modelling and Algorithms 5(1), 91–110 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  4. Birattari, M., Stützle, T., Paquete, L., Varrentrapp, K.: A racing algorithm for configuring metaheuristics. In: Langdon, W. (ed.) GECCO 2002. Proceedings of the Genetic and Evolutionary Computation Conference, pp. 11–18. Morgan Kaufmann, San Francisco (2002)

    Google Scholar 

  5. Birattari, M.: The problem of tuning metaheuristics as seen from a machine learning perspective. PhD thesis, Université Libre de Bruxelles, Brussels, Belgium (2005)

    Google Scholar 

  6. Bartz-Beielstein, T.: Experimental analysis of evolution strategies - overview and comprehensive introduction. Technical Report CI-157/03, Interner Bericht des Sonderforschungsbereichs 531 Computational Intelligence, Universität Dortmund, Dortmund, Germany (2003)

    Google Scholar 

  7. Tillman, F.: The multiple terminal delivery problem with probabilistic demands. Transportation Science 3, 192–204 (1969)

    Google Scholar 

  8. Stewart, W., Golden, B.: Stochastic vehicle routing: a comprehensive approach. European Journal of Operational Research 14, 371–385 (1983)

    Article  MATH  Google Scholar 

  9. Dror, M., Trudeau, P.: Stochastic vehicle routing with modified saving algorithm. European Journal of Operational Research 23, 228–235 (1986)

    Article  MATH  MathSciNet  Google Scholar 

  10. Laporte, G., Louveau, F., Mercure, H.: Models and exact solutions for a class of stochastic location-routing problems. Technical Report G-87-14, Ecole des Hautes Etudes Commerciale, University of Montreal, Montreal, Canada (1987)

    Google Scholar 

  11. Bertsimas, D.: A vehicle routing problem with stochastic demand. Operations Research 40(3), 574–585 (1992)

    MATH  MathSciNet  Google Scholar 

  12. Bertsimas, D., Simchi-Levi, D.: A new generation of vehicle routing research: robust algorithms, addressing uncertainty. Operations Research 44(3), 286–304 (1996)

    Article  MATH  Google Scholar 

  13. Yang, W., Mathur, K., Ballou, R.: Stochastic vehicle routing problem with restocking. Transportation Science 34(1), 99–112 (2000)

    Article  MATH  Google Scholar 

  14. Secomandi, N.: A rollout policy for the vehicle routing problem with stochastic demands. Operations Research 49, 796–802 (2001)

    Article  MATH  Google Scholar 

  15. Secomandi, N.: Analysis of a rollout approach to sequencing problems with stochastic routing applications. Journal of Heuristics 9, 321–352 (2003)

    Article  MATH  Google Scholar 

  16. Teodorović, D., Pavković, G.: A simulated annealing technique approach to the VRP in the case of stochastic demand. Transportation Planning and Technology 16, 261–273 (1992)

    Article  Google Scholar 

  17. Gendreau, M., Laporte, G., Séguin, R.: A tabu search heuristic for the vehicle routing problem with stochastic demands and customers. Working paper, CRT, University of Montreal, Montreal, Canada (1994)

    Google Scholar 

  18. Glover, F., Laguna, M.: Tabu Search. Kluwer Academic Publishers, Norwell (1997)

    MATH  Google Scholar 

  19. Ingber, L.: Adaptive simulated annealing (ASA): lessons learned. Control and Cybernetics 26(1), 33–54 (1996)

    Google Scholar 

  20. Bäck, T., Fogel, D., Michalewicz, Z. (eds.): Handbook of Evolutionary Computation. IOP Publishing Ltd. Bristol, UK (1997)

    MATH  Google Scholar 

  21. Laurenço, H., Martin, O., Stützle, T.: Iterated local search. In: Glover, F., Kochenberger, G. (eds.) Handbook of Metaheuristics, pp. 321–353. Kluwer Academic Publishers, Norwell (2002)

    Google Scholar 

  22. Zlochin, M., Birattari, M., Meuleau, N., Dorigo, M.: Model-based search for combinatorial optimization: A critical survey. Annals of Operations Research 131(1–4), 373–395 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  23. Adenso-Díaz, B., Laguna, M.: Fine-tuning of algorithms using fractional experimental designs and local search. Operations Research 54(1), 99–114 (2006)

    Article  Google Scholar 

  24. Barr, R., Kelly, J., Resende, M., Stewart, W.: Designing and reporting computational experiments with heuristic methods. Journal of Heuristics 1(1), 9–32 (1995)

    Article  MATH  Google Scholar 

  25. Bartz-Beielstein, T., Markon, S.: Tuning search algorithms for real-world applications: A regression tree based approach. In: Greenwood, G. (ed.) CEC 2004. Proc. 2004 Congress on Evolutionary Computation, Piscataway, NJ, USA, pp. 1111–1118. IEEE Computer Society Press, Los Alamitos (2004)

    Google Scholar 

  26. Coy, S., Golden, B., Runger, G., Wasil, E.: Using experimental design to find effective parameter settings for heuristics. Journal of Heuristics 7(1), 77–97 (2001)

    Article  MATH  Google Scholar 

  27. Xu, J., Kelly, J.: A network flow-based tabu search heuristic for the vehicle routing problem. Transportation Science 30, 379–393 (1996)

    MATH  Google Scholar 

  28. Parson, R., Johnson, M.: A case study in experimental design applied to genetic algorithms with applications to dna sequence assembly. American Journal of Mathematical and Management Sciences 17, 369–396 (1997)

    Google Scholar 

  29. Breedam, A.V.: An analysis od the effect of local improvement operators in genetic algorithms and simulated annealing for the vehicle routing problem. Technical Report TR 96/14, Faculty of Applied Economics, University of Antwerp, Antwerp, Belgium (1996)

    Google Scholar 

  30. Xu, J., Chiu, S., Glover, F.: Fine-tuning a tabu search algorithm with statistical tests. International Transactions on Operational Research 5(3), 233–244 (1998)

    Article  Google Scholar 

  31. Pellegrini, P., Birattari, M.: Instances generator for the vehicle routing problem with stochastic demand. Technical Report TR/IRIDIA/2005-10, IRIDIA, Université Libre de Bruxelles, Brussels, Belgium (2005)

    Google Scholar 

  32. Pellegrini, P., Birattari, M.: The relevance of tuning the parameters of metaheuristics. A case study: The vehicle routing problem with stochastic demand. Technical Report TR/IRIDIA/2006-008, IRIDIA, Université Libre de Bruxelles, Brussels, Belgium (submitted for journal publication, 2006)

    Google Scholar 

  33. Aarts, E., Korst, J., van Laarhoven, P.: Simulated annealing. In: Aarts, E., Lenstra, J. (eds.) Local Search in Combinatorial Optimization, pp. 91–120. John Wiley & Sons, Inc. New York, USA (1997)

    Google Scholar 

  34. Whitley, D., Starkweather, T., Shaner, D.: The traveling salesman problem and sequence scheduling: quality solutions using genetic edge recombination. In: Davis, L. (ed.) Handbook of Genetic Algorithms, pp. 350–372. Van Nostrand Reinhold, New York, USA (1991)

    Google Scholar 

  35. Friedman, J.: Multivariate adaptive regression splines. The Annals of Statistics 19, 1–141 (1991)

    MATH  MathSciNet  Google Scholar 

  36. Birattari, M., Zlochin, M., Dorigo, M.: Towards a theory of practice in metaheuristics design: A machine learning perspective. Theoretical Informatics and Applications, Accepted for publication (2006)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Thomas Stützle Mauro Birattari Holger H. Hoos

Rights and permissions

Reprints and permissions

Copyright information

© 2007 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Pellegrini, P., Birattari, M. (2007). Implementation Effort and Performance. In: Stützle, T., Birattari, M., H. Hoos, H. (eds) Engineering Stochastic Local Search Algorithms. Designing, Implementing and Analyzing Effective Heuristics. SLS 2007. Lecture Notes in Computer Science, vol 4638. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74446-7_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-74446-7_3

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-74445-0

  • Online ISBN: 978-3-540-74446-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics