Skip to main content
SpringerLink
Log in
Book cover

Experimental Methods for the Analysis of Optimization Algorithms pp 225–264Cite as

Mixed Models for the Analysis of Optimization Algorithms

  • Chapter
  • First Online:

Abstract

We review linear statistical models for the analysis of computational experiments on optimization algorithms. The models offer the mathematical framework to separate the effects of algorithmic components and instance features included in the analysis. We regard test instances as drawn from a population and we focus our interest not on those single instances but on the whole population. Hence, instances are treated as a random factor. Overall these experimental designs lead to mixed effects linear models. We present both the theory to justify these models and a computational example in which we analyze and comment on several possible experimental designs. The example is a component-wise analysis of local search algorithms for the 2-edge-connectivity augmentation problem. We use standard statistical software to perform the analysis and report the R commands. Data sets and the analysis in SAS are available in an online compendium.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   159.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Bang-Jensen J, Chiarandini M, Morling P (2009) A computational investigation of heuristic algorithms for 2-edge-connectivity augmentation.Networks (In print.)

    Google Scholar 

  • Barr R, Golden B, Kelly J, Resende M, Stewart W (1995) Designing and reporting on computational experiments with heuristic methods.Journal of Heuristics 1(1):9–32

    Article  MATH  Google Scholar 

  • Bates D (2007) Personal Communication

    Google Scholar 

  • Bates D, Maechler M, Dai B (2008) lme4:Linear mixed-effects models using S4 classes. URL http://lme4.r-forge.r-project.org/, R package version 0.999375-28

  • Birattari M (2004) On the estimation of the expected performance of a metaheuristic on a class of instances. How many instances, how many runs? Tech. Rep. TR/IRIDIA/2004-01, IRIDIA, Université Libre de Bruxelles, Brussels, Belgium

    Google Scholar 

  • Bondy J, Murty U (2008) Graph Theory. Graduate Texts in Mathematics, Vol. 244,Springer

    Google Scholar 

  • Chiarandini M (2005) Stochastic local search methods for highly constrained combinatorial optimisation problems. PhD thesis, Computer Science Department, Darmstadt University of Technology, Darmstadt, Germany

    Google Scholar 

  • Coffin M, Saltzman MJ (2000)Statistical analysis of computational tests of algorithms and heuristics. INFORMS Journal on Computing 12(1):24–44

    Article  MATH  Google Scholar 

  • Conforti M, Galluccio A, Proietti G (2004) Edge-connectivity augmentation and network matrices. In:Workshop on Graph-Theoretic Concepts in Computer Science, Springer, Lecture Notes in Computer Science, vol 3353, pp 355–364

    Chapter  Google Scholar 

  • Cormen T, Leiserson C, Rivest R (2001) Introduction to Algorithms, 2nd edn.MIT press

    MATH  Google Scholar 

  • Czarn A, MacNish C, Vijayan K, Turlach B, Gupta R (2004) Statistical exploratory analysis of genetic algorithms. Evolutionary Computation, IEEE Transactions on 8(4):405–421

    Article  Google Scholar 

  • Fox J (2002) Linear mixed models. Appendix to An R and S-PLUS Companion to Applied Regression,URL http://cran.r-project.org/doc/ contrib/Fox-Companion/appendix-mixed-models.pdf

  • Glover F, Kochenberger G (eds) (2002) Handbook of Metaheuristics, International Series in Operations Research & Management Science, vol 57. Kluwer Academic, Norwell, MA

    Google Scholar 

  • Hooker JN (1996) Testing heuristics: We have it all wrong. Journal of Heuristics 1:32–42

    Google Scholar 

  • Johnson R, Wichern D (2007) Applied Multivariate Statistical Analysis, 6th edn.Prentice-Hall International

    MATH  Google Scholar 

  • MKutner MH, Nachtsheim CJ, Neter J, Li W (2005) Applied Linear Statistical Models, 5th edn. McGraw-Hill

    Google Scholar 

  • Lehmann E (2003) Theory of point estimation.Springer

    Google Scholar 

  • Lehmann E, Romano J (2008) Testing statistical hypothesis.Springer

    Google Scholar 

  • Lenth RV (2006) Java applets for power and sample size [computer software]. Retrieved 29 January 2009 from http://www.stat.uiowa.edu/˜rlenth/ Power

  • Lin BW, Rardin RL (1979) Controlled experimental design for statistical comparison of integer programming algorithms. Management Science 25(12):1258–1271

    Article  MathSciNet  Google Scholar 

  • McGeoch CC (1996) Toward an experimental method for algorithm simulation. INFORMS Journal on Computing 8(1):1–15, this journal issue contains also commentaries by Pierre L’Ecuyer, James B. Orlin and Douglas R. Shier, and a rejoinder by C. McGeoch

    Article  MATH  MathSciNet  Google Scholar 

  • Michiels W, Aarts E, Korst J (2007) Theoretical Aspects of Local Search. Monographs in Theoretical Computer Science, An EATCS Series, Springer

    Google Scholar 

  • Molenberghs G, Verbeke G (eds) (1997)Linear Mixed Models in Practice - A SASOriented Approach. >Springer

    Google Scholar 

  • Molenberghs G, Verbeke G (2005) Models for Discrete Longitudinal Data. Springer

    MATH  Google Scholar 

  • Montgomery DC (2005) Design and Analysis of Experiments, 6th edn. Wiley

    MATH  Google Scholar 

  • Pinheiro J, Bates D, DebRoy S, Sarkar D, the R Core team (2008) nlme: Linear and Nonlinear Mixed Effects Models. R package version 3.1–89

    Google Scholar 

  • Pinheiro JC, Bates DM (2000) Mixed-Effects Models in S and S-Plus. Springer

    Book  MATH  Google Scholar 

  • R Development Core Team (2008) R: A Language and Environment for StatisticalComputing. R Foundation for Statistical Computing, Vienna, Austria, URL http://www.R-project.org

  • Rardin RL, Uzsoy R (2001) Experimental evaluation of heuristic optimization algorithms: A tutorial. Journal of Heuristics 7(3):261–304

    Article  MATH  Google Scholar 

  • Rousseeuw PJ (1984) Least median of squares regression. Journal of the American Statistical Association 79(388):871–880, URL http://www.jstor.org/ stable/2288718 SAS Institute Inc. (2007) SAS online documentation: Parameterization of mixed models. http://www.webcitation.org/5h0u00trT, retrieved on 2009-05-24

  • Stram D, Lee J (1994) Variance components testing in the longitudinal mixed effects model. Biometrics 50(4):1171–1177

    Article  MATH  Google Scholar 

  • Stram D, Lee J (1995) Correction to ’Variance components testing in the longitudinal mixed effects model’. Biometrics 51(3):1196

    Google Scholar 

  • Wolpert DH, Macready WG (1997) No free lunch theorems for optimization. IEEE Transactions on Evolutionary Computation 1(1):67–82

    Article  Google Scholar 

  • Zanakis SH (1977) Heuristic 0-1 linear programming: An experimental comparison of three methods. Management Science 24(1):91–104

    Article  MATH  MathSciNet  Google Scholar 

  • Zemel E (1981) Measuring the quality of approximate solutions to zero-one programming problems. Mathematics of Operations Research 6(3):319–332

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics and Computer Science, University of Southern Denmark, Odense, Denmark

    Marco Chiarandini & Yuri Goegebeur

  2. Research Group Quantitative Psychology and Individual Differences, K.U.Leuven, Belgium

    Yuri Goegebeur

Authors
  1. Marco Chiarandini
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuri Goegebeur
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Marco Chiarandini or Yuri Goegebeur .

Editor information

Editors and Affiliations

  1. Institute of Computer Science, Faculty of Computer Science, Cologne University of Applied Sciences, Steinmüllerallee 1, Gummersbach, 51643, Germany

    Thomas Bartz-Beielstein

  2. , Department of Mathematics, University of Southern Denmark, Campusvej 55, Odense, 5230, Denmark

    Marco Chiarandini

  3. CISUC, Department of Informatics Engineering, University of Coimbra, Coimbra, 3030-290, Portugal

    Luís Paquete

  4. Algorithm Engineering, TU Dortmund, Otto-Hahn-Str. 14, Dortmund, 44227, Germany

    Mike Preuss

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Chiarandini, M., Goegebeur, Y. (2010). Mixed Models for the Analysis of Optimization Algorithms. In: Bartz-Beielstein, T., Chiarandini, M., Paquete, L., Preuss, M. (eds) Experimental Methods for the Analysis of Optimization Algorithms. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-02538-9_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-02538-9_10

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-02537-2

  • Online ISBN: 978-3-642-02538-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation