Skip to main content
SpringerLink
Log in

How to Create Generalizable Results

  • Chapter
Book cover Springer Handbook of Computational Intelligence

Part of the book series: Springer Handbooks ((SHB))

Abstract

Basically, this chapter tries to find answers for the following fundamental questions in experimental research.

  1. (Q-1)

    How can problem instances be generated?

  2. (Q-2)

    How can experimental results be generalized?

The chapter is structured as follows. Section 56.2 introduces real-world and artificial optimization problems. Algorithms are described in Sect. 56.3. Objective functions and statistical models are introduced in Sect. 56.4; these models take problem and algorithm features into consideration. Section 56.5 presents case studies that illustrate our methodology. The chapter closes with a summary and an outlook.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 269.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 349.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Abbreviations

ANOVA:

analysis of variance

CI:

computational intelligence

EA:

evolutionary algorithm

ES:

evolution strategy

FX:

foreign exchange

i.i.d.:

independent, identically distributed

MAMP:

multiple algorithms, multiple problems

MAMS:

multiple algorithms and multiple problem instances

MASP:

multiple algorithms and one single problem

MSG:

max-set of Gaussian landscape generator

Q–Q:

quantile–quantile

REML:

restricted maximum likelihood estimator

SAMP:

one single algorithm and multiple problems

SASP:

one single algorithm and one single problem

SPOT:

sequential parameter optimization toolbox

US  EPA:

United States Environmental Protection Agency

References

  1. P.J. Brockwell, R.A. Davis: Introduction to Time Series and Forecasting (Springer, New York 2002)

    Book  MATH  Google Scholar 

  2. D.C. Montgomery: Design and Analysis of Experiments (Wiley, New York 2001)

    Google Scholar 

  3. M. Chiarandini, Y. Goegebeur: Mixed models for the analysis of optimization algorithms. In: Experimental Methods for the Analysis of Optimization Algorithms, ed. by T. Bartz-Beielstein, M. Chiarandini, L. Paquete, M. Preuss (Springer, Berlin Heidelberg 2010) pp. 225–264, preliminary version available as Tech. Rep. DMF-2009-07-001 at the The Danish Mathematical Society

    Chapter  Google Scholar 

  4. C.C. McGeoch: Toward an experimental method for algorithm simulation, INFORMS J. Comput. 8(1), 1–15 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  5. M. Birattari: On the Estimation of the Expected Performance of a Metaheuristic on a Class of Instances 2004), Tech. Rep. (IRIDIA, Bruxelles 2004)

    Google Scholar 

  6. M. Gallagher, B. Yuan: A general-purpose tunable landscape generator, IEEE Trans. Evol. Comput. 10(5), 590–603 (2006)

    Article  Google Scholar 

  7. I. Castiñeiras, M.D. Cauwer, B. O'Sullivan: Weibull-based benchmarks for bin packing, Lect. Notes Comput. Sci. 7514, 207–222 (2012)

    Article  Google Scholar 

  8. G.E.P. Box, G.M. Jenkins, G.C. Reinsel: Time Series Analysis, Forecasting and Control (Holden-Day, San Francisco 1976)

    MATH  Google Scholar 

  9. T. Bartz-Beielstein: Beyond Particular Problem Instances: How to Create Meaningful and Generalizable Results, Tech. Rep. TR 03/2012 (Cologne University of Applied Sciences, 2012)

    Google Scholar 

  10. T. Bartz-Beielstein, M. Friese, B. Naujoks, M. Zaefferer: SPOT applied to non-stochastic optimization problems – An experimental study, Genet. Evol. Comput. Conf. (GECCO 2012) (ACM, Philadelphia 2012)

    Google Scholar 

  11. M. Zaefferer: Optimization and Empirical Analysis of an Event Detection Software for Water Quality Monitoring, Master Thesis (University of Applied Sciences, Cologne 2012)

    Google Scholar 

  12. O. Flasch, T. Bartz-Beielstein, D. Bicker, W. Kantschik, C. von Strachwitz: Results of the GECCO 2011 Industrial Challenge: Optimizing Foreign Exchange Trading Strategies. CIOP Tech. Rep. 10/11, Res. Center CIOP (Cologne University of Applied Science, Cologne 2011)

    Google Scholar 

  13. E. Vladislavleva: Model–based Problem Solving through Symbolic Regression via Pareto Genetic Programming, Ph.D. Thesis (Tilburg University, Tilburg 2008)

    Google Scholar 

  14. H.-G. Beyer, H.-P. Schwefel: Evolution strategies – A comprehensive introduction, Nat. Comput. 1, 3–52 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  15. H.-P. Schwefel: Numerical Optimization of Computer Models (Wiley, Chichester 1981)

    MATH  Google Scholar 

  16. T. Bartz-Beielstein: Experimental Research in Evolutionary Computation – The New Experimentalism, Natural Computing Series (Springer, Berlin, Heidelberg, New York 2006)

    MATH  Google Scholar 

  17. T. Bartz-Beielstein, M. Preuss: Automatic and interactive tuning of algorithms, Proc. 13th Annu. Conf. Companion Genet. Evol. Comput. (GECCO) (ACM, New York 2011) pp. 1361–1380

    Google Scholar 

  18. J. Pinheiro, D. Bates: Mixed-Effects Models in S and S-PLUS (Springer, New York 2000)

    Book  MATH  Google Scholar 

  19. T. Bartz-Beielstein, M. Zaefferer: A Gentle Introduction to Sequential Parameter Optimization, Tech. Rep. TR 01/2012 CIplus Bd. 1/2012 (Cologne University of Applied Sciences, Cologne 2012)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Computer Science and Engineering Science, Cologne University of Applied Sciences, Steinmüllerallee 1, 51643, Gummersbach, Germany

    Thomas Bartz-Beielstein

Authors
  1. Thomas Bartz-Beielstein
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Thomas Bartz-Beielstein .

Editor information

Editors and Affiliations

  1. Systems Research Inst., Polish Academy of Sciences, ul. Newelska 6, 01-447, Warsaw, Poland

    Janusz Kacprzyk

  2. Dep. Electrical and Computer Engineering, University of Alberta, 116 Street 9107, T6J 2V4, Edmonton, Alberta, Canada

    Witold Pedrycz

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Bartz-Beielstein, T. (2015). How to Create Generalizable Results. In: Kacprzyk, J., Pedrycz, W. (eds) Springer Handbook of Computational Intelligence. Springer Handbooks. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-43505-2_56

Download citation

  • DOI: https://doi.org/10.1007/978-3-662-43505-2_56

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-662-43504-5

  • Online ISBN: 978-3-662-43505-2

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation