skip to main content
10.1145/2598394.2609850acmconferencesArticle/Chapter ViewAbstractPublication PagesgeccoConference Proceedingsconference-collections
technical-note

The executable experimental template pattern for the systematic comparison of metaheuristics: Extended Abstract

Published: 12 July 2014 Publication History
First page of PDF

References

[1]
A. Arcuri and L. Briand. A practical guide for using statistical tests to assess randomized algorithms in software engineering. In ACM/IEEE Int. Conf. on Software Engineering (ICSE), pages 1--10, 2011.
[2]
Andrea Arcuri and Lionel Briand. A hitchhiker's guide to statistical tests for assessing randomized algorithms in software engineering. Software Testing, Verification and Reliability, 2012.
[3]
Morton B. Brown and Alan B. Forsythe. Robust tests for the equality of variances. Journ. of the American Statistical Association, 69(346):364--367, 1974.
[4]
Edgar Brunner and Ullrich Munzel. The nonparametric behrens-fisher problem: Asymptotic theory and a small-sample approximation. Biometrical Journ., 42(1):17--25, 2000.
[5]
Norman Cliff. Dominance statistics: Ordinal analyses to answer ordinal questions. Psychological Bulletin, 114(3):494, 1993.
[6]
G. Kendall, G. Ochoa, E. Özcan, E. K. Burke, M. Hyde and J. R. Woodward. A Classification of Hyper-heuristic Approaches, chapter Handbook of Meta-Heuristics, pages 449--468. Kluwer, 2010.
[7]
Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides. Design patterns: elements of reusable object-oriented software. Pearson Education, 1994.
[8]
Greffenstette. Optimization of control parameters for genetic algorithms. In IEEE Trans. Syst. Man Cybern. SMC-16, volume 1, pages 122--128, Jan/Feb 1986.
[9]
Phil Souza, Jerffeson Teixeira de Harman, Mark McMinn and Shin Yoo. Search based software engineering: Techniques, taxonomy, tutorial. In Bertrand Meyer and Martin Nordio, editors, Empirical software engineering and verification: LASER 2009-2010, pages 1--59. Springer, 2012. LNCS 7007.
[10]
W. M. Hathaway. Automated hypothesis testing, February 5 2013. US Patent 8,370,107.
[11]
John N. Hooker. Needed: An empirical science of algorithms. Operations Research, 42(2):201--212, 1994.
[12]
Sean Luke. Essentials of Metaheuristics. Lulu, second edition, 2013. Available for free at http://cs.gmu.edu/~sean/book/metaheuristics/.
[13]
Henry B. Mann and Donald R. Whitney. On a test of whether one of two random variables is stochastically larger than the other. The annals of mathematical statistics, 18(1):50--60, 1947.
[14]
J. Neyman and E. S. Pearson. On the Use and Interpretation of Certain Test Criteria for Purposes of Statistical Inference: Part I. Biometrika, 20A(1/2):175--240, July 1928.
[15]
R. Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria, 2013.
[16]
Martin B. Shapiro, Samuel S. Wilk and Hwei J. Chen. A comparative study of various tests for normality. Journ. of the American Statistical Association, 63(324):1343--1372, 1968.
[17]
Samuel S. Shapiro and R. S. Francia. An approximate analysis of variance test for normality. Journ. of the American Statistical Association, 67(337):215--216, 1972.
[18]
Tiantian Wang, Mark Harman, Yue Jia, and Jens Krinke. Searching for better configurations: a rigorous approach to clone evaluation. In European Software Engineering Conf. and the ACM SIGSOFT Symposium on the Foundations of Software Engineering, ESEC/FSE'13, Saint Petersburg, Russian Federation, August 2013. ACM.

Cited By

View all
  • (2022)QUAntum Particle Swarm Optimization: an auto-adaptive PSO for local and global optimizationComputational Optimization and Applications10.1007/s10589-022-00362-282:2(525-559)Online publication date: 27-Apr-2022
  • (2019)Nature-Inspired Algorithms for Bi-Criteria Parallel Machine SchedulingExploring Critical Approaches of Evolutionary Computation10.4018/978-1-5225-5832-3.ch007(122-148)Online publication date: 2019
  • (2019)A Systematic Review of Interaction in Search-Based Software EngineeringIEEE Transactions on Software Engineering10.1109/TSE.2018.280305545:8(760-781)Online publication date: 1-Aug-2019
  • Show More Cited By

Index Terms

  1. The executable experimental template pattern for the systematic comparison of metaheuristics: Extended Abstract

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    GECCO Comp '14: Proceedings of the Companion Publication of the 2014 Annual Conference on Genetic and Evolutionary Computation
    July 2014
    1524 pages
    ISBN:9781450328814
    DOI:10.1145/2598394
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 12 July 2014

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. effect size
    2. hypothesis testing
    3. statistical significance

    Qualifiers

    • Technical-note

    Conference

    GECCO '14
    Sponsor:
    GECCO '14: Genetic and Evolutionary Computation Conference
    July 12 - 16, 2014
    BC, Vancouver, Canada

    Acceptance Rates

    GECCO Comp '14 Paper Acceptance Rate 180 of 544 submissions, 33%;
    Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)3
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 20 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2022)QUAntum Particle Swarm Optimization: an auto-adaptive PSO for local and global optimizationComputational Optimization and Applications10.1007/s10589-022-00362-282:2(525-559)Online publication date: 27-Apr-2022
    • (2019)Nature-Inspired Algorithms for Bi-Criteria Parallel Machine SchedulingExploring Critical Approaches of Evolutionary Computation10.4018/978-1-5225-5832-3.ch007(122-148)Online publication date: 2019
    • (2019)A Systematic Review of Interaction in Search-Based Software EngineeringIEEE Transactions on Software Engineering10.1109/TSE.2018.280305545:8(760-781)Online publication date: 1-Aug-2019
    • (2018)Metaheuristic Design PatternsHandbook of Research on Emergent Applications of Optimization Algorithms10.4018/978-1-5225-2990-3.ch001(1-36)Online publication date: 2018
    • (2017)Search-Based Energy Optimization of Some Ubiquitous AlgorithmsIEEE Transactions on Emerging Topics in Computational Intelligence10.1109/TETCI.2017.26991931:3(188-201)Online publication date: Jun-2017
    • (2015)Multi-objective Module Clustering for KateSearch-Based Software Engineering10.1007/978-3-319-22183-0_24(282-288)Online publication date: 28-Jul-2015
    • (2015)Specialising Guava’s Cache to Reduce Energy ConsumptionSearch-Based Software Engineering10.1007/978-3-319-22183-0_23(276-281)Online publication date: 28-Jul-2015
    • (2015)Object-Oriented Genetic Improvement for Improved Energy Consumption in Google GuavaSearch-Based Software Engineering10.1007/978-3-319-22183-0_20(255-261)Online publication date: 28-Jul-2015

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media