Welcome to the InfoSci Platform
Reference Hub1
The Meaningfulness of Statistical Significance Tests in the Analysis of Simulation Results

The Meaningfulness of Statistical Significance Tests in the Analysis of Simulation Results

Klaus G. Troitzsch
Copyright: © 2016 |Volume: 8 |Issue: 1 |Pages: 28
ISSN: 1943-0744|EISSN: 1943-0752|EISBN13: 9781466690127|DOI: 10.4018/IJATS.2016010102
Cite Article Cite Article

MLA

Troitzsch, Klaus G. "The Meaningfulness of Statistical Significance Tests in the Analysis of Simulation Results." IJATS vol.8, no.1 2016: pp.18-45. http://doi.org/10.4018/IJATS.2016010102

APA

Troitzsch, K. G. (2016). The Meaningfulness of Statistical Significance Tests in the Analysis of Simulation Results. International Journal of Agent Technologies and Systems (IJATS), 8(1), 18-45. http://doi.org/10.4018/IJATS.2016010102

Chicago

Troitzsch, Klaus G. "The Meaningfulness of Statistical Significance Tests in the Analysis of Simulation Results," International Journal of Agent Technologies and Systems (IJATS) 8, no.1: 18-45. http://doi.org/10.4018/IJATS.2016010102

Export Reference

Mendeley
Favorite Full-Issue Download

Abstract

This article discusses the question of whether significance tests on simulation results are meaningful at all. It is also argued that it is the effect size much more than the existence of the effect is what matters. It is the description of the distribution function of the stochastic process incorporated in the simulation model which is important. This is particularly when this distribution is far from normal, which is particularly often the case when the simulation model is nonlinear. To this end, this article uses three different agent-based models to demonstrate that the effects of input parameters on output metrics can often be made “statistically significant” on any desired level by increasing the number of runs, even for negligible effect sizes. The examples are also used to give hints as to how many runs are necessary to estimate effect sizes and how the input parameters determine output metrics.

Request Access

You do not own this content. Please login to recommend this title to your institution's librarian or purchase it from the IGI Global bookstore.