Significance Testing: An Overview

Introduction

A significance test is a statistical procedure for testing a hypothesis based on experimental or observational data. Let, for example, $\bar{{X}}_{1}$ and $\bar{{X}}_{2}$ be the average scores obtained in two groups of randomly selected subjects and let μ ₁ and μ ₂ denote the corresponding population averages. The observed averages can be used to test the null hypothesis μ ₁ = μ ₂, which expresses the idea that both populations have equal average scores. A significant result occurs if $\bar{{X}}_{1}$ and $\bar{{X}}_{2}$ are very different from each other, because this contradicts or falsifies the null hypothesis. If the two group averages are similar to each other, the null hypothesis is not contradicted by the data. What exact values of the difference $\bar{{X}}_{1} -\bar{ {X}}_{2}$of the group averages are judged as significant depends on various elements. The variation of the scores between the subjects, for example, must be taken into account. This...