A short history of statistical association: From correlation to correspondence analysis to copulas

Abstract

We present in three parts different concepts of correlation and statistical association, with some historical notes, starting with Galton’s notion of correlation, subsequently improved by Pearson. Continuing in this first part, we discuss the correlation ratio, the intraclass correlation, multiple correlation, and redundancy analysis. Throughout we use the classic data set of Galton on the heights of parents and their children. In the second part we explain how these same data can be studied from a multivariate viewpoint, using canonical correlation analysis, Procrustes correlation and simple/multiple correspondence analysis. For correspondence analysis, we use the same data as categorized by Galton into intervals of heights for the parents and their children. In this part we also make an incursion into the continuous form of correspondence analysis. The third part is dedicated to bivariate distributions, where we give the main results of bivariate distributions with given marginals, commenting on the correlations of Spearman and Kendall. Seeing that a bivariate distribution can be generated using a copula, we fit Galton’s data to two copulas: the Gaussian copula and the copula which has the best fit.

Introduction

The very early history of Statistics goes back to ancient times, in the Greek, Roman and Arab civilizations, and mostly involves counting: counting the population, the extent of properties and wealth. As the motto goes: “Statisticians count!”. But trying to understand the interrelationship between statistical variables can perhaps be traced back to the early Egyptians with their 8th and 9th century Nilometers [67], which continuously measured the height of the Nile. These data allowed them to predict whether the harvest would be normal, or whether there would be extremes of drought or flooding, information that would determine the levels of taxation. An inherent association between the Nile’s height and expected tax income was thus assumed.

So how can the relationship between two or more variables be described and how can the strength of this association be measured? This paper presents a relaxed review of this fundamental problem of variable association in Statistics, interspersed with some historical details from the latter part of the 19th century onwards. There are three main parts: Section 2 deals with the concept of bivariate correlation and its origins in the work of Bravais and Galton, subsequently improved by Pearson, as well as many variations on this theme developed by other pioneering statisticians such as Fisher. Section 3 exposes the history and concepts underlying correlation between two or more sets of variables, with correspondence analysis as the special case for categorical variables. Section 4 describes the general modelling of bivariate association in the form of copulas. Throughout we will use the famous data [29], [30] on mothers’, fathers’, daughters’ and sons’ heights used by Francis Galton, the father of regression analysis.