CLT for spiked eigenvalues of a sample covariance matrix from high-dimensional Gaussian mean mixtures

Abstract

Sample covariance matrices from a finite mean mixture model naturally carry certain spiked eigenvalues, which are generated by the differences among the mean vectors. However, their asymptotic behaviors remain largely unknown when the population dimension $p$ grows proportionally to the sample size $n$. In this paper, a new CLT is established for the spiked eigenvalues by considering a Gaussian mean mixture in such high-dimensional asymptotic frameworks. It shows that the convergence rate of these eigenvalues is $O(1/\sqrt{n})$ and their fluctuations can be characterized by the mixing proportions, the eigenvalues of the common covariance matrix, and the inner products between the mean vectors and the eigenvectors of the covariance matrix.

Introduction

Spiked population model originally introduced in [16] assumes that the covariance matrix $\mathit{\Sigma }$ of a population $\mathbf{x}\in {\mathbb{R}}^p$ is a finite-rank perturbation of a base matrix. For the simplest case where the base matrix is an identity and the perturbation matrix is nonnegative, the eigenvalues of $\mathit{\Sigma }$ can be grouped into two separated classes

$$\mathrm{Spec}\left(\mathit{\Sigma }\right)=\left(\underset{M}{\underbrace{{\alpha }_1,\dots ,{\alpha }_M}},\underset{p-M}{\underbrace{1,\dots ,1}}\right).$$

Here the top $M$ eigenvalues ${\alpha }_1\ge \cdots \ge {\alpha }_M>1$ are called spiked eigenvalues of $\mathit{\Sigma }$. These spikes often carry a wealth of information about the dependence among the components of $\mathbf{x}$ and can be inferred from their empirical counterparts, say the eigenvalues of a sample covariance matrix ${\mathbf{S}}_n$. However, in high-dimensional situations where the dimension $p$ is non-negligible with respect to the sample size $n$, it is well known that the eigenvalues of ${\mathbf{S}}_n$ deviate from their population ones in a subtle manner. See [3], [19], [22], etc.

Let us consider the widely used independent components (IC) model [3] for the population $\mathbf{x}$, admitting the following stochastic representation

$$\mathbf{x}=\mathit{\mu }+{\mathit{\Sigma }}^{\frac{1}{2}}\mathbf{z},$$

where $\mathit{\mu }\in {\mathbb{R}}^p$ denotes the population mean and $\mathbf{z}\in {\mathbb{R}}^p$ is a vector of independent and standardized random variables. Let ${\mathbf{x}}_1,\dots ,{\mathbf{x}}_n$ be $n$ independent and identically distributed (i.i.d.) observations from this population. The sample covariance matrix is

$${\mathbf{S}}_n=\frac{1}{n-1}\sum _{i=1}^n({\mathbf{x}}_i-\bar{\mathbf{x}}){({\mathbf{x}}_i-\bar{\mathbf{x}})}^{\top },\bar{\mathbf{x}}=\frac{1}{n}\sum _{i=1}^n{\mathbf{x}}_i,$$

the eigenvalues of which are denote as $\left\{{\lambda }_{\ell }\right\}$, arranged in descending order and referred to as sample eigenvalues. These eigenvalues have been well studied in the so-called Marčenko–Pastur (MP) asymptotic regime [19] where

$$n\to \infty ,p=p_n\to \infty ,c_n\triangleq p/n\to c\in (0,\infty ).$$

In particular, under the spiked population model (1) with $\left({\alpha }_{\ell }\right)$ pair-wise different, the $\ell$-th ($1\le \ell \le M$) largest eigenvalue ${\lambda }_{\ell }$ of ${\mathbf{S}}_n$ converges to ${\alpha }_{\ell }+c{\alpha }_{\ell }/({\alpha }_{\ell }-1)$ with a Gaussian fluctuation if ${\alpha }_{\ell }$ is larger than the critical value $1+\sqrt{c}$, otherwise to ${(1+\sqrt{c})}^2$ the right edge point of the MP law with a Tracy–Widom fluctuation, see [5], [7], [16], [21]. Some extensions of these results can be found in [6], [14], where the base matrix can be general having a non-degenerate limiting spectral distribution. In addition to these literatures focusing on the sample covariance matrix, the spiked population model has also been adopted in the study of sample canonical correlation matrix, Fisher matrix, and separable covariance matrix. See [8], [13], [15], [23].

Most recently, [18] considered a mean mixture (MM) model when investigating the problem of high-dimensional clustering. An interesting phenomenon is that a sample covariance matrix from such mixture inherently carries some spiked eigenvalues. To fully understand such phenomenon, let us consider an MM $\mathbf{x}$ of $\tau$ subpopulations $\left\{G_i,i\in \{1\dots ,\tau \}\right\}$ with mean vectors $\left\{{\mathit{\mu }}_i,i\in \{1\dots ,\tau \}\right\}$ and a common covariance matrix ${\mathit{\Sigma }}_0$, that is,

$$\mathbf{x}|G_i={\mathit{\mu }}_i+{\mathit{\Sigma }}_0^{\frac{1}{2}}\mathbf{z},i\in \{1\dots ,\tau \},$$

where $\{\mathbf{x}\mid G_i\}$ denote the $\tau$ subpopulations and $\mathbf{z}\in {\mathbb{R}}^p$ is a random vector similarly defined as in the IC model (2). Let $\left\{{\mathbf{x}}_j\right\}$ be $n$ observations from this mixture and denote by $\left\{n_i,i\in \{1\dots ,\tau \}\right\}$ the (unknown) sizes of samples from each of the subpopulations. Then, given these sizes, the expectation of the sample covariance matrix is

$${\mathit{\Sigma }}_n\triangleq \mathrm{E}\left({\mathbf{S}}_n\right)={\mathit{\Sigma }}_0+\frac{1}{n(n-1)}\sum _{1\le i<j\le \tau }{n}_i{n}_j\left({\mathit{\mu }}_i-{\mathit{\mu }}_j\right){\left({\mathit{\mu }}_i-{\mathit{\mu }}_j\right)}^{\top }.$$

Clearly, this covariance matrix ${\mathit{\Sigma }}_n$ is a sum of the base matrix ${\mathit{\Sigma }}_0$ and a finite rank perturbation matrix, which forms a spiked population model and the spikes are generated by the differences among the $\tau$ subpopulation means. The main results in [18] imply that, in the MP asymptotic regime (3), spiked eigenvalues of ${\mathbf{S}}_n$ from the MM model converge almost surely to some limits in the same way as that already established under the IC model (2) with covariance matrix ${\mathit{\Sigma }}_n$.

This paper takes a step further to investigate the second order limits of these spiked sample eigenvalues. A Gaussian mean mixture (GMM) population is considered in our study, i.e., the MM model (4) with standard Gaussian vector $\mathbf{z}\sim N(\mathbf{0},{\mathbf{I}}_p)$. This mixture is one of fundamental probabilistic models in statistical inference and has a wide range of applications, such as in pattern recognition, image processing and unsupervised machine learning. Compared with the IC model (2), the main difference of the GMM lies in that the observations are heterogeneous in conditional mean, given the group information. Hence our main task here is to quantify the effect of such heterogeneity on the fluctuation of spiked sample eigenvalues.

The main contribution of this paper is a new central limit theorem (CLT) for sample spikes under the GMM model. Our results show that these sample eigenvalues asymptotically act as the eigenvalues of certain Gaussian random matrix, the distribution of which is jointly determined by the mixing proportions $\{n_i/n\}$, the mean differences $\{{\mathit{\mu }}_i-{\mathit{\mu }}_j\}$ and the common covariance matrix ${\mathit{\Sigma }}_0$ of subpopulations. In particular, the inner products between the means $\left\{{\mathit{\mu }}_j\right\}$ and the eigenvectors of ${\mathit{\Sigma }}_0$ play important roles in the fluctuation of the spikes. As these determining factors are not functions of ${\mathit{\Sigma }}_n$, our CLT is essentially different from the case under the IC model as illustrated in [5], [6].

The rest of the paper is organized as follows. Section 2 details our model and assumptions. Section 3 presents our new CLT for spiked sample eigenvalues. Technical proofs are presented in Section 4.