Use of SVD-based probit transformation in clustering gene expression profiles

Abstract

The mixture-Gaussian model-based clustering method has received much attention in clustering gene expression profiles in the literature of bioinformatics. However, this method suffers from two difficulties in applications. The first one is on the parameter estimation, which becomes difficult when the dimension of the data is high or the size of a cluster is small. The second one is on the normality assumption for gene expression levels, which is seldom satisfied by real data. In this paper, we propose to overcome these two difficulties by the probit transformation in conjunction with the singular value decomposition (SVD). SVD reduces the dimensionality of the data, and the probit transformation converts the scaled eigensamples, which can be interpreted as correlation coefficients as explained in the text, into Gaussian random variables. Our numerical results show that the SVD-based probit transformation enhances the ability of the mixture-Gaussian model-based clustering method for identifying prominent patterns of the data. As a by-product, we show that the SVD-based probit transformation also improves the performance of the model-free clustering methods, such as hierarchical, $K$-means and self-organizing maps (SOM), for the data sets containing scattered genes. In this paper, we also propose a run test-based rule for selection of eigensamples used for clustering.

Introduction

In attempts to understand biological systems, large amount of gene expression data have been generated by researchers. Due to the large number of genes and the complexity of the biological systems, clustering has been one of the most important exploratory tools for analyzing these data. Clustering identifies groups of genes that exhibit similar expression profiles. The popular clustering methods can be divided into two categories, namely, model-free methods and model-based methods. In the model-free clustering methods, no probabilistic model is specified for the data, and clustering is made either by optimizing a certain target function or iteratively agglomerating/dividing genes to form a bottom-up/top-down tree. Examples include $K$-means (Tavazoie et al., 1999, Tou and Gonzalez, 1979), hierarchical clustering (Carr et al., 1997, Eisen et al., 1998), self-organizing maps (Tamayo et al., 1999), among others. The model-based clustering methods construct clusters based on the assumption that the data follows a mixture distribution. A non-exhaustive list of recent works in this direction include Banfield and Raftery (1993), Biernacki et al. (1999), Fraley and Raftery (2002), Yeung et al. (2001), Medvedovic and Sivaganesan (2002), McLachlan et al. (2002), Wakefield et al. (2003) and Medvedovic et al. (2004). One advantage of the model-based clustering methods is that the probability model can be used in criteria to choose an appropriate number of clusters.

Among the model-based clustering methods, the one based on the mixture-Gaussian distribution is much more interesting due to its simplicity in computation. Henceforth, the mixture-Gaussian model-based clustering method will be abbreviated as the MG method. The MG method assumes that the observations ${\mathit{x}}_1,\dots ,{\mathit{x}}_n$ are generated from a mixture Gaussian distribution with an unknown number of components. The corresponding likelihood function is

$$L({\mathit{x}}_1,\dots ,{\mathit{x}}_n|{\omega }_k,{\mu }_k,{\Sigma }_k,k=1,\dots ,G)=\prod _{i=1}^n\left[\sum _{k=1}^G\frac{{\omega }_k}{(2\pi {)}^{p/2}|{\Sigma }_k{|}^{1/2}}{\mathrm{e}}^{-1/2({\mathit{x}}_i-{\mathit{\mu }}_k{)}^{\mathrm{T}}{\Sigma }_k^{-1}({\mathit{x}}_i-{\mathit{\mu }}_k)}\right]\text{,}$$

where G is the (unknown) number of components, p is the dimension of the observations, ${\omega }_k$ is the probability that an observation belongs to component k (${\omega }_k⩾0$ and ${\sum }_{k=1}^m{\omega }_k=1$), and ${\mathit{\mu }}_k$ and ${\Sigma }_k$ are the mean vector and covariance matrix of component k, respectively. Banfield and Raftery (1993) proposed to reparametrize the covariance matrices by eigenvalue decomposition in the form:

$${\Sigma }_k={\lambda }_k{D}_k{A}_k{D}_k^{\mathrm{T}}\text{,}$$

where ${\lambda }_k=|{\Sigma }_k{|}^{1/d}$, $D_k$ is the matrix of eigenvectors of ${\Sigma }_k$, and $A_k$ is a diagonal matrix such that $\left|A_k\right|=1$. The parameter ${\lambda }_k$ determines the volume of component k, $D_k$ determines its orientation and $A_k$ its shape. Allowing some but not all of these quantities to vary between components results in a set of parsimonious models which are appropriate to describe various clustering situations. Fraley and Raftery (2002) considered 10 different models related to different assumptions on the component variance matrices. Each model is denoted by a string of three letters E (equal), I (identical) and V (variable). The first letter of the string states the assumption on the volumes of the clusters, the second letter on the shapes, and the third letter on the orientation. For example, VEI represents a model in which the volumes of clusters may vary (V), the shapes of all clusters are equal (E), and the orientation is the identity (I). The MG method is implemented in the software MCLUST, which is downloadable at http://www.stat.washington.edu/mclust. In MCLUST, the model parameters are estimated using the EM algorithm (Dempster et al., 1977), and the BIC criterion is adopted for determining the number of clusters and the covariance structure.

Although the MG method has achieved great successes in clustering gene expression profiles (Yeung et al., 2001), its applications may be seriously hindered by the following two difficulties. The first one is on the parameter estimation, which becomes difficult when the dimension of the data is high or the size of a cluster is small. The second one is on the validity of the normality assumption. This assumption is seldom satisfied by real data. Applying the MG method to a data set with distribution deviated from Gaussian will result in a sub-optimal clustering result.

In this paper, we propose to overcome the above two difficulties by a SVD-based probit transformation. SVD reduces the dimension of the observations, and the probit transformation converts the scaled eigensamples, which can be interpreted as correlation coefficients as explained below, into Gaussian random variables. Our numerical results show that the transformation enhances the ability of the MG method for identifying prominent patterns of the data. In this paper, we also propose a run test-based rule for selection of eigensamples used for clustering. The new rule works well for all examples studied in this paper. Although the main theme of this paper is to show that the SVD-based probit transformation generally improves the performance of the MG method in clustering gene expression profiles, as a by-product we show that the transformation also improves the performance of the model-free clustering methods for the data sets containing scattered genes (Tseng and Wong, 2005).

The remaining part of this paper is organized as follows. In Section 2, we describe the SVD-based probit transformation and illustrate it using two motivating examples. In Section 3, we test the performance of the transformation on three simulated examples. In Section 4, we apply the transformation to a real data example. In Section 5, we conclude the paper with a brief discussion.