Kernel based gene expression pattern discovery and its application on cancer classification

Abstract

Association rules have been widely used in gene expression data analysis. However, there is no systematical way to select interesting rules from the millions of rules generated from high dimensional gene expression data. In this study, a kernel density estimation based measurement is proposed to evaluate the interestingness of the association rules. Several pruning strategies are also devised to efficiently discover the approximate top-k interesting patterns. Finally, over-fitting problem of the classification model is addressed by using conditional independence test to eliminate redundant rules. Experimental results show the effectiveness of the proposed interestingness measure and classification model.

Introduction

With the development of genomic techniques, research on molecular biology has shifted from individual genes to the entire genome. Microarray, SAGE are such techniques, which can measure the expression levels of thousands of genes in a single experiment. These techniques are very suitable for comparing the gene expression levels in tissues under different conditions, for example, healthy versus diseased. To accurately predict the stages of samples is of great interest for biomedical applications.

However, high dimensionality and a small number of noisy samples pose great challenges to the existing machine learning methods. The main approach to this problem is tailoring the existing algorithms to the needs of gene expression data analysis, for example, support vector machines (SVM) [1], [2], neural networks [3], logistic regression [4], GA/KNN [5] and between-group analysis [6]. However, almost all of those classifiers are ‘black-box’ and hard to interpret. Interpretation of classifier is an important task of gene expression classification which can help biologists gain insight into biological process and inspire further biological research.

Since the first work of Creighton [7], association rule based gene expression data analysis has received a lot of attention [8], [9], [10], because it is simple and easy to interpret. A typical association rule $\gamma$ is in the form: $\mathit{LHS}\to \mathit{RHS}$, where LHS is the antecedent and RHS is the consequent. Support and confidence are two commonly used interestingness measures of the rules. Support, $\mathit{sup}(\gamma )$, refers to the number of rows containing the item set $\mathit{LHS}\cup \mathit{RHS}$; confidence, $\mathit{conf}(\gamma )$, is computed as $\mathit{sup}(\gamma )/\mathit{sup}(\mathit{LHS})$.

In 1998, Liu [11] firstly used a special class of association rules, whose consequent is a single class label, to build rule based classifiers, and the results are promising. Since then, many different rule based classification models have been proposed. In 2004, Gao [8] successfully extend the rule based classifier to the high dimensional gene expression data, with the help of the proposed row-combination association rule mining algorithm. For example, $\mathit{CST}3\uparrow \to \mathit{AML}$ is a rule discovered from the Leukemia gene expression data, $\mathit{CST}3\uparrow$ (a gene expression pattern) is the LHS, and AML (a specified class of Leukemia) is RHS. By taking more characteristic of the gene expression data into consideration, results are further improved by Gao [9]. Georgii's Quantitative Association Rule [10] and Li's Emerging Patterns [12] are another publications on gene expression data.

Discretization is a necessary preprocess before conducting associate rule mining on gene expression data, which transforms continuous gene expression levels to categorical item sets. If the gene g's expression level is higher than the discretization threshold, the gene is considered as expressed and presented as $g\uparrow$; otherwise it is marked as repressed and denoted as $g\downarrow$. Take the Leukemia data set [13] for example, if the expression level of gene CST3 falls in $[1419.5,+\infty ]$, it is taken as expressed, otherwise if it falls in $[-\infty ,1419.5]$, it is taken as repressed.

However, a lot of information is lost in the above discretization procedure. Considering the following two expression levels of the gene CST3, one is 1, the other is 1419. According to the discretization procedure, CST3 is considered as repressed in both two samples. Though the two expression levels are quite different, both of them are considered repressed in the discretization procedure. Obviously a lot of information is lost in the above transformation, which is especially precious in the case of small sample gene expression data. Because of the information loss, millions of association rules have been reported with the same support and confidence [8], [9], which makes it is hard to discover the real interesting rules. But discretization is a necessary preprocess for association rule mining, and we cannot sacrifice interpretability for accuracy.

Fortunately, we find a new way to explore the original data: discover patterns in the discretized data, but evaluate them in the original data using the kernel density estimation method.

Fig. 1 illuminates the proposed kernel density estimation based approach. In the figure, pattern $g1\uparrow g2\uparrow$ and $g3\uparrow g4\uparrow$ are equivalent to the shadow subspace of Fig. 1(a) and (b), respectively. Thus, the support of $g1\uparrow g2\uparrow$ is corresponding to the cumulative probability that samples fall in the shadow region of Fig. 1(a). Here, we use the kernel density estimation framework to estimate the cumulative probability. According to the property of kernel density estimation and the position of the samples, we can get that the support of $g1\uparrow g2\uparrow$ is higher than that of $g3\uparrow g4\uparrow$, this is more reasonable than that $g1\uparrow g2\uparrow$ and $g3\uparrow g4\uparrow$ are with the same support. This concept is also closely related to the sub-space clustering algorithm [14].

Extended to association rule context, the association rule can be consider as a special item set with the target as a special item. Then, we use the similar kernel density estimation framework to estimate the support and confidence of association rules. According to the distribution of sample classes, $g1\uparrow g2\uparrow \to +$ is more interesting than $g3\uparrow g4\uparrow \to +$, which is in accordance with our intuition.

However, the new rule interestingness evaluation framework brings new challenges to the existing association rule mining methods. Firstly, it needs to estimate the joint probability density function of multi-genes’ expression levels. Secondly, the important downward closure property is not maintained in the proposed measure of support. For the first problem, kernel density estimation method is proposed to efficiently estimate the joint probability density function without any prior assumption of the distribution function. Moreover, cumulative density can be efficiently integrated under kernel density estimation framework. For the second challenge, we choose to mining approximate top-k interesting rules. Several pruning strategies are also devised to accelerate the mining procedure.

Serious over-fit phenomenon is another problem that need to be considered while constructing a rule based classifier on gene expression data. The main reason is that existing rule based classifiers are too much tailored to the training set by selecting rules to cover each training sample [8], [9]. In small sample size gene expression data, the full cover strategy raises very serious over-fit problem. In this study, we use conditional independence test to eliminate redundant rules, and alleviate this problem.