Evaluating switching neural networks through artificial and real gene expression data

Summary

Objective

DNA microarrays offer the possibility of analyzing the expression level for thousands of genes concerning a specific tissue. An important target of this analysis is to derive the subset of genes involved in a biological process of interest. Here, a new promising method for gene selection is proposed, which presents a good level of accuracy and reliability.

Methods and materials

The proposed technique adopts switching neural networks (SNN), a particular kind of connectionist models, to assign a relevance value to each gene, thus employing recursive feature addition (RFA) to derive the final list of relevant genes. To fairly evaluate the quality of the new approach, called SNN-RFA, its application on three real and three artificial gene expression datasets, generated according to a proper mathematical model that possesses biological and statistical plausibility, has been considered. In particular, a comparison with other two widely used gene selection methods, namely the signal to noise ratio (S2N) and support vector machines with recursive feature elimination (SVM-RFE), has been performed.

Results

In all the considered cases SNN-RFA achieves the best performances, arriving to determine the whole collection of relevant genes in one of the three artificial datasets. The S2N method exhibits a quality similar to that of SNN-RFA, whereas SVM-RFE shows the worst behavior.

Conclusion

The quality of the proposed method SNN-RFA has been established together with the usefulness of the mathematical model adopted to generate the artificial datasets of gene expression levels.

Introduction

DNA microarrays provide the gene expression level for thousands of genes pertaining to a given tissue, thus allowing to understand mechanisms regulating biological processes, such as the onset of a disease or the effects of a drug [1], [2], [3], [4], [5]. Nevertheless, treating such a huge amount of data requires appropriate statistic and information analysis tools. An important challenge in this analysis is to determine the subset of genes involved in the biological process under examination. Such problem is generally referred to as gene selection and several statistic and machine learning techniques have been proposed in literature to face with it [6], [7], [8], [9].

Golub et al. [10] have obtained interesting results in discriminating two different kinds of leukemia by adopting a simple univariate statistic method, involving a measure of the signal to noise ratio (S2N). This technique has been recently recognized as a particular member of a more general family of feature selection algorithms, denoted as BAHSIC (based on the Hilbert–Schmidt independence criterion) [11]. To the same family belong other interesting gene selection methods, such as the PAM technique [12] and algorithms relying on Pearson’s correlation [13], T-test [14], B-statistics [15].

An alternative approach has been proposed by Guyon et al. [16]: it is based on an iterative procedure, called recursive feature elimination (RFE), which subsequently removes genes marked as less relevant by a specific classifier. To this end, Guyon et al. decided to employ linear support vector machines (SVM), whose quality has been theoretically and experimentally demonstrated; the resulting gene selection procedure is usually referred to as SVM-RFE. Further refinements and modifications of SVM-RFE have been recently proposed [17], [18], [19]; moreover, the RFE approach has been adopted to rank relevant genes produced with other classification algorithms, such as the maximum margin criterion (MMC) [20].

Another promising class of machine learning techniques for gene selection is rule generation methods, which solve a classification problem by generating a collection of intelligible rules in the if-then form. In particular, switching neural networks (SNN) [21] have been shown to obtain an excellent accuracy, when applied to solve real world problems deriving from DNA microarray. This paper proposes to employ SNN for gene selection by adopting the opposite approach with respect to RFE: it subsequently adds the features considered as more relevant by a proper classifier. Since this approach is called recursive feature addition (RFA) [22], the proposed gene selection method will be denoted as SNN-RFA.

Unfortunately, real data cannot be adopted to evaluate in an objective way the quality of a gene selection method, such as S2N, SVM-RFE, or SNN-RFA. In fact, the whole set of genes really involved in a biological process is not known: medical and biological literature provide at most a partial knowledge about it.

To overcome this problem, the subset of genes found by the method at hand is usually considered for the construction of a classifier, whose accuracy provides a measure of the validity of the gene selection task. In fact, when redundant input variables are ignored, a better solution of a classification problem can be attained; therefore, the identification of a good subset of genes must lead to an improvement in the generalization ability of classifiers relying on that subset.

However, this approach for evaluating gene selection method is affected by the technique adopted for the construction of the classifier, as it is pointed out by results presented in many papers among which [19]. A valid alternative approach consists in using the biologically plausible mathematical model described in [23], which is able to generate artificial expression data that present the same statistic behavior as those deriving from DNA microarray experiments. In this case, the whole set of artificial genes involved in the construction of the examples is known, thus allowing a fair evaluation of different techniques for gene selection.

The quality of SNN-RFA is analyzed by considering three real problems involving microarray experiments, described in [10], [24], [25], together with three artificial datasets possessing similar statistic behavior. In particular, the results obtained by SNN-RFA are compared with those produced by S2N and SVM-RFE, by evaluating the subsets of common genes retrieved and by assessing the number of correct genes detected in the artificial cases.