An ensemble of RBF neural networks in decision tree structure with knowledge transferring to accelerate multi-classification

Abstract

This paper treats with a decision tree consisting of RBF neural networks in the nodes to decrease the classification time and to improve accuracy as well as generalization. We propose two knowledge transferring mechanisms between nodes to reduce the duplicate computations in training process. The resulted classifier is titled as ensemble of RBF neural networks in decision tree structure with knowledge transferring or shortly ERDK. We accelerate ERDK by applying a cut-point mechanism to prune its tree structure. The results on a great number of benchmark datasets show that ERDK provides promising results. Particularly, AUCarea, F-measure, and G-mean for ERDK are 95, 90, and 90%, respectively. As a case study, we finally apply ERDK on Britain incident dataset.

Appendices

Appendix A: Evaluation measures

There are different measures for performance analysis of classification algorithms. This subsection describes some of the popular ones used in this paper. Some of the useful parameters are as follows:

• True Positive (TP): number of positive classified data, which are positive in reality.

• True Negative (TN): number of negative classified data, which are negative in reality.

• False Positive (FP): number of positive classified data, which are negative in reality.

• False Negative (FN): number of negative classified data, which are positive in reality.

• Precision This measure is useful to compare the performances of different algorithms. It computes based on (11), see [16].

  $${\text{Percision}} = \frac{\text{TP}}{{\left( {{\text{TP}} + {\text{FP}}} \right)}}$$

  (11)

• Recall The recall measure computes based on (12), see [16].

  $${\text{Recall}} = \frac{\text{TP}}{{\left( {{\text{TP}} + {\text{FN}}} \right)}}$$

  (12)

• F-measure This measure is one of the other popular measures, which determines the performance of imbalanced data classification. F-measure computes based on (13), see [16].

  $$F{\text{-measure}} = \frac{{2 \times {\text{Percision}} \times {\text{Recall}}}}{{{\text{Percision}} + {\text{Recall}}}}$$

  (13)

• G-mean The G-mean is computed based on (14), see [16].

  $$G{\text{-mean}} = \left( {{\text{Percision}} \times {\text{Recall}}} \right)^{{{\raise0.7ex\hbox{$1$} \!\mathord{\left/ {\vphantom {1 2}}\right.\kern-0pt} \!\lower0.7ex\hbox{$2$}}}}$$

  (14)

• AUCarea In [15], a new measure named AUCarea is introduced. In AUCarea, all the AUC values are plotted in a polar, and, finally, the area, covered by these polar coordinates is computed. So an AUCarea could compute by (15).

  $${\text{AUCarea}} = \frac{{\frac{1}{2}\sin \left( {\frac{2\varPi }{q}} \right)\left( {\left( {\mathop \sum \nolimits_{i = 1}^{q - 1} r_{i} \times r_{i + 1} } \right) + \left( {r_{q} \times r_{1} } \right)} \right)}}{{\frac{1}{2}\sin \left( {\frac{2\varPi }{q}} \right) \times q}} = \frac{{\mathop \sum \nolimits_{i = 1}^{q - 1} \left( {r_{i} + r_{i + 1} } \right) + \left( {r_{q} \times r_{1} } \right)}}{q}$$

  (15)

  where $r_{i}$ is the AUC value of binary combinations of different classes, and computed based on Eq. (16).

  $$r_{i} = \frac{{{\text{Percision}} - {\text{Recall}} + 1}}{2}$$

  (16)

Appendix B: Dataset description

Number	Datasets	Number of classes	Number of samples	Number of features
1	Zoo	7	101	16
2	Yeast	10	1484	8
3	Aba	29	4177	8
4	Lym	4	148	18
5	Ecoli	4	358	7
6	Car	4	1728	6
7	Pen-digit	10	1100	16
8	Mf-mor	6	2000	6
9	Led	7	500	7
10	Wine	3	178	13
11	New-thyroid	3	215	5
12	HayesR	3	160	5
13	Satellitea	7	6435	4
14	Glass	7	214	10
15	Vehicle	4	946	18
16	Letter	26	20,000	16
17	Segment	7	2310	19
18	Iris	3	150	4
19	WDBC	2	569	30
20	Ionosphere	2	350	34
21	Breast	2	799	9
22	Heart	2	270	13
23	Hepatitis	2	155	19

1. ^aFeatures of 17th–20th, as recommended by the database designers and mentioned by authors of reference [16]