Improvement of the Neural Network Trees through Fine-Tuning of the Threshold of Each Internal Node

Abstract

Neural network tree (NNTree) is a decision tree (DT) in which each internal node contains a neural network (NN). Experimental results show that the performance of the NNTrees is usually better than that of the traditional univariate DTs. In addition, the NNTrees are more usable than the single model fully connected NNs because their structures can be determined automatically in the induction process. Recently, we proposed an algorithm that can induce the NNTrees efficiently and effectively. In this paper, we propose to improve the performance of the NNTrees further through fine-tuning of the threshold of each internal node. Experimental results on several public databases show that, although the proposed method is very simple, the performance of the NNTrees can be improved in most cases, and in some cases, the improvement is even significant.