A Simplex Design of Linear Hyperplane Decision Networks

Abstract

A new type of two-layer aritficial neural network is presented. In contrast to its conventional counterpart, the new network is capable of forming any desired decision region in the input space. The new network consists of a conventional input (hidden) layer and an output layer that is, in essence, a Boolean function of the hidden layer output. Each node of the hidden layer forms a decision hyperplane and the set of hyperplanes constituted by the hidden layers perform a partition of the input space into a number of cells. These cells can be labeled according to their location (binary code) relative to all hyperplanes. The sole function of the output layer is then to group all these cells together appropriately, to form an arbitrary decision region in the input space. In conventional approaches, this is accomplished by a linear combination of the binary decisions (outputs) of the nodes of the hidden layer. This traditional approach is very limited concerning the possible shape of the decision region. A much more natural approach is to form the decision region as a Boolean function of the biniary hidden layer “word” which has as many digits as there are nodes in the hidden layer. The training procedure of the new network can be split in two completely decoupled phases. First, the construction of the decision hyperplanes formed by the hidden layer can be posed as a linear programming problem which can be solved using the Simplex algorithm. Moreover, a fast algorithm for approximate solution of this linear programming problem based on an elliptic approximation of the Simplex can be devised. The second step in the design of the network is the appropriate construction of the Boolean function of the output layer. The key trick here is the adequate incorporation of don’t cares in the Boolean function so that the decision regions formed by the output layer cover the entire input space but are still not overlapping.