An improved neural classification network for the two-group problem

doi:10.1016/S0305-0548(98)00076-8

Computers & Operations Research

Volume 26, Issue 5, April 1999, Pages 443-460

https://doi.org/10.1016/S0305-0548(98)00076-8 Get rights and content

Abstract

In this paper we present the neural network model known as the mixture-of-experts (MOE) and determine its accuracy and its robustness. We do this by comparing the classification accuracy of MOE, backpropagation neural network (BPN), Fisher’s discriminant analysis, logistics regression, k nearest neighbor, and the kernel density on five real-world two-group data sets. Our results lead to three major conclusions: (1) the MOE network architecture is more accurate than BPN; (2) MOE tends to be more accurate than the parametric and non-parametric methods investigated; (3) MOE is a far more robust classifier than the other methods for the two-group problem.

Scope and purpose

The two-group classification problem is the assignment of objects to one of two predetermined groups. These classification decisions are routinely made in business, health care, and government. Classification errors result when an object is assigned to the wrong group. A healthy patient might be diagnosed with cancer, or a patient suffering from cancer might be diagnosed as healthy. A firm assigned the wrong bond rating will incur additional cost of capital of millions of dollars. Unfortunately, there is no classification method that is the “best” for all data sets. Some classification methods will do well for a given data set only to perform poorly on another. What is needed, therefore, is a robust methodology that classifies accurately across a wide range of two-group data sets. Researchers and practitioners have turned to neural classification models in a quest for a robust method. The purpose of this research is to investigate the accuracy and robust nature of the mixture-of-experts neural network (MOE). Our research demonstrates that MOE performs very well against back-propagation neural networks and four traditional statistical classification procedures based upon results from five diverse real world data sets. We conclude that the mixture-of-experts neural network may be the robust classification scheme being sought for use with the two-group problem.

References (0)

Cited by (28)

Comparing performances of backpropagation and genetic algorithms in the data classification
2011, Expert Systems with Applications
Citation Excerpt :
In these mathematical programming techniques, minimization of the sum of deviations approaches, maximum of the minimum deviation approaches, goal programming approaches, mixed-integer programming approaches and hyper-box representation approaches are the most commonly used optimization approaches. Artificial neural networks (ANNs) have popularity in solving several business and technical problems that involve prediction, and have also a wide ranging usage area in the classification problems (Denton, Hung, & Osyk, 1990; Holmstrom, Koistinen, Laaksonen, & Oja, 1997; Mangiameli & West, 1999; Patwo, Hu, & Hung, 1993; Pendharkar, 2005; Yim & Mitchell, 2005). One of the important issues on the neural networks is training of the networks.
Artificial neural networks (ANN) have a wide ranging usage area in the data classification problems. Backpropagation algorithm is classical technique used in the training of the artificial neural networks. Since this algorithm has many disadvantages, the training of the neural networks has been implemented with the binary and real-coded genetic algorithms. These algorithms can be used for the solutions of the classification problems. The real-coded genetic algorithm has been compared with other training methods in the few works. It is known that the comparison of the approaches is as important as proposing a new classification approach. For this reason, in this study, a large-scale comparison of performances of the neural network training methods is examined on the data classification datasets. The experimental comparison contains different real classification data taken from the literature and a simulation study. A comparative analysis on the real data sets and simulation data shows that the real-coded genetic algorithm may offer efficient alternative to traditional training methods for the classification problem.
Wavelet/mixture of experts network structure for EEG signals classification
2008, Expert Systems with Applications
Mixture of experts (ME) is a modular neural network architecture for supervised learning. This paper illustrates the use of ME network structure to guide model selection for classification of electroencephalogram (EEG) signals. Expectation-maximization (EM) algorithm was used for training the ME so that the learning process is decoupled in a manner that fits well with the modular structure. The EEG signals were decomposed into time–frequency representations using discrete wavelet transform and statistical features were calculated to depict their distribution. The ME network structure was implemented for classification of the EEG signals using the statistical features as inputs. To improve classification accuracy, the outputs of expert networks were combined by a gating network simultaneously trained in order to stochastically select the expert that is performing the best at solving the problem. Three types of EEG signals (EEG signals recorded from healthy volunteers with eyes open, epilepsy patients in the epileptogenic zone during a seizure-free interval, and epilepsy patients during epileptic seizures) were classified with the accuracy of 93.17% by the ME network structure. The ME network structure achieved accuracy rates which were higher than that of the stand-alone neural network models.
Statistics over features for internal carotid arterial disorders detection
2008, Computers in Biology and Medicine
Citation Excerpt :
The outputs of expert networks are combined by a gating network simultaneously trained in order to stochastically select the expert that is performing the best at solving the problem [2,3]. As pointed out by Jordan and Jacobs [4], the gating network performs a typical multiclass classification task [5–7]. Although the ME architecture has been successfully applied to several supervised learning tasks, it can only use a composite feature for classification with diverse features, since both gating and expert networks need to receive the same input.
The objective of the present study is to extract the representative features of the internal carotid arterial (ICA) Doppler ultrasound signals and to present the accurate classification model. This paper presented the usage of statistics over the set of the extracted features (Lyapunov exponents and the power levels of the power spectral density estimates obtained by the eigenvector methods) in order to reduce the dimensionality of the extracted feature vectors. Since classification is more accurate when the pattern is simplified through representation by important features, feature extraction and selection play an important role in classifying systems such as neural networks. Mixture of experts (ME) and modified mixture of experts (MME) architectures were formulated and used as basis for detection of arterial disorders. Three types of ICA Doppler signals (Doppler signals recorded from healthy subjects, subjects having stenosis, and subjects having occlusion) were classified. The classification results confirmed that the proposed ME and MME has potential in detecting the arterial disorders.
Detecting variabilities of Doppler ultrasound signals by a modified mixture of experts with diverse features
2008, Digital Signal Processing: A Review Journal
This paper illustrates the use of modified mixture of experts (MME) network structure to guide model selection for classification of Doppler ultrasound signals with diverse features. The MME is a modular neural network architecture for supervised learning. Expectation–maximization (EM) algorithm was used for training the MME so that the learning process is decoupled in a manner that fits well with the modular structure. In practical applications of pattern recognition, there are often diverse features extracted from raw data which needs recognizing. Decision making was performed in two stages: feature extraction by spectral analysis methods (fast Fourier transform and model-based methods) and classification using the classifiers trained on the extracted features. In order to discriminate the Doppler ultrasound signals, the ability of designed and trained MME network structure combined with spectral analysis methods was explored. The MME achieved accuracy rates which were higher than that of the mixture of experts (ME) and feedforward neural network models (multilayer perceptron neural network—MLPNN). The proposed MME approach can be useful in classifying the Doppler ultrasound signals for early detection of arterial diseases.
Expert systems for time-varying biomedical signals using eigenvector methods
2007, Expert Systems with Applications
In this paper, we present the expert systems for time-varying biomedical signals classification and determine their accuracies. The combined neural network (CNN), mixture of experts (ME), and modified mixture of experts (MME) were tested and benchmarked for their performance on the classification of the studied time-varying biomedical signals (ophthalmic arterial Doppler signals, internal carotid arterial Doppler signals and electroencephalogram signals). Decision making was performed in two stages: feature extraction by eigenvector methods and classification using the classifiers trained on the extracted features. The inputs of these expert systems composed of diverse or composite features were chosen according to the network structures. The present study was conducted with the purpose of answering the question of whether the expert system with diverse features (MME) or composite feature (CNN, ME) improve the capability of classification of the time-varying biomedical signals. The purpose was to determine an optimum classification scheme for the problem and also to infer clues about the extracted features. Our research demonstrated that the power levels of power spectral density (PSD) estimations obtained by the eigenvector methods are the valuable features which are representing the time-varying biomedical signals and the CNN, ME, and MME trained on these features achieved high classification accuracies.
EEG signal classification using wavelet feature extraction and a mixture of expert model
2007, Expert Systems with Applications
Mixture of experts (ME) is modular neural network architecture for supervised learning. A double-loop Expectation-Maximization (EM) algorithm has been introduced to the ME network structure for detection of epileptic seizure. The detection of epileptiform discharges in the EEG is an important component in the diagnosis of epilepsy. EEG signals were decomposed into the frequency sub-bands using discrete wavelet transform (DWT). Then these sub-band frequencies were used as an input to a ME network with two discrete outputs: normal and epileptic. In order to improve accuracy, the outputs of expert networks were combined according to a set of local weights called the “gating function”. The invariant transformations of the ME probability density functions include the permutations of the expert labels and the translations of the parameters in the gating functions. The performance of the proposed model was evaluated in terms of classification accuracies and the results confirmed that the proposed ME network structure has some potential in detecting epileptic seizures. The ME network structure achieved accuracy rates which were higher than that of the stand-alone neural network model.

View all citing articles on Scopus

¹: Paul Mangiameli is a Professor of Management Science and Information Systems in the College of Business Administration of the University of Rhode Island. His research interests are in neural network applications, process control, and quality management. He has published extensively in such journals as Decision Sciences, Journal of Operations Management, European Journal of Operations Research, Omega – The International Journal of Management Science, Managerial and Decision Economics, International Review of Economics and Finance, and Interfaces.

²: David West is an Associate Professor of Decision Sciences at East Carolina University in Greenville, North Carolina. He received his Ph.D. in Business Administration from the University of Rhode Island. His research interests include the application of neural network technology to such areas as classification decisions, manufacturing process control, and group clustering. He has published in the European Journal of Operations Research and Omega – The International Journal of Management Science.

View full text