Using genetic algorithm to select the presentation order of training patterns that improves simplified fuzzy ARTMAP classification performance

doi:10.1016/j.asoc.2008.03.003

Applied Soft Computing

Volume 9, Issue 1, January 2009, Pages 100-106

https://doi.org/10.1016/j.asoc.2008.03.003 Get rights and content

Abstract

The presentation order of training patterns to a simplified fuzzy ARTMAP (SFAM) neural network affects the classification performance. The common method to solve this problem is to use several simulations with training patterns presented in random order, where voting strategy is used to compute the final performance. Recently, an ordering method based on min–max clustering was introduced to select the presentation order of training patterns based on a single simulation. In this paper, another single simulation method based on genetic algorithm is proposed to obtain the presentation order of training patterns for improving the performance of SFAM. The proposed method is applied to a 40-class individual classification problem using visual evoked potential signals and three other datasets from UCI repository. The proposed method has the advantages of improved classification performance, smaller network size and lower training time compared to the random ordering and min–max methods. When compared to the random ordering method, the new ordering scheme has the additional advantage of requiring only a single simulation. As the proposed method is general, it can also be applied to a fuzzy ARTMAP neural network when it is used as a classifier.

Introduction

Fuzzy ARTMAP (FAM) [1] and simplified fuzzy ARTMAP (SFAM) [2] belong to a special class of neural networks (NNs) which are capable of incremental learning. In the fast learning mode, these networks have lower training time compared to other NN architectures like Multilayer Perceptron. SFAM and FAM have been used in numerous classification problems [1], [2], [3], [4], [5], [6]. FAM structure has three modules: Fuzzy ARTa, Fuzzy ARTb and Inter ART. SFAM differs from FAM in that its main purpose is for classification and as such, does not have Fuzzy ARTb, which becomes redundant for this purpose.

The presentation order of training patterns affects the classification (i.e. generalisation) performance of SFAM. To solve this problem, SFAM is trained several times using training patterns presented in random order (i.e. permutations of the training patterns) and then the predicted class of the test patterns are stored. Majority votes are used to determine the final class prediction for the test patterns [1]. It is also customary to state the average classification of test patterns from all the simulations in addition to the voting results. To solve the problem of having to run many simulations, a single simulation method based on min–max clustering was proposed [3]. For a c-class problem, the method works by ordering the c training patterns that are maximally distant in the training feature space. Next, for the rest of the patterns, the method orders training patterns that are minimally distant from these c patterns. Hence, it is known as min–max ordering.

In this paper, a method that uses genetic algorithm (GA) [7] to select the presentation order of training patterns is proposed. The method works by using the selection, mutation and inversion operators in GA to select the presentation order of training patterns that maximises the SFAM classification performance. Once the order is selected, only a single SFAM training simulation (similar to min–max ordering) will be required for classification of test patterns. The performance of the proposed technique is compared with training patterns ordered by min–max and random ordering using classifications of visual evoked potential (VEP) patterns to identify individuals [8]. In addition, three data sets from UCI repository [9] are also used to measure the performances of these ordering methods. From this point onwards, these three methods will be denoted simply as GA method, random ordering method and min–max method. All the discussions in this paper on SFAM could be equally applied to FAM with the condition that FAM is used as a classifier.

Section snippets

Simplified fuzzy ARTMAP

Fig. 1 shows the architecture of SFAM. It consists of two modules (Fuzzy ART and Inter ART) that create stable recognition categories in response to sequence of input patterns. During supervised learning, Fuzzy ART receives a stream of input features representing the pattern that map to the output classes in the category layer. Inter ART module works by increasing the vigilance parameter (VP) of Fuzzy ART by a minimal amount to correct a predictive error in the output category layer. VP

Pattern ordering using GA method

The overall methodology could be divided into two separate phases. In the first phase, either GA or min–max (as comparison) was used to order the input training patterns. The second phase involved testing the performance of SFAM for all the ordering methods (i.e. our proposed method, min–max and random). This second phase was conducted to show the improvement in SFAM performances when trained by training patterns ordered by GA as compared to min–max and random ordering.

Initially, the available

Experimental study

An experimental study is conducted to show the superior performance of the GA method compared to the random ordering and min–max methods. For this purpose, the dataset used in an earlier work to identify individuals [8] was used. In addition, three datasets, namely, wine, glass and iris, from UCI repository [9] were also used.

Conclusion

This paper has proposed the use of GA to select the presentation order of training patterns for SFAM. The new method could also be applied to FAM. The performances of the proposed method have been compared with the performances of the random ordering with a voting strategy and the min–max method for solving an individual classification problem using VEP signals and three data sets from UCI repository. Though there are computational overheads for the proposed method, it will be only during the

Acknowledgment

We thank Prof. Henri Begleiter at the Neurodynamics Laboratory at the State University of New York Health Centre at Brooklyn, USA who generated the raw VEP data and Mr. Paul Conlon, of Sasco Hill Research, USA for sending us the database. A part of this work was funded by University of Essex Research Promotion Fund (DDQP40).

References (10)

P. Raveendran et al.
Fuzzy ARTMAP classification of invariant features derived using angle of rotation from a neural network
Inf. Sci.: Int. J.
(2000)
G.A. Carpenter et al.
Fuzzy ARTMAP: a neural network architecture for incremental supervised learning of analog multidimensional maps
IEEE Trans. Neural Netw.
(1992)
T. Kasuba
Simplified fuzzy ARTMAP
AI Expert
(1993)
I. Dagher et al.
An ordering algorithm for pattern presentation in Fuzzy ARTMAP that tends to improve generalization performance
IEEE Trans. Neural Netw.
(1999)
R. Palaniappan et al.
A new brain-computer interface design using fuzzy ARTMAP
IEEE Trans. Neural Syst. Rehabil. Eng.
(2002)

There are more references available in the full text version of this article.

Cited by (45)

A survey of adaptive resonance theory neural network models for engineering applications
2019, Neural Networks
Citation Excerpt :
Many approaches have been developed to mitigate such ordering effects, and they mostly consist of suitable pre- and post-processing strategies (cf. Brito da Silva and Wunsch II (2018) and the references cited within). Particularly, for supervised ART models, these strategies include Max–Min clustering (Tou & Gonzalez, 1974) in Dagher, Georgiopoulos, Heileman, and Bebis (1998) and Dagher et al. (1999); class-by-class presentation in Sit, Mak, and Ng (2009), genetic algorithms (Eiben & Smith, 2015) in Baek, Lee, Lee, Lee, and Kim (2014) and Palaniappan and Eswaran (2009); uncorrelated feature-based ordering in Oong and Isa (2014); featural biasing in Carpenter and Gaddam (2010); and voting strategies in Amis and Carpenter (2007, 2010), Carpenter (2003), Carpenter and Markuzon (1998), Carpenter et al. (1992), Lim and Harrison (2000a, 2000b) and Williamson (1996). In regard to unsupervised ART models, examples of strategies are split, merge and delete operations in Lughofer (2008); merging methods in Brito da Silva et al. (2020) and Isawa et al. (2008a, 2008b, 2009); cluster validity index-based vigilance tests in Brito da Silva and Wunsch II (2017); learning topologies in Masuyama et al. (2019) and Tscherepanow (2010, 2012); and exploiting the ordering properties of visual assessment of cluster tendency (VAT) (Bezdek, 2017; Bezdek & Hathaway, 2002) in Brito da Silva and Wunsch II (2018).
This survey samples from the ever-growing family of adaptive resonance theory (ART) neural network models used to perform the three primary machine learning modalities, namely, unsupervised, supervised and reinforcement learning. It comprises a representative list from classic to contemporary ART models, thereby painting a general picture of the architectures developed by researchers over the past 30 years. The learning dynamics of these ART models are briefly described, and their distinctive characteristics such as code representation, long-term memory, and corresponding geometric interpretation are discussed. Useful engineering properties of ART (speed, configurability, explainability, parallelization and hardware implementation) are examined along with current challenges. Finally, a compilation of online software libraries is provided. It is expected that this overview will be helpful to new and seasoned ART researchers.
Improving the performance of inductive learning classifiers through the presentation order of the training patterns
2016, Expert Systems with Applications
Although the development of new supervised learning algorithms for machine learning techniques are mostly oriented to improve the predictive power or classification accuracy, the capacity to understand how the classification process is carried out is of great interest for many applications in business and industry. Inductive learning algorithms, like the Rules family, induce semantically interpretable classification rules in the form of if-then rules. Although the effectiveness of the Rules family has been studied thoroughly and new and improved versions are constantly been developed, one important drawback is the effect of the presentation order of the training patterns which has not been studied in depth previously. In this paper this issue is addressed, first by studying empirically the effect of random presentation orders in the number of rules and the generalization power of the resulting classifier. Then a presentation order method for the training examples is proposed which combines a clustering stage with a new density measure developed specifically for this problem. The results using benchmark datasets and a real application of wood defect classification show the effectiveness of the proposed method. Also, since the presentation order method is employed as a preprocessing stage, the simplicity of the Rules family is not affected but instead it enables the generation of fewer and more accurate rules, which can have a direct impact in the performance and usefulness of the Rules family in an expert system context.
A selective fuzzy ARTMAP ensemble and its application to the fault diagnosis of rolling element bearing
2016, Neurocomputing
Citation Excerpt :
But, in reality, when a single FAM is applied to the classification fields, the classification accuracy is not much satisfactory due to the limitation of FAM, such as the effect of the order of sample presentation for the off-line mode of training [15], overlapping effect extended from node expansion [14,16] and etc. To solve the problem many researchers have proposed some preprocessing procedures, known as the ordering algorithms such as min-max clustering and genetic algorithm, to improve the classification accuracy of single FAM classifier[17,18], but these ordering methods can not consider the effect of the number of training samples, and their reliability of classification is not high. Afterwards, the ensemble of multiple neural network classifiers based on the decision fusion technique has been presented to overcome these limitations of individual FAM classifier and achieve higher accuracy and reliability [19,20].
A novel intelligent fault diagnosis method based on feature extraction methods, features selection using modified distance discriminant technique and selective ensemble of multiple fuzzy ARTMAP (FAM) classifiers is proposed in this paper. The method consists of three stages. Firstly, different features in multiple symptom domains, such as time-domain features, frequency-domain features, wavelet grey moments, wavelet packet energy spectrum and auto-regression model parameters, are extracted from the raw vibration signals. Secondly, with the modified distance discriminant technique five salient feature sets are selected from the five original feature sets in different symptom domains respectively. Finally, these optimal feature sets are input the selective ensemble of multiple FAM classifiers based on the correlation measure method and Bayesian belief method to identify different abnormal cases. The proposed method is applied to the fault diagnosis of rolling element bearings, the test result shows that the selective ensemble of four FAM classifiers can identify the different fault conditions accurately and has a better classification performance compared to the single FAM and ensemble of all FAM classifiers. Besides, the diagnosis performance of the selective ensemble is analyzed by the bootstrap method. All experiment results have demonstrated that the selective ensemble of FAM classifiers has the effectiveness, stability, generalization, reliability and robustness.
Artificial neural networks (Fuzzy ARTMAP) analysis of the data obtained with an electronic tongue applied to a ham-curing process with different salt formulations
2015, Applied Soft Computing Journal
This paper describes the determination of optimum values of the parameters of a Simplified Fuzzy ARTMAP neural network for monitoring dry-cured ham processing with different salt formulations to be implemented in a microcontroller device. The employed network must be set to the limited microcontroller memory but, at the same time, should achieve optimal performance to classify the samples obtained from this application.
Hams salted with different salt formulations (100% NaCl; 50% NaCl + 50% KCl and 55% NaCl + 25% KCl + 15% CaCl₂ + 5% MgCl₂) were checked at four processing times, from post-salting to the end of their processing (2, 4, 8 and 12 months).
Measurements were taken with a potentiometric electronic tongue system formed by metal electrodes of different materials that worked as nonspecific sensors. This study aimed to discriminate ham samples according to two parameters: processing time and salt formulation.
The results were analyzed with an artificial neural network of the Simplified Fuzzy ARTMAP (SFAM) type. During the training and validation process of the neural network, optimum values of the control parameters of the neural network were determined for easy implementation in a microcontroller, and to simultaneously achieve maximum sample discrimination. The test process was run in a PIC18F450 microcontroller, where the SFAM algorithm was implemented with the optimal parameters. A data analysis with the optimized neural network was achieved, and samples were perfectly discriminated according to processing time (100%). It is more difficult to discriminate all samples according to salt formulation type, but it is easy to achieve salt type discrimination within each processing block time. Thus, we conclude that the processing time effect dominates salt formulation effects.
Affect classification using genetic-optimized ensembles of fuzzy ARTMAPs
2015, Applied Soft Computing
Training neural networks in distinguishing different emotions from physiological signals frequently involves fuzzy definitions of each affective state. In addition, manual design of classification tasks often uses sub-optimum classifier parameter settings, leading to average classification performance. In this study, an attempt to create a framework for multi-layered optimization of an ensemble of classifiers to maximize the system's ability to learn and classify affect, and to minimize human involvement in setting optimum parameters for the classification system is proposed. Using fuzzy adaptive resonance theory mapping (ARTMAP) as the classifier template, genetic algorithms (GAs) were employed to perform exhaustive search for the best combination of parameter settings for individual classifier performance. Speciation was implemented using subset selection of classification data attributes, as well as using an island model genetic algorithms method. Subsequently, the generated population of optimum classifier configurations was used as candidates to form an ensemble of classifiers. Another set of GAs were used to search for the combination of classifiers that would result in the best classification ensemble accuracy. The proposed methodology was tested using two affective data sets and was able to produce relatively small ensembles of fuzzy ARTMAPs with excellent affect recognition accuracy.
An efficient genetic selection of the presentation order in simplified fuzzy ARTMAP patterns
2014, Applied Soft Computing Journal
Citation Excerpt :
Fig. 4 shows the average learning results of the ten runs in all of the databases. The dashed lines in the figure correspond to the results of the conventional method [7], while the solid lines correspond to the results of the proposed method. The vertical axis denotes the fitness value of (21), and the horizontal axis denotes the number of generations.
Simplified fuzzy ARTMAP (SFAM) is used in numerous classification problems due to its high discriminant power and low training time. However, the performance of SFAM is affected by the presentation order of the training patterns. The genetic algorithm (GA) can be considered as a solution to the problem because the selection of the training pattern order is a complicated combinatorial problem in a large search space. In this paper, a new genetic ordering method for SFAM is proposed to improve the performance of the algorithm. Special genetic operators are employed in the genetic evolution. Compared to the conventional methods, the proposed SFAM demonstrates better classification performance since it can efficiently deliver the desirable properties of parents to their offspring. To demonstrate the performance of the proposed method, we perform experiments on various databases from the UCI repository.

View all citing articles on Scopus

View full text

Using genetic algorithm to select the presentation order of training patterns that improves simplified fuzzy ARTMAP classification performance

Abstract

Introduction

Section snippets

Simplified fuzzy ARTMAP

Pattern ordering using GA method

Experimental study

Conclusion

Acknowledgment

Inf. Sci.: Int. J.

Fuzzy ARTMAP: a neural network architecture for incremental supervised learning of analog multidimensional maps

IEEE Trans. Neural Netw.

Simplified fuzzy ARTMAP

AI Expert

An ordering algorithm for pattern presentation in Fuzzy ARTMAP that tends to improve generalization performance

IEEE Trans. Neural Netw.

A new brain-computer interface design using fuzzy ARTMAP

IEEE Trans. Neural Syst. Rehabil. Eng.