TPPFAM: Use of threshold and posterior probability for category reduction in fuzzy ARTMAP
Introduction
The fuzzy ARTMAP (FAM) is a supervised clustering algorithm which can be considered as one of the leading neural network (NN) algorithms for classification tasks [1]. It is based on Adaptive Resonance Theory (ART) which constitutes a model for artificial NN inspired by the studies of Grossberg and Carpenter [2]. These supervised learning systems have many merits, such as online real-time and incremental learning [3], the use of few training epochs to achieve reasonably accuracy [4], allowing learning new data without forgetting past data, and tacking the so-called “plasticity-stability dilemma” [5], and so on. The FAM and its modified versions have been widely used in many application fields, including robotic application scenarios [6], information fusion [7], data mining [8], image classification, genetic abnormality diagnosis [9], etc.
Though large [10] or noisy databases [11] can cause FAM generating too many categories, i.e., category proliferation, the problem is most serious when the network architecture is trained with data of overlapping classes [12]. When a similar input pattern lies in the overlapping region, it is difficult to make a more reliable and accurate prediction. Therefore, many categories are created to map onto this overlapping region, leading to heavy computational burden, more memory requirement and possible degradation of the classification accuracy [13], [14].
In order to cope with the category proliferation problem in FAM, many researchers have proposed various schemes which can be divided into two major classes—the new FAM-based algorithms and the modified versions of FAM. Nevertheless, the borderline between two classes is thin and sometimes we may have crossed it unintentially [15]. The former includes Boosted ARTMAP [16], Gaussian ARTMAP [11], ARTMAP-IC [17], and Bayesian ARTMAP [18]. The latter includes Distributed ARTMAP [19], rule pruning [20], [21], Koufakou et al. use cross-validation techniques to reduce category proliferation [14], and Sit et al. adopt a number of measures to reduce category proliferation [22].
In this paper, in order to reduce the number of categories and improve the classification accuracy, a threshold adjustment parameter is introduced in both training and testing algorithms, and a new algorithm is proposed, called TTPFAM. The proposed algorithm makes some modifications of the FAM algorithm in the following aspects: (1) During training, in order to reduce category proliferation, a threshold filtering is performed before creating a new category. Then the corresponding supervised class posterior probabilities of the given categories filtered are estimated, the category with the maximum posterior probability will be chosen as the winner. (2) During testing, in order to improve classification accuracy, the system combines the category information distributed by a dynamic Q-max rule with the estimated posterior probability matrix to get the most reliable and accurate class for each testing pattern. A comparison of TTPFAM with FAM, MT-, ARTMAP-IC and the Algorithm 1 in [22] is made on artificial and real data, and the results show that the TTPFAM algorithm is superior to the others in terms of the classification accuracy, the number of categories, and the sensitivity to statistical overlapping.
The remainder of the paper is organized as follows. Section 2 briefly summarizes the principles and dynamics of FAM, and discusses the category proliferation problem. The proposed algorithm is described in detail in Section 3, including the derivation of the threshold, the estimation of the posterior probability, and the description of the TTPFAM algorithm. In Section 4, the experiments and results analysis are presented. The conclusions are drawn in Section 5.
Section snippets
Fuzzy ARTMAP
The FAM is the most important neural network architecture derived from ART. It consists of two unsupervised fuzzy ART [23] modules, i.e., ARTa and ARTb, which cluster the patterns from the input space and the output space into categories, respectively. Two modules are linked by the map field which is used to ensure that each category from ARTa is associated with only one category from ARTb, i.e., a many-to-one mapping. However, the frequency of the associations between ARTa and ARTb categories
TTPFAM
As the description above, when an input pattern lies in the overlapping region, it is difficult to make one prediction for the input pattern. In FAM, if the activation values of two (or more) categories are the same and pass the vigilance test, the one with smaller index will be selected as the winner. Thus, it would be unfair to other categories which have the same activation value and pass the vigilance mapped to a class prediction accurately. Therefore, predicting only one category mapping
Experimental results
In the following sections, we evaluate the performance of TPPFAM in terms of the classification accuracy, the number of categories and the sensitivity to class overlapping. TPPFAM is tested on both artificial and real databases. In the paper, each database is split into 70% for training and 30% for testing. The overlapping degree of data directly affects the difficulty of the classification problem. Obviously, the problem of category proliferation in FAM becomes more and more serious as the
Conclusions
Overlapping classes present a difficulty in classification and cause category proliferation in FAM. In the paper, a new FAM-based neural architecture called TPPFAM has been proposed as a solution to the category proliferation problem. During training of TPPFAM, when an input pattern lies in the overlapping region, a threshold filtering mechanism is performed before creating a new category. Then according to the corresponding supervised class posterior probabilities of the given categories
Acknowledgments
This work was supported by the National Natural Science Foundation of China under Grant no. 60871074.
Yongquan Zhang was born in Gansu, China in 1985. He received B.S. (2007) degree in Computer Science and Technology, and M.S. (2010) degree in Computer Applications Technology from Lanzhou University of Technology, respectively.
Currently, he is pursuing a Ph.D. degree in the School of Electronic Engineering at Xidian University. His research interests include machine learning, signal processing, target tracking and data fusion.
References (35)
- et al.
ARTMAP: supervised real-time learning and classification of nonstationary data by a self-organizing neural network
Neural Networks
(1991) - et al.
ART-based fusion of multi-modal perception for robots
Neurocomputing
(2013) - et al.
Self-organizing information fusion and hierarchical knowledge discovery: a new framework using ARTMAP neural networks
Neural Networks
(2005) - et al.
ARTMAP neural networks for information fusion and data mining: map production and target recognition methodologies
(2003) Gaussian ARTMAP: A neural network for fast incremental learning of noisy multidimensional maps
Neural Networks
(1996)- et al.
A modified fuzzy ARTMAP architecture for the approximation of noisy mappings
Neural Networks
(1995) - et al.
Cross-validation in fuzzy ARTMAP for large databases
Neural Networks
(2001) - et al.
ARTMAP-IC and medical diagnosis: Instance counting and inconsistent cases
Neural Network
(1998) - et al.
Distributed ARTMAP: a neural network for fast distributed supervised learning
Neural Network
(1998) - et al.
Fast stable learning and categorization of analog patterns by an adaptive resonance system
Neural Networks
(1991)
Biased ART: a neural architecture that shifts attention toward previously disregarded features following an incorrect prediction
Neural Networks
Self-organizing ARTMAP rule discovery
Neural Networks
Analysis of hidden units in a layered network trained to classify sonar targets
Neural Networks
Fuzzy ARTMAP: a neural-network architecture for incremental supervised learning of analog multidimensional maps
IEEE Transactions on Neural Networks
The ART of adaptive pattern recognition by a self-organizing neural network
Computer
Adaptive pattern recognition and universal encoding II: feedback, expectation, olfaction, and illusions
Biological Cybernetics
Cited by (22)
A new approach to online training for the Fuzzy ARTMAP artificial neural network
2021, Applied Soft ComputingA novel fuzzy ARTMAP with area of influence
2021, NeurocomputingCitation Excerpt :Over the years, many authors have proposed strategies to attenuate the category proliferation in FAM networks. These strategies are based on different approaches, such as changing the FAM architecture [8–15], applying post-processing methods for pruning categories [16,17], or even replacing the hyper-rectangular shape of FAM categories with hyper-spheres, Gaussians, polytopes, and so on [18–23]. In this work, our aim is to approach the category proliferation problem while maintaining the FAM categories geometry and its online and incremental learning capabilities – these properties are directly linked to the works that change the FAM architecture.
Feature selection based on brain storm optimization for data classification
2019, Applied Soft Computing JournalAn ellipse extended target CBMeMBer filter using gamma and box-particle implementation
2018, Signal ProcessingOnARTMAP: A Fuzzy ARTMAP-based Architecture
2018, Neural Networks
Yongquan Zhang was born in Gansu, China in 1985. He received B.S. (2007) degree in Computer Science and Technology, and M.S. (2010) degree in Computer Applications Technology from Lanzhou University of Technology, respectively.
Currently, he is pursuing a Ph.D. degree in the School of Electronic Engineering at Xidian University. His research interests include machine learning, signal processing, target tracking and data fusion.
Hong-bing Ji was born in Shaanxi, China in 1963. He graduated from Northern West Telecommunications Engineering College (the predecessor of Xidian University) and earned B.S. degree in Radar Engineering in 1983. He received M.S. (1989) degree in Circuit, Signals and Systems and Ph.D. (1999) degree in Signal and Information Processing from Xidian University, respectively.
After graduation in 1989, he has been with the School of Electronic Engineering at Xidian University, a lecturer from 1990 to1995, an associate professor from 1995 to 2000, a professor from 2000. From 1996 to 2002 he served as a vice dean of School of Electronic Engineering. From 2002, he was the executive dean of graduate school of Xidian University, also a vice chairman of the Academic Degree Evaluation Committee. His primary areas of research have been radar signal processing, automatic targets recognition, multisensor information fusion and target tracking.
Prof. Ji is a senior member of the Institute of Electrical and Electronics Engineers (IEEE), a member of IEEE Signal Processing Society and a member of IEEE Aerospace & Electronic Systems Society.
Wenbo Zhang was born in Shaanxi, China in 1985. He received B.S. (2005) degree and M.S. (2009) degree in the School of Telecommunications Engineering at Xidian University.
Currently, he is pursuing a Ph.D. degree in the School of Electronic Engineering at Xidian University. His research interests include pattern recognition, support vector machine, extreme learning machine.